initial upload
This commit is contained in:
@@ -0,0 +1,8 @@
|
|||||||
|
__pycache__/
|
||||||
|
*.pyc
|
||||||
|
.venv/
|
||||||
|
venv/
|
||||||
|
.pytest_cache/
|
||||||
|
*.db
|
||||||
|
*.db-wal
|
||||||
|
*.db-shm
|
||||||
@@ -1,3 +1,101 @@
|
|||||||
# Backup-Dashboard
|
# Backup-Dashboard
|
||||||
|
|
||||||
Proof of concept backup dashboard, intended to display backup jobs from multiple sources
|
Proof of concept backup dashboard, intended to display backup jobs from multiple sources.
|
||||||
|
|
||||||
|
```
|
||||||
|
collectors ──POST /api/v1/ingest (API key)──▶ ┌──────────────┐
|
||||||
|
│ FastAPI + │
|
||||||
|
browser ──nginx (basic auth, TLS)──▶ /api/* ─▶│ SQLite │
|
||||||
|
└─ static files: public/ └──────────────┘
|
||||||
|
```
|
||||||
|
|
||||||
|
- `public/`: static frontend (no build step). Add `?mock=1` to the URL to use built-in sample data.
|
||||||
|
- `backend/`: FastAPI API backed by a single SQLite file.
|
||||||
|
- `deploy/`: nginx site and systemd unit.
|
||||||
|
|
||||||
|
## Data model
|
||||||
|
|
||||||
|
- **Tenant** (company) → **Job** → **Run**.
|
||||||
|
- A job is keyed by `(tenant, source_system, job.external_id)`, so renaming a job at the source updates it instead of creating a duplicate.
|
||||||
|
- A run has `status` (`running` | `success` | `warning` | `failed`), `started_at` and `finished_at`.
|
||||||
|
The dashboard's **Last Run** is the most recent `finished_at`. While a job is running it shows
|
||||||
|
**Running**, and Last Run still shows when the previous run finished.
|
||||||
|
- Runs older than `BACKUP_DASH_RETENTION_DAYS` (default 90, minimum 30) are purged every 6 hours.
|
||||||
|
|
||||||
|
## Reporting results (collectors)
|
||||||
|
|
||||||
|
Each collector (a script on a Veeam server, a webhook relay, etc.) gets its own API key:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd backend && python manage.py create-key veeam-acme
|
||||||
|
```
|
||||||
|
|
||||||
|
Then it POSTs one report per run. Send `running` when a run starts (optional) and the final status
|
||||||
|
when it ends. Reports with the same `started_at` (or `run.external_id`) update the same run, and
|
||||||
|
retries are safe.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST https://backups.example.com/api/v1/ingest \
|
||||||
|
-H "Authorization: Bearer bdk_..." -H "Content-Type: application/json" \
|
||||||
|
-d '{
|
||||||
|
"tenant": "Acme Corp",
|
||||||
|
"source_system": "Veeam B&R",
|
||||||
|
"job": { "external_id": "4f1c…", "name": "File Server – Nightly", "next_run_at": "2026-10-05T22:00:00-04:00" },
|
||||||
|
"run": {
|
||||||
|
"status": "success",
|
||||||
|
"started_at": "2026-10-04T22:00:05-04:00",
|
||||||
|
"finished_at": "2026-10-04T22:47:12-04:00",
|
||||||
|
"message": "Job completed successfully.",
|
||||||
|
"size_bytes": 128849018880,
|
||||||
|
"details": { "repository": "REPO-01" }
|
||||||
|
}
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
|
- All timestamps must include a timezone offset (`Get-Date -Format o` in PowerShell does this).
|
||||||
|
- `job.next_run_at` controls the **Overdue** flag. If you omit it, the stored value is kept. If you send `null`, it's cleared.
|
||||||
|
- `run` can be omitted to only register or update a job.
|
||||||
|
- Unknown fields are rejected (HTTP 422), so typos in collector scripts fail loudly.
|
||||||
|
|
||||||
|
## API (read)
|
||||||
|
|
||||||
|
| Endpoint | Returns |
|
||||||
|
|---|---|
|
||||||
|
| `GET /api/v1/jobs` | All jobs with current status, `running_since`, `last_run_at`, `next_run_at` |
|
||||||
|
| `GET /api/v1/jobs/{id}` | One job, plus the last finished run's duration, size, message and details |
|
||||||
|
| `GET /api/v1/jobs/{id}/runs?days=30` | Run history (no messages), newest first |
|
||||||
|
| `GET /api/v1/runs/{id}` | One run including message and details |
|
||||||
|
| `GET /api/v1/health` | `{"status": "ok"}` |
|
||||||
|
|
||||||
|
The OpenAPI schema is at `/api/v1/openapi.json`.
|
||||||
|
|
||||||
|
## Authentication
|
||||||
|
|
||||||
|
- **Dashboard users:** nginx basic auth over HTTPS (`deploy/nginx.conf`). Every user can see every tenant.
|
||||||
|
*Planned:* Microsoft 365 sign-in through [oauth2-proxy](https://oauth2-proxy.github.io/oauth2-proxy/)
|
||||||
|
with the Entra ID provider and nginx `auth_request`. That's an nginx-only change. The frontend
|
||||||
|
already tells the user to reload when a session expires (HTTP 401).
|
||||||
|
- **Collectors:** per-collector bearer keys, checked by the app. Only a SHA-256 hash is stored.
|
||||||
|
To revoke one, run `python manage.py revoke-key NAME`.
|
||||||
|
|
||||||
|
## Development
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd backend
|
||||||
|
python -m venv .venv && . .venv/bin/activate
|
||||||
|
pip install -r requirements.txt -r requirements-dev.txt
|
||||||
|
pytest
|
||||||
|
python manage.py seed-demo # 30 days of fake history
|
||||||
|
BACKUP_DASH_SERVE_STATIC=1 uvicorn --factory app.main:create_app --reload
|
||||||
|
# → http://localhost:8000
|
||||||
|
```
|
||||||
|
|
||||||
|
## Deployment (Linux, nginx)
|
||||||
|
|
||||||
|
1. Copy the repo to `/opt/backup-dashboard`, create a venv at `/opt/backup-dashboard/venv`, and run `pip install -r backend/requirements.txt`.
|
||||||
|
2. Create a `backupdash` system user, then install `deploy/backup-dashboard-api.service` into `/etc/systemd/system/` and enable it.
|
||||||
|
3. Copy `public/` to `/var/www/backup-dashboard/public`, install `deploy/nginx.conf`, get a certificate with certbot, and create users with `htpasswd -B`.
|
||||||
|
4. Run the `manage.py` commands as the service user with `BACKUP_DASH_DB=/var/lib/backup-dashboard/backup-dashboard.db`.
|
||||||
|
|
||||||
|
Back up the SQLite file with `sqlite3 backup-dashboard.db ".backup copy.db"`. Don't use a plain `cp`
|
||||||
|
while the service is running.
|
||||||
|
|||||||
@@ -0,0 +1,116 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Admin CLI. Uses the same BACKUP_DASH_DB env var as the API.
|
||||||
|
|
||||||
|
python manage.py create-key veeam-acme # prints the key once
|
||||||
|
python manage.py list-keys
|
||||||
|
python manage.py revoke-key veeam-acme
|
||||||
|
python manage.py list-jobs
|
||||||
|
python manage.py delete-job 12 # e.g. a decommissioned job stuck as Overdue
|
||||||
|
python manage.py seed-demo # 30 days of fake history for trying the UI
|
||||||
|
"""
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import random
|
||||||
|
import sys
|
||||||
|
from datetime import timedelta
|
||||||
|
|
||||||
|
from app import db, repo
|
||||||
|
from app.auth import generate_key, hash_key
|
||||||
|
from app.models import IngestIn
|
||||||
|
from app.settings import load_settings
|
||||||
|
from app.timeutil import to_db, utcnow
|
||||||
|
|
||||||
|
|
||||||
|
def create_key(conn, args):
|
||||||
|
key = generate_key()
|
||||||
|
try:
|
||||||
|
conn.execute(
|
||||||
|
"INSERT INTO api_keys (name, key_hash, created_at) VALUES (?, ?, ?)",
|
||||||
|
(args.name, hash_key(key), to_db(utcnow())),
|
||||||
|
)
|
||||||
|
except db.sqlite3.IntegrityError:
|
||||||
|
sys.exit(f"A key named {args.name!r} already exists")
|
||||||
|
print(f"API key for {args.name!r} (shown once, store it in the collector's config):\n{key}")
|
||||||
|
|
||||||
|
|
||||||
|
def list_keys(conn, args):
|
||||||
|
for r in conn.execute("SELECT name, created_at, last_used_at, revoked_at FROM api_keys ORDER BY name"):
|
||||||
|
state = f"revoked {r['revoked_at']}" if r["revoked_at"] else "active"
|
||||||
|
print(f"{r['name']:<30} {state:<30} created {r['created_at']} last used {r['last_used_at'] or 'never'}")
|
||||||
|
|
||||||
|
|
||||||
|
def revoke_key(conn, args):
|
||||||
|
cur = conn.execute(
|
||||||
|
"UPDATE api_keys SET revoked_at = ? WHERE name = ? AND revoked_at IS NULL", (to_db(utcnow()), args.name)
|
||||||
|
)
|
||||||
|
sys.exit(0 if cur.rowcount else f"No active key named {args.name!r}")
|
||||||
|
|
||||||
|
|
||||||
|
def list_jobs(conn, args):
|
||||||
|
for j in repo.list_jobs(conn):
|
||||||
|
print(f"{j['id']:>5} {j['company_name']:<28} {j['job_name']:<40} {j['status'] or '-':<8} last {j['last_run_at'] or 'never'}")
|
||||||
|
|
||||||
|
|
||||||
|
def delete_job(conn, args):
|
||||||
|
cur = conn.execute("DELETE FROM jobs WHERE id = ?", (args.job_id,))
|
||||||
|
sys.exit(0 if cur.rowcount else f"No job with id {args.job_id}")
|
||||||
|
|
||||||
|
|
||||||
|
DEMO = [
|
||||||
|
("Acme Corp", "Veeam B&R", "File Server - Nightly", 24),
|
||||||
|
("Acme Corp", "Veeam B&R", "SQL01 - Transaction Logs", 4),
|
||||||
|
("Bluewater Dental", "Datto", "Workstations - Daily", 24),
|
||||||
|
("Bluewater Dental", "Datto", "Imaging Server - Hourly", 1),
|
||||||
|
("Northfield Logistics", "Acronis", "Exchange - Nightly", 24),
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def seed_demo(conn, args):
|
||||||
|
rng = random.Random(42)
|
||||||
|
now = utcnow()
|
||||||
|
for tenant, source, name, every_h in DEMO:
|
||||||
|
ext = name.lower().replace(" ", "-")
|
||||||
|
# Offset so runs don't all line up exactly on the current time
|
||||||
|
start = now - timedelta(days=30) + timedelta(minutes=rng.randint(5, every_h * 60 - 5))
|
||||||
|
while start < now:
|
||||||
|
dur = timedelta(minutes=rng.randint(2, 50 if every_h > 1 else 15))
|
||||||
|
end = start + dur
|
||||||
|
status = "running" if end > now else rng.choices(["success", "warning", "failed"], [90, 7, 3])[0]
|
||||||
|
run = {"status": status, "started_at": start.isoformat()}
|
||||||
|
if status != "running":
|
||||||
|
run |= {"finished_at": end.isoformat(), "size_bytes": rng.randint(10**8, 10**11),
|
||||||
|
"message": {"success": "Job completed successfully.",
|
||||||
|
"warning": "Completed with warnings: 1 file skipped (locked).",
|
||||||
|
"failed": "Error: target repository unreachable."}[status]}
|
||||||
|
repo.ingest(conn, IngestIn.model_validate({
|
||||||
|
"tenant": tenant, "source_system": source,
|
||||||
|
"job": {"external_id": ext, "name": name,
|
||||||
|
"next_run_at": (start + timedelta(hours=every_h)).isoformat()},
|
||||||
|
"run": run,
|
||||||
|
}))
|
||||||
|
start += timedelta(hours=every_h)
|
||||||
|
print(f"Seeded {len(DEMO)} demo jobs")
|
||||||
|
|
||||||
|
|
||||||
|
def main():
|
||||||
|
p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||||
|
sub = p.add_subparsers(required=True)
|
||||||
|
s = sub.add_parser("create-key"); s.add_argument("name"); s.set_defaults(fn=create_key)
|
||||||
|
s = sub.add_parser("list-keys"); s.set_defaults(fn=list_keys)
|
||||||
|
s = sub.add_parser("revoke-key"); s.add_argument("name"); s.set_defaults(fn=revoke_key)
|
||||||
|
s = sub.add_parser("list-jobs"); s.set_defaults(fn=list_jobs)
|
||||||
|
s = sub.add_parser("delete-job"); s.add_argument("job_id", type=int); s.set_defaults(fn=delete_job)
|
||||||
|
s = sub.add_parser("seed-demo"); s.set_defaults(fn=seed_demo)
|
||||||
|
args = p.parse_args()
|
||||||
|
|
||||||
|
settings = load_settings()
|
||||||
|
db.init_db(settings.db_path)
|
||||||
|
conn = db.connect(settings.db_path)
|
||||||
|
try:
|
||||||
|
args.fn(conn, args)
|
||||||
|
finally:
|
||||||
|
conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@@ -0,0 +1,2 @@
|
|||||||
|
httpx
|
||||||
|
pytest
|
||||||
@@ -0,0 +1,2 @@
|
|||||||
|
fastapi>=0.115
|
||||||
|
uvicorn[standard]>=0.30
|
||||||
Reference in New Issue
Block a user