Add backend api #1

Manually merged
kyle merged 5 commits from add-backend-api into main 2026-10-07 15:00:07 -06:00
32 changed files with 3004 additions and 1 deletions
+8
View File
@@ -0,0 +1,8 @@
__pycache__/
*.pyc
.venv/
venv/
.pytest_cache/
*.db
*.db-wal
*.db-shm
+106 -1
View File
@@ -1,3 +1,108 @@
# Backup-Dashboard # Backup-Dashboard
Proof of concept backup dashboard, intended to display backup jobs from multiple sources Proof of concept backup dashboard, intended to display backup jobs from multiple sources.
```
collectors ──POST /api/v1/ingest (API key)──▶ ┌──────────────┐
│ FastAPI + │
browser ──nginx (basic auth, TLS)──▶ /api/* ─▶│ SQLite │
└─ static files: public/ └──────────────┘
```
- `public/`: static frontend (no build step). Add `?mock=1` to the URL to use built-in sample data.
- `backend/`: FastAPI API backed by a single SQLite file.
- `deploy/`: nginx site and systemd unit.
- `collectors/`: scripts that report results from backup products ([Veeam B&R](collectors/veeam/README.md)).
## Data model
- **Tenant** (company) → **Job** → **Run**.
- A job is keyed by `(tenant, source_system, job.external_id)`, so renaming a job at the source updates it instead of creating a duplicate.
- A run has `status` (`running` | `success` | `warning` | `failed`), `started_at` and `finished_at`.
The dashboard's **Last Run** is the most recent `finished_at`. While a job is running it shows
**Running**, and Last Run still shows when the previous run finished.
- Runs older than `BACKUP_DASH_RETENTION_DAYS` (default 90, minimum 30) are purged every 6 hours.
## Reporting results (collectors)
Each collector (a script on a Veeam server, a webhook relay, etc.) gets its own API key:
```bash
cd backend && python manage.py create-key veeam-acme
```
Then it POSTs one report per run. Send `running` when a run starts (optional) and the final status
when it ends. Reports with the same `started_at` (or `run.external_id`) update the same run, and
retries are safe.
```bash
curl -X POST https://backups.example.com/api/v1/ingest \
-H "Authorization: Bearer bdk_..." -H "Content-Type: application/json" \
-d '{
"tenant": "Acme Corp",
"source_system": "Veeam B&R",
"job": { "external_id": "4f1c…", "name": "File Server – Nightly", "next_run_at": "2026-10-05T22:00:00-04:00" },
"run": {
"status": "success",
"started_at": "2026-10-04T22:00:05-04:00",
"finished_at": "2026-10-04T22:47:12-04:00",
"message": "Job completed successfully.",
"size_bytes": 128849018880,
"details": { "repository": "REPO-01" }
}
}'
```
- All timestamps must include a timezone offset (`Get-Date -Format o` in PowerShell does this).
- `job.next_run_at` controls the **Overdue** flag. If you omit it, the stored value is kept. If you send `null`, it's cleared.
- `run` can be omitted to only register or update a job.
- Unknown fields are rejected (HTTP 422), so typos in collector scripts fail loudly.
## API (read)
| Endpoint | Returns |
|---|---|
| `GET /api/v1/jobs` | All jobs with current status, `running_since`, `last_run_at`, `next_run_at` |
| `GET /api/v1/jobs/{id}` | One job, plus the last finished run's duration, size, message and details |
| `GET /api/v1/jobs/{id}/runs?days=30` | Run history (no messages), newest first |
| `GET /api/v1/runs/{id}` | One run including message and details |
| `GET /api/v1/health` | `{"status": "ok"}` |
The OpenAPI schema is at `/api/v1/openapi.json`.
## Authentication
- **Dashboard users:** nginx basic auth over HTTPS (`deploy/nginx.conf`). Every user can see every tenant.
*Planned:* Microsoft 365 sign-in through [oauth2-proxy](https://oauth2-proxy.github.io/oauth2-proxy/)
with the Entra ID provider and nginx `auth_request`. That's an nginx-only change. The frontend
already tells the user to reload when a session expires (HTTP 401).
- **Collectors:** per-collector bearer keys, checked by the app. Only a SHA-256 hash is stored.
To revoke one, run `python manage.py revoke-key NAME`.
## Development
```bash
cd backend
python -m venv .venv && . .venv/bin/activate
pip install -r requirements.txt -r requirements-dev.txt
pytest
python manage.py --create seed-demo # 30 days of fake history in ./backup-dashboard.db
BACKUP_DASH_SERVE_STATIC=1 uvicorn --factory app.main:create_app --reload
# → http://localhost:8000
```
## Deployment (Linux, nginx)
1. Copy the repo to `/opt/backup-dashboard`, create a venv at `/opt/backup-dashboard/venv`, and run `pip install -r backend/requirements.txt`.
2. Copy `deploy/backup-dashboard.env` to `/etc/backup-dashboard/backup-dashboard.env`. Both the service and `manage.py` read their settings from it.
3. Create a `backupdash` system user, then install `deploy/backup-dashboard-api.service` into `/etc/systemd/system/` and enable it. The database is created on first start.
4. Copy `public/` to `/var/www/backup-dashboard/public`, install `deploy/nginx.conf`, get a certificate with certbot, and create users with `htpasswd -B`.
5. Run `manage.py` as the service user. It finds the database through the env file:
```bash
cd /opt/backup-dashboard/backend
sudo -u backupdash /opt/backup-dashboard/venv/bin/python manage.py create-key veeam-acme
```
It prints the database path it's using and won't create a new database unless you pass `--create`.
Back up the SQLite file with `sqlite3 backup-dashboard.db ".backup copy.db"`. Don't use a plain `cp`
while the service is running.
View File
+42
View File
@@ -0,0 +1,42 @@
"""Collector (machine) authentication.
Dashboard *users* are authenticated by nginx in front of the app (basic auth today,
oauth2-proxy + Microsoft Entra ID later), so nothing here deals with people.
"""
import hashlib
import secrets
import sqlite3
from fastapi import Depends, Header, HTTPException
from .db import get_db
from .timeutil import to_db, utcnow
KEY_PREFIX = "bdk_"
def generate_key() -> str:
return KEY_PREFIX + secrets.token_urlsafe(32)
def hash_key(key: str) -> str:
# Keys are 256-bit random, so a fast unsalted hash is fine (no dictionary to attack).
return hashlib.sha256(key.encode()).hexdigest()
def require_ingest_key(
authorization: str | None = Header(None),
conn: sqlite3.Connection = Depends(get_db),
) -> str:
scheme, _, token = (authorization or "").partition(" ")
token = token.strip()
if scheme.lower() != "bearer" or not token:
raise HTTPException(401, "Missing bearer token", headers={"WWW-Authenticate": "Bearer"})
row = conn.execute(
"SELECT id, name FROM api_keys WHERE key_hash = ? AND revoked_at IS NULL", (hash_key(token),)
).fetchone()
if row is None:
raise HTTPException(401, "Invalid or revoked API key", headers={"WWW-Authenticate": "Bearer"})
conn.execute("UPDATE api_keys SET last_used_at = ? WHERE id = ?", (to_db(utcnow()), row["id"]))
return row["name"]
+99
View File
@@ -0,0 +1,99 @@
import sqlite3
from collections.abc import Iterator
from contextlib import contextmanager
from pathlib import Path
from fastapi import Request
SCHEMA_VERSION = 1
SCHEMA = """
CREATE TABLE IF NOT EXISTS tenants (
id INTEGER PRIMARY KEY,
name TEXT NOT NULL UNIQUE COLLATE NOCASE
);
-- A job is identified by (tenant, source system, the source's own job id), so two
-- products can't collide and renaming a job in Veeam doesn't create a duplicate.
CREATE TABLE IF NOT EXISTS jobs (
id INTEGER PRIMARY KEY,
tenant_id INTEGER NOT NULL REFERENCES tenants(id) ON DELETE CASCADE,
source_system TEXT NOT NULL,
external_id TEXT NOT NULL,
name TEXT NOT NULL,
next_run_at TEXT,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
UNIQUE (tenant_id, source_system, external_id)
);
CREATE TABLE IF NOT EXISTS runs (
id INTEGER PRIMARY KEY,
job_id INTEGER NOT NULL REFERENCES jobs(id) ON DELETE CASCADE,
external_id TEXT NOT NULL,
status TEXT NOT NULL CHECK (status IN ('running', 'success', 'warning', 'failed')),
started_at TEXT NOT NULL,
finished_at TEXT,
message TEXT NOT NULL DEFAULT '',
size_bytes INTEGER,
details TEXT NOT NULL DEFAULT '{}',
received_at TEXT NOT NULL,
CHECK ((status = 'running') = (finished_at IS NULL)),
UNIQUE (job_id, external_id)
);
CREATE INDEX IF NOT EXISTS runs_job_started ON runs (job_id, started_at);
CREATE INDEX IF NOT EXISTS runs_job_finished ON runs (job_id, finished_at);
-- Keys used by collectors to push results. Only the SHA-256 is stored.
CREATE TABLE IF NOT EXISTS api_keys (
id INTEGER PRIMARY KEY,
name TEXT NOT NULL UNIQUE,
key_hash TEXT NOT NULL UNIQUE,
created_at TEXT NOT NULL,
last_used_at TEXT,
revoked_at TEXT
);
"""
def connect(path: Path | str) -> sqlite3.Connection:
# Autocommit mode; writes that must be atomic go through transaction().
# check_same_thread=False: FastAPI may run a dependency and its endpoint on different
# threadpool threads, but a connection is only ever used by one request at a time.
conn = sqlite3.connect(path, timeout=10, isolation_level=None, check_same_thread=False)
conn.row_factory = sqlite3.Row
conn.execute("PRAGMA foreign_keys = ON")
conn.execute("PRAGMA busy_timeout = 10000")
return conn
def init_db(path: Path | str) -> None:
conn = connect(path)
try:
conn.execute("PRAGMA journal_mode = WAL")
version = conn.execute("PRAGMA user_version").fetchone()[0]
if version > SCHEMA_VERSION:
raise RuntimeError(f"Database schema v{version} is newer than this app (v{SCHEMA_VERSION})")
conn.executescript(SCHEMA)
conn.execute(f"PRAGMA user_version = {SCHEMA_VERSION}")
finally:
conn.close()
@contextmanager
def transaction(conn: sqlite3.Connection) -> Iterator[None]:
conn.execute("BEGIN IMMEDIATE")
try:
yield
except BaseException:
conn.execute("ROLLBACK")
raise
conn.execute("COMMIT")
def get_db(request: Request) -> Iterator[sqlite3.Connection]:
conn = connect(request.app.state.settings.db_path)
try:
yield conn
finally:
conn.close()
+118
View File
@@ -0,0 +1,118 @@
import asyncio
import logging
import sqlite3
from contextlib import asynccontextmanager
from datetime import timedelta
from pathlib import Path
from fastapi import APIRouter, Depends, FastAPI, HTTPException, Query, Request
from . import db, repo
from .auth import require_ingest_key
from .models import IngestIn
from .settings import HISTORY_DAYS, Settings, load_settings
from .timeutil import to_db, utcnow
log = logging.getLogger("backup_dashboard")
PURGE_INTERVAL_SECONDS = 6 * 3600
router = APIRouter(prefix="/api/v1")
@router.get("/health")
def health(conn: sqlite3.Connection = Depends(db.get_db)):
conn.execute("SELECT 1")
return {"status": "ok"}
@router.get("/jobs")
def list_jobs(conn: sqlite3.Connection = Depends(db.get_db)):
return {"generated_at": to_db(utcnow()), "jobs": repo.list_jobs(conn)}
@router.get("/jobs/{job_id}")
def get_job(job_id: int, conn: sqlite3.Connection = Depends(db.get_db)):
job = repo.get_job(conn, job_id)
if job is None:
raise HTTPException(404, "Job not found")
return job
@router.get("/jobs/{job_id}/runs")
def list_runs(
job_id: int,
days: int = Query(HISTORY_DAYS, ge=1, le=HISTORY_DAYS),
conn: sqlite3.Connection = Depends(db.get_db),
):
if conn.execute("SELECT 1 FROM jobs WHERE id = ?", (job_id,)).fetchone() is None:
raise HTTPException(404, "Job not found")
since = utcnow() - timedelta(days=days)
return {"job_id": job_id, "since": to_db(since), "runs": repo.list_runs(conn, job_id, since)}
@router.get("/runs/{run_id}")
def get_run(run_id: int, conn: sqlite3.Connection = Depends(db.get_db)):
run = repo.get_run(conn, run_id)
if run is None:
raise HTTPException(404, "Run not found")
return run
@router.post("/ingest")
def ingest(
item: IngestIn,
key_name: str = Depends(require_ingest_key),
conn: sqlite3.Connection = Depends(db.get_db),
):
job_id, run_id = repo.ingest(conn, item)
log.info("ingest key=%s tenant=%r job=%r run=%s", key_name, item.tenant, item.job.name, run_id)
return {"job_id": job_id, "run_id": run_id}
async def _purge_loop(settings: Settings) -> None:
def purge() -> int:
conn = db.connect(settings.db_path)
try:
return repo.purge_old_runs(conn, settings.retention_days)
finally:
conn.close()
while True:
try:
if deleted := await asyncio.to_thread(purge):
log.info("purged %d runs older than %d days", deleted, settings.retention_days)
except Exception:
log.exception("run purge failed")
await asyncio.sleep(PURGE_INTERVAL_SECONDS)
def create_app(settings: Settings | None = None) -> FastAPI:
settings = settings or load_settings()
@asynccontextmanager
async def lifespan(app: FastAPI):
db.init_db(settings.db_path)
task = asyncio.create_task(_purge_loop(settings))
try:
yield
finally:
task.cancel()
# Interactive docs are off: nginx's strict CSP would block their CDN assets anyway.
app = FastAPI(
title="Backup Dashboard API",
lifespan=lifespan,
docs_url=None,
redoc_url=None,
openapi_url="/api/v1/openapi.json",
)
app.state.settings = settings
app.include_router(router)
if settings.serve_static:
from fastapi.staticfiles import StaticFiles
public = Path(__file__).resolve().parents[2] / "public"
app.mount("/", StaticFiles(directory=public, html=True), name="static")
return app
+63
View File
@@ -0,0 +1,63 @@
import json
from typing import Annotated, Any, Literal
from pydantic import AwareDatetime, BaseModel, ConfigDict, Field, StringConstraints, field_validator, model_validator
RunStatus = Literal["running", "success", "warning", "failed"]
Name = Annotated[str, StringConstraints(strip_whitespace=True, min_length=1, max_length=200)]
MAX_MESSAGE_CHARS = 32_000
MAX_DETAILS_BYTES = 16_000
class Strict(BaseModel):
# Reject unknown fields so a typo in a collector script fails loudly instead of being dropped.
model_config = ConfigDict(extra="forbid")
class JobIn(Strict):
external_id: Name = Field(description="The source system's own stable ID for the job")
name: Name
# Omit to leave the stored value untouched; send null to clear it.
next_run_at: AwareDatetime | None = None
class RunIn(Strict):
external_id: Name | None = Field(
None, description="The source's ID for this run. Defaults to started_at, so a 'running' "
"report and its later completion report merge into one run.")
status: RunStatus
started_at: AwareDatetime
# Omit for a finished run to use the time the report was received.
finished_at: AwareDatetime | None = None
message: str = ""
size_bytes: int | None = Field(None, ge=0)
details: dict[str, Any] = Field(default_factory=dict)
@field_validator("message")
@classmethod
def truncate_message(cls, v: str) -> str:
# Backup logs can be huge; keep the head, which is where the summary usually is.
return v if len(v) <= MAX_MESSAGE_CHARS else v[:MAX_MESSAGE_CHARS] + "\n… [truncated]"
@field_validator("details")
@classmethod
def limit_details(cls, v: dict[str, Any]) -> dict[str, Any]:
if len(json.dumps(v)) > MAX_DETAILS_BYTES:
raise ValueError(f"details must serialize to at most {MAX_DETAILS_BYTES} bytes")
return v
@model_validator(mode="after")
def check_times(self) -> "RunIn":
if self.status == "running" and self.finished_at is not None:
raise ValueError("a running run cannot have finished_at")
if self.finished_at is not None and self.finished_at < self.started_at:
raise ValueError("finished_at is before started_at")
return self
class IngestIn(Strict):
tenant: Name = Field(description="Company name as shown on the dashboard")
source_system: Annotated[str, StringConstraints(strip_whitespace=True, min_length=1, max_length=100)]
job: JobIn
run: RunIn | None = Field(None, description="Omit to only register/update the job (e.g. next_run_at)")
+162
View File
@@ -0,0 +1,162 @@
import json
import sqlite3
from datetime import datetime, timedelta
from typing import Any
from .db import transaction
from .models import IngestIn
from .timeutil import to_db, utcnow
# One row per job: `cur` is the most recent run (may be in progress), `fin` the most recent
# finished one. "Last run" on the dashboard is fin.finished_at, so a job that's currently
# running still shows when its previous run completed.
_JOB_SQL = """
SELECT j.id, t.name AS company_name, j.name AS job_name, j.source_system, j.next_run_at,
cur.status,
CASE WHEN cur.status = 'running' THEN cur.started_at END AS running_since,
fin.status AS last_status,
fin.started_at AS last_started_at,
fin.finished_at AS last_run_at,
unixepoch(fin.finished_at) - unixepoch(fin.started_at) AS duration_seconds,
fin.size_bytes,
fin.details,
COALESCE(fin.message, cur.message, '') AS message
FROM jobs j
JOIN tenants t ON t.id = j.tenant_id
LEFT JOIN runs cur ON cur.id = (
SELECT r.id FROM runs r WHERE r.job_id = j.id
ORDER BY r.started_at DESC, r.id DESC LIMIT 1)
LEFT JOIN runs fin ON fin.id = (
SELECT r.id FROM runs r WHERE r.job_id = j.id AND r.finished_at IS NOT NULL
ORDER BY r.finished_at DESC, r.id DESC LIMIT 1)
"""
_SUMMARY_FIELDS = (
"id", "company_name", "job_name", "source_system", "status",
"running_since", "last_run_at", "next_run_at", "message",
)
def list_jobs(conn: sqlite3.Connection) -> list[dict[str, Any]]:
rows = conn.execute(_JOB_SQL + " ORDER BY t.name, j.name").fetchall()
return [{k: row[k] for k in _SUMMARY_FIELDS} for row in rows]
def get_job(conn: sqlite3.Connection, job_id: int) -> dict[str, Any] | None:
row = conn.execute(_JOB_SQL + " WHERE j.id = ?", (job_id,)).fetchone()
if row is None:
return None
job = dict(row)
job["details"] = json.loads(job["details"]) if job["details"] else {}
return job
def list_runs(conn: sqlite3.Connection, job_id: int, since: datetime) -> list[dict[str, Any]]:
# Messages/details are left out: an hourly job has ~720 runs a month and logs can be large.
# The UI fetches a single run's full record on demand.
rows = conn.execute(
"""
SELECT id, status, started_at, finished_at,
unixepoch(finished_at) - unixepoch(started_at) AS duration_seconds, size_bytes
FROM runs WHERE job_id = ? AND started_at >= ?
ORDER BY started_at DESC, id DESC
""",
(job_id, to_db(since)),
).fetchall()
return [dict(r) for r in rows]
def get_run(conn: sqlite3.Connection, run_id: int) -> dict[str, Any] | None:
row = conn.execute(
"""
SELECT id, job_id, status, started_at, finished_at,
unixepoch(finished_at) - unixepoch(started_at) AS duration_seconds,
size_bytes, message, details
FROM runs WHERE id = ?
""",
(run_id,),
).fetchone()
if row is None:
return None
run = dict(row)
run["details"] = json.loads(run["details"])
return run
def ingest(conn: sqlite3.Connection, item: IngestIn) -> tuple[int, int | None]:
"""Upsert tenant → job → run. Safe to call repeatedly with the same payload."""
now = utcnow()
now_s = to_db(now)
with transaction(conn):
conn.execute("INSERT INTO tenants (name) VALUES (?) ON CONFLICT (name) DO NOTHING", (item.tenant,))
tenant_id = conn.execute("SELECT id FROM tenants WHERE name = ?", (item.tenant,)).fetchone()[0]
job_id = conn.execute(
"""
INSERT INTO jobs (tenant_id, source_system, external_id, name, next_run_at, created_at, updated_at)
VALUES (:tenant_id, :source, :ext, :name, :next_run, :now, :now)
ON CONFLICT (tenant_id, source_system, external_id) DO UPDATE SET
name = excluded.name,
next_run_at = CASE WHEN :set_next_run THEN excluded.next_run_at ELSE jobs.next_run_at END,
updated_at = excluded.updated_at
RETURNING id
""",
{
"tenant_id": tenant_id,
"source": item.source_system,
"ext": item.job.external_id,
"name": item.job.name,
"next_run": to_db(item.job.next_run_at) if item.job.next_run_at else None,
"set_next_run": "next_run_at" in item.job.model_fields_set,
"now": now_s,
},
).fetchone()[0]
if item.run is None:
return job_id, None
run = item.run
started = to_db(run.started_at)
finished = None
if run.status != "running":
finished = to_db(run.finished_at or max(now, run.started_at))
params = {
"job_id": job_id,
"ext": run.external_id or started,
"status": run.status,
"started": started,
"finished": finished,
"message": run.message,
"size": run.size_bytes,
"details": json.dumps(run.details),
"now": now_s,
}
# The WHERE stops a late/retried "running" report from reopening a run that already finished.
row = conn.execute(
"""
INSERT INTO runs (job_id, external_id, status, started_at, finished_at, message,
size_bytes, details, received_at)
VALUES (:job_id, :ext, :status, :started, :finished, :message, :size, :details, :now)
ON CONFLICT (job_id, external_id) DO UPDATE SET
status = excluded.status,
started_at = excluded.started_at,
finished_at = excluded.finished_at,
message = excluded.message,
size_bytes = excluded.size_bytes,
details = excluded.details,
received_at = excluded.received_at
WHERE NOT (runs.status != 'running' AND excluded.status = 'running')
RETURNING id
""",
params,
).fetchone()
if row is None:
row = conn.execute(
"SELECT id FROM runs WHERE job_id = :job_id AND external_id = :ext", params
).fetchone()
return job_id, row[0]
def purge_old_runs(conn: sqlite3.Connection, retention_days: int) -> int:
cutoff = to_db(utcnow() - timedelta(days=retention_days))
return conn.execute("DELETE FROM runs WHERE started_at < ?", (cutoff,)).rowcount
+43
View File
@@ -0,0 +1,43 @@
import os
from dataclasses import dataclass
from pathlib import Path
# The modal shows this many days of run history; retention can never be shorter.
HISTORY_DAYS = 30
# Shared by the systemd service (EnvironmentFile=) and manage.py, so both always use the same DB.
DEFAULT_ENV_FILE = "/etc/backup-dashboard/backup-dashboard.env"
@dataclass(frozen=True)
class Settings:
db_path: Path
retention_days: int = 90
# Dev only: serve ../public from the API so you can run without nginx.
serve_static: bool = False
def read_env_file(path: Path) -> dict[str, str]:
"""Parse simple KEY=VALUE lines (the subset systemd's EnvironmentFile accepts)."""
values = {}
try:
lines = path.read_text().splitlines()
except FileNotFoundError:
return values
for line in lines:
line = line.strip()
if not line or line.startswith("#") or "=" not in line:
continue
key, _, value = line.partition("=")
values[key.strip()] = value.strip().strip("\"'")
return values
def load_settings() -> Settings:
# Real environment variables win over the env file; the env file wins over dev defaults.
env = read_env_file(Path(os.environ.get("BACKUP_DASH_ENV_FILE", DEFAULT_ENV_FILE))) | os.environ
return Settings(
db_path=Path(env.get("BACKUP_DASH_DB", "backup-dashboard.db")),
retention_days=max(HISTORY_DAYS, int(env.get("BACKUP_DASH_RETENTION_DAYS", "90"))),
serve_static=env.get("BACKUP_DASH_SERVE_STATIC") == "1",
)
+12
View File
@@ -0,0 +1,12 @@
from datetime import UTC, datetime
# Every timestamp in the DB uses this exact format, so string comparison == chronological comparison.
DB_FORMAT = "%Y-%m-%dT%H:%M:%SZ"
def utcnow() -> datetime:
return datetime.now(UTC).replace(microsecond=0)
def to_db(dt: datetime) -> str:
return dt.astimezone(UTC).strftime(DB_FORMAT)
+133
View File
@@ -0,0 +1,133 @@
#!/usr/bin/env python3
"""Admin CLI. Finds the database the same way the API does: BACKUP_DASH_DB from the environment,
else from /etc/backup-dashboard/backup-dashboard.env, else ./backup-dashboard.db (dev).
Run it as the service user so files it creates stay writable by the API:
sudo -u backupdash /opt/backup-dashboard/venv/bin/python manage.py create-key veeam-acme
python manage.py create-key veeam-acme # prints the key once
python manage.py list-keys
python manage.py revoke-key veeam-acme
python manage.py list-jobs
python manage.py delete-job 12 # e.g. a decommissioned job stuck as Overdue
python manage.py seed-demo # 30 days of fake history for trying the UI
python manage.py --db ./dev.db --create seed-demo
"""
import argparse
import random
import sys
from datetime import timedelta
from pathlib import Path
from app import db, repo
from app.auth import generate_key, hash_key
from app.models import IngestIn
from app.settings import DEFAULT_ENV_FILE, load_settings
from app.timeutil import to_db, utcnow
def create_key(conn, args):
key = generate_key()
try:
conn.execute(
"INSERT INTO api_keys (name, key_hash, created_at) VALUES (?, ?, ?)",
(args.name, hash_key(key), to_db(utcnow())),
)
except db.sqlite3.IntegrityError:
sys.exit(f"A key named {args.name!r} already exists")
print(f"API key for {args.name!r} (shown once, store it in the collector's config):\n{key}")
def list_keys(conn, args):
for r in conn.execute("SELECT name, created_at, last_used_at, revoked_at FROM api_keys ORDER BY name"):
state = f"revoked {r['revoked_at']}" if r["revoked_at"] else "active"
print(f"{r['name']:<30} {state:<30} created {r['created_at']} last used {r['last_used_at'] or 'never'}")
def revoke_key(conn, args):
cur = conn.execute(
"UPDATE api_keys SET revoked_at = ? WHERE name = ? AND revoked_at IS NULL", (to_db(utcnow()), args.name)
)
sys.exit(0 if cur.rowcount else f"No active key named {args.name!r}")
def list_jobs(conn, args):
for j in repo.list_jobs(conn):
print(f"{j['id']:>5} {j['company_name']:<28} {j['job_name']:<40} {j['status'] or '-':<8} last {j['last_run_at'] or 'never'}")
def delete_job(conn, args):
cur = conn.execute("DELETE FROM jobs WHERE id = ?", (args.job_id,))
sys.exit(0 if cur.rowcount else f"No job with id {args.job_id}")
DEMO = [
("Acme Corp", "Veeam B&R", "File Server - Nightly", 24),
("Acme Corp", "Veeam B&R", "SQL01 - Transaction Logs", 4),
("Bluewater Dental", "Datto", "Workstations - Daily", 24),
("Bluewater Dental", "Datto", "Imaging Server - Hourly", 1),
("Northfield Logistics", "Acronis", "Exchange - Nightly", 24),
]
def seed_demo(conn, args):
rng = random.Random(42)
now = utcnow()
for tenant, source, name, every_h in DEMO:
ext = name.lower().replace(" ", "-")
# Offset so runs don't all line up exactly on the current time
start = now - timedelta(days=30) + timedelta(minutes=rng.randint(5, every_h * 60 - 5))
while start < now:
dur = timedelta(minutes=rng.randint(2, 50 if every_h > 1 else 15))
end = start + dur
status = "running" if end > now else rng.choices(["success", "warning", "failed"], [90, 7, 3])[0]
run = {"status": status, "started_at": start.isoformat()}
if status != "running":
run |= {"finished_at": end.isoformat(), "size_bytes": rng.randint(10**8, 10**11),
"message": {"success": "Job completed successfully.",
"warning": "Completed with warnings: 1 file skipped (locked).",
"failed": "Error: target repository unreachable."}[status]}
repo.ingest(conn, IngestIn.model_validate({
"tenant": tenant, "source_system": source,
"job": {"external_id": ext, "name": name,
"next_run_at": (start + timedelta(hours=every_h)).isoformat()},
"run": run,
}))
start += timedelta(hours=every_h)
print(f"Seeded {len(DEMO)} demo jobs")
def main():
p = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter)
p.add_argument("--db", type=Path, help="database file (overrides BACKUP_DASH_DB and the env file)")
p.add_argument("--create", action="store_true", help="create the database if it doesn't exist")
sub = p.add_subparsers(required=True)
s = sub.add_parser("create-key"); s.add_argument("name"); s.set_defaults(fn=create_key)
s = sub.add_parser("list-keys"); s.set_defaults(fn=list_keys)
s = sub.add_parser("revoke-key"); s.add_argument("name"); s.set_defaults(fn=revoke_key)
s = sub.add_parser("list-jobs"); s.set_defaults(fn=list_jobs)
s = sub.add_parser("delete-job"); s.add_argument("job_id", type=int); s.set_defaults(fn=delete_job)
s = sub.add_parser("seed-demo"); s.set_defaults(fn=seed_demo)
args = p.parse_args()
db_path = args.db or load_settings().db_path
# Refuse to silently create a fresh DB: that almost always means we're looking in the wrong
# place, and anything written there (like a new API key) would be invisible to the service.
if not db_path.exists() and not args.create:
sys.exit(
f"Database not found: {db_path.resolve()}\n"
f"Set BACKUP_DASH_DB in {DEFAULT_ENV_FILE} (or pass --db), "
"or add --create to start a new database there."
)
print(f"Using database {db_path.resolve()}", file=sys.stderr)
db.init_db(db_path)
conn = db.connect(db_path)
try:
args.fn(conn, args)
finally:
conn.close()
if __name__ == "__main__":
main()
+2
View File
@@ -0,0 +1,2 @@
[pytest]
pythonpath = .
+2
View File
@@ -0,0 +1,2 @@
httpx
pytest
+2
View File
@@ -0,0 +1,2 @@
fastapi>=0.115
uvicorn[standard]>=0.30
View File
+124
View File
@@ -0,0 +1,124 @@
from datetime import timedelta
import pytest
from fastapi.testclient import TestClient
from app import db
from app.auth import generate_key, hash_key
from app.main import create_app
from app.settings import Settings
from app.timeutil import to_db, utcnow
@pytest.fixture
def client(tmp_path):
settings = Settings(db_path=tmp_path / "test.db")
with TestClient(create_app(settings)) as c:
key = generate_key()
conn = db.connect(settings.db_path)
conn.execute("INSERT INTO api_keys (name, key_hash, created_at) VALUES ('test', ?, ?)",
(hash_key(key), to_db(utcnow())))
conn.close()
c.headers["Authorization"] = f"Bearer {key}"
yield c
def iso(delta: timedelta) -> str:
return (utcnow() + delta).isoformat()
def payload(run=None, **job):
body = {"tenant": "Acme Corp", "source_system": "Veeam B&R",
"job": {"external_id": "job-1", "name": "File Server", **job}}
if run is not None:
body["run"] = run
return body
def test_ingest_requires_valid_key(client):
assert client.post("/api/v1/ingest", json=payload(), headers={"Authorization": ""}).status_code == 401
assert client.post("/api/v1/ingest", json=payload(), headers={"Authorization": "Bearer nope"}).status_code == 401
def test_running_then_finished_merges_into_one_run(client):
started = iso(-timedelta(hours=1))
r = client.post("/api/v1/ingest", json=payload({"status": "running", "started_at": started}))
assert r.status_code == 200, r.text
job_id = r.json()["job_id"]
job = client.get("/api/v1/jobs").json()["jobs"][0]
assert job["status"] == "running"
assert job["running_since"] is not None
assert job["last_run_at"] is None # nothing has finished yet
finished = iso(-timedelta(minutes=10))
client.post("/api/v1/ingest", json=payload(
{"status": "warning", "started_at": started, "finished_at": finished, "message": "1 file skipped"}))
detail = client.get(f"/api/v1/jobs/{job_id}").json()
assert detail["status"] == "warning"
assert detail["running_since"] is None
assert detail["last_run_at"] == to_db(utcnow().fromisoformat(finished))
assert detail["duration_seconds"] == 50 * 60
assert detail["message"] == "1 file skipped"
assert len(client.get(f"/api/v1/jobs/{job_id}/runs").json()["runs"]) == 1
def test_late_running_report_does_not_reopen_finished_run(client):
started = iso(-timedelta(hours=1))
client.post("/api/v1/ingest", json=payload({"status": "success", "started_at": started, "finished_at": iso(-timedelta(minutes=5))}))
client.post("/api/v1/ingest", json=payload({"status": "running", "started_at": started}))
assert client.get("/api/v1/jobs").json()["jobs"][0]["status"] == "success"
def test_last_run_is_previous_finish_while_new_run_in_progress(client):
prev_end = iso(-timedelta(hours=23))
client.post("/api/v1/ingest", json=payload({"status": "failed", "started_at": iso(-timedelta(hours=24)), "finished_at": prev_end, "message": "boom"}))
client.post("/api/v1/ingest", json=payload({"status": "running", "started_at": iso(-timedelta(minutes=5))}))
job = client.get("/api/v1/jobs").json()["jobs"][0]
assert job["status"] == "running"
assert job["last_run_at"] == to_db(utcnow().fromisoformat(prev_end))
detail = client.get(f"/api/v1/jobs/{job['id']}").json()
assert detail["last_status"] == "failed"
assert detail["message"] == "boom"
def test_next_run_at_only_changes_when_sent(client):
nxt = iso(timedelta(hours=5))
client.post("/api/v1/ingest", json=payload(next_run_at=nxt))
client.post("/api/v1/ingest", json=payload()) # omitted → keep
assert client.get("/api/v1/jobs").json()["jobs"][0]["next_run_at"] == to_db(utcnow().fromisoformat(nxt))
client.post("/api/v1/ingest", json=payload(next_run_at=None)) # explicit null → clear
assert client.get("/api/v1/jobs").json()["jobs"][0]["next_run_at"] is None
def test_history_window_and_run_detail(client):
for days_ago in (40, 20, 1):
start = iso(-timedelta(days=days_ago))
client.post("/api/v1/ingest", json=payload({"status": "success", "started_at": start,
"finished_at": start, "message": f"{days_ago}d"}))
job_id = client.get("/api/v1/jobs").json()["jobs"][0]["id"]
runs = client.get(f"/api/v1/jobs/{job_id}/runs").json()["runs"]
assert len(runs) == 2 # the 40-day-old run is outside the 30-day window
assert "message" not in runs[0]
run = client.get(f"/api/v1/runs/{runs[0]['id']}").json()
assert run["message"] == "1d"
assert client.get(f"/api/v1/jobs/{job_id}/runs?days=31").status_code == 422
@pytest.mark.parametrize("run", [
{"status": "running", "started_at": iso(-timedelta(hours=1)), "finished_at": iso(timedelta(0))},
{"status": "success", "started_at": iso(timedelta(0)), "finished_at": iso(-timedelta(hours=1))},
{"status": "success", "started_at": "2026-01-01T00:00:00"}, # no timezone
{"status": "done", "started_at": iso(timedelta(0))},
])
def test_rejects_bad_runs(client, run):
assert client.post("/api/v1/ingest", json=payload(run)).status_code == 422
def test_tenant_name_is_case_insensitive(client):
client.post("/api/v1/ingest", json=payload())
body = payload()
body["tenant"] = "ACME CORP"
client.post("/api/v1/ingest", json=body)
assert len(client.get("/api/v1/jobs").json()["jobs"]) == 1
+52
View File
@@ -0,0 +1,52 @@
import subprocess
import sys
from pathlib import Path
from app.settings import load_settings
BACKEND = Path(__file__).resolve().parents[1]
def run_manage(*args, env, cwd):
return subprocess.run([sys.executable, str(BACKEND / "manage.py"), *args],
capture_output=True, text=True, env=env, cwd=cwd)
def base_env(tmp_path, monkeypatch):
env_file = tmp_path / "backup-dashboard.env"
monkeypatch.setenv("BACKUP_DASH_ENV_FILE", str(env_file))
monkeypatch.delenv("BACKUP_DASH_DB", raising=False)
return env_file, {"BACKUP_DASH_ENV_FILE": str(env_file), "PYTHONPATH": str(BACKEND), "PATH": "/usr/bin:/bin"}
def test_env_file_supplies_db_path_and_real_env_wins(tmp_path, monkeypatch):
env_file, _ = base_env(tmp_path, monkeypatch)
env_file.write_text('# comment\nBACKUP_DASH_DB="/srv/from-file.db"\nBACKUP_DASH_RETENTION_DAYS=120\n')
assert load_settings().db_path == Path("/srv/from-file.db")
assert load_settings().retention_days == 120
monkeypatch.setenv("BACKUP_DASH_DB", "/srv/from-env.db")
assert load_settings().db_path == Path("/srv/from-env.db")
def test_manage_refuses_to_create_db_implicitly(tmp_path, monkeypatch):
_, env = base_env(tmp_path, monkeypatch)
workdir = tmp_path / "somewhere-else"
workdir.mkdir()
r = run_manage("create-key", "k", env=env, cwd=workdir)
assert r.returncode != 0
assert "Database not found" in r.stderr
assert not (workdir / "backup-dashboard.db").exists()
def test_manage_uses_db_from_env_file(tmp_path, monkeypatch):
env_file, env = base_env(tmp_path, monkeypatch)
service_db = tmp_path / "service.db"
env_file.write_text(f"BACKUP_DASH_DB={service_db}\n")
assert run_manage("--create", "list-keys", env=env, cwd=tmp_path).returncode == 0 # service's first start
workdir = tmp_path / "somewhere-else"
workdir.mkdir()
r = run_manage("create-key", "veeam-acme", env=env, cwd=workdir)
assert r.returncode == 0, r.stderr
assert str(service_db) in r.stderr
assert "veeam-acme" in run_manage("list-keys", env=env, cwd=tmp_path).stdout
assert not (workdir / "backup-dashboard.db").exists()
@@ -0,0 +1,154 @@
<#
.SYNOPSIS
Read-only report of how Veeam jobs and sessions relate, for troubleshooting the collector.
.DESCRIPTION
The collector matches sessions to jobs by Session.JobId = Job.Id. When a job shows as
"Unknown" on the dashboard, no session matched. This lists:
1. every job from Get-VBRJob and how many sessions matched it
2. sessions whose JobId matches no job (the runs that are going missing)
3. for jobs with no matching sessions, their type/link-related properties and methods
4. what other job cmdlets (e.g. Get-VBRBackupCopyJob) return
Nothing is changed in Veeam or the dashboard. Output is also saved to a text file.
.EXAMPLE
.\Get-VeeamCollectorDiagnostics.ps1
.EXAMPLE
.\Get-VeeamCollectorDiagnostics.ps1 -LookbackHours 168
#>
[CmdletBinding()]
param(
[int]$LookbackHours = 48,
[string]$OutFile = (Join-Path $env:TEMP 'veeam-collector-diagnostics.txt')
)
$ErrorActionPreference = 'Stop'
$LinkPattern = 'Parent|Linked|Worker|Child|Policy|Source|Target|Type|Cloud|Mode|Session'
if (-not (Get-Command Get-VBRJob -ErrorAction SilentlyContinue)) {
if (Get-Module -ListAvailable -Name Veeam.Backup.PowerShell) {
Import-Module Veeam.Backup.PowerShell -WarningAction SilentlyContinue
} else {
Add-PSSnapin VeeamPSSnapin
}
}
function Format-Value($Value) {
if ($null -eq $Value) { return '<null>' }
$text = ($Value | ForEach-Object { [string]$_ }) -join ', '
if ($text.Length -gt 120) { $text = $text.Substring(0, 117) + '...' }
return $text
}
# Property and method names matching $LinkPattern, with property values. Every read is guarded:
# some Veeam properties throw when accessed.
function Show-LinkMembers($Object) {
foreach ($p in $Object.PSObject.Properties | Where-Object { $_.Name -match $LinkPattern }) {
$value = try { Format-Value $p.Value } catch { "<error: $($_.Exception.Message)>" }
' {0} = {1}' -f $p.Name, $value
}
$methods = @($Object | Get-Member -MemberType Methods -ErrorAction SilentlyContinue |
Where-Object { $_.Name -match $LinkPattern } | ForEach-Object Name)
if ($methods) { ' methods: ' + ($methods -join ', ') }
}
function Get-Prop($Object, [string]$Name) {
try { Format-Value $Object.$Name } catch { '<error>' }
}
$report = & {
$vbrVersion = try {
(Get-Item (Join-Path ${env:ProgramFiles} 'Veeam\Backup and Replication\Backup\Veeam.Backup.Core.dll')).VersionInfo.ProductVersion
} catch { 'unknown' }
"Generated $(Get-Date -Format o) on $env:COMPUTERNAME, VBR $vbrVersion, PowerShell $($PSVersionTable.PSVersion)"
"Lookback: $LookbackHours hours"
''
$since = (Get-Date).AddHours(-$LookbackHours)
$jobs = @(Get-VBRJob -WarningAction SilentlyContinue)
$sessions = @(Get-VBRBackupSession -WarningAction SilentlyContinue | Where-Object { $_.CreationTime -ge $since })
$jobIds = @{}
foreach ($j in $jobs) { $jobIds[$j.Id.ToString()] = $j }
'== 1. Jobs from Get-VBRJob'
$jobs | ForEach-Object {
$id = $_.Id.ToString()
[pscustomobject]@{
Name = $_.Name
JobType = Get-Prop $_ 'JobType'
TypeToString = Get-Prop $_ 'TypeToString'
Id = $id
MatchingSessions = @($sessions | Where-Object { $_.JobId.ToString() -eq $id }).Count
}
} | Format-Table -AutoSize | Out-String -Width 300
'== 2. Sessions whose JobId matches no job from Get-VBRJob'
$orphans = @($sessions | Where-Object { -not $jobIds.ContainsKey($_.JobId.ToString()) })
if ($orphans.Count -eq 0) {
' (none)'
} else {
$orphans | Group-Object { $_.JobId.ToString() } | ForEach-Object {
$latest = $_.Group | Sort-Object CreationTime | Select-Object -Last 1
[pscustomobject]@{
JobId = $_.Name
JobName = Get-Prop $latest 'JobName'
JobType = Get-Prop $latest 'JobType'
Sessions = $_.Count
LatestStart = $latest.CreationTime
State = Get-Prop $latest 'State'
Result = Get-Prop $latest 'Result'
}
} | Format-Table -AutoSize | Out-String -Width 300
' Link-related members of one orphan session:'
Show-LinkMembers $orphans[0]
''
' Job object for that session (via GetJob(), if available):'
try {
$owner = $orphans[0].GetJob()
' Name = {0}, Id = {1}' -f (Get-Prop $owner 'Name'), (Get-Prop $owner 'Id')
Show-LinkMembers $owner
} catch {
" not available: $($_.Exception.Message)"
}
}
''
'== 3. Jobs with no matching sessions in the window'
$empty = @($jobs | Where-Object { $id = $_.Id.ToString(); -not ($sessions | Where-Object { $_.JobId.ToString() -eq $id }) })
if ($empty.Count -eq 0) { ' (none)' }
foreach ($j in $empty) {
" $($j.Name) [$($j.Id)]"
Show-LinkMembers $j
try {
$last = $j.FindLastSession()
if ($last) {
' FindLastSession(): JobId = {0}, JobName = {1}, Start = {2}, Result = {3}' -f `
(Get-Prop $last 'JobId'), (Get-Prop $last 'JobName'), $last.CreationTime, (Get-Prop $last 'Result')
} else {
' FindLastSession(): <null>'
}
} catch {
" FindLastSession(): not available: $($_.Exception.Message)"
}
''
}
'== 4. Other job cmdlets'
foreach ($cmd in 'Get-VBRBackupCopyJob', 'Get-VBRComputerBackupCopyJob', 'Get-VBRComputerBackupJob') {
if (-not (Get-Command $cmd -ErrorAction SilentlyContinue)) { " ${cmd}: not available"; continue }
try {
$found = @(& $cmd -WarningAction SilentlyContinue)
" ${cmd}: $($found.Count) job(s)"
foreach ($f in $found) {
' {0} [{1}] Type = {2}, Mode = {3}' -f (Get-Prop $f 'Name'), (Get-Prop $f 'Id'), (Get-Prop $f 'Type'), (Get-Prop $f 'Mode')
}
} catch {
" ${cmd}: error: $($_.Exception.Message)"
}
}
} | Out-String -Width 300
$report
$report | Set-Content -Path $OutFile -Encoding UTF8
Write-Host "Saved to $OutFile"
+102
View File
@@ -0,0 +1,102 @@
<#
.SYNOPSIS
Installs the Veeam collector on a VBR server as a SYSTEM scheduled task.
.DESCRIPTION
- Copies Send-VeeamBackupStatus.ps1 to Program Files (only admins can modify it, which
matters because it runs as SYSTEM).
- Writes the config to ProgramData\BackupDashboard and restricts that folder to SYSTEM and
Administrators, since it contains the API key.
- Registers a scheduled task that runs every IntervalMinutes under PowerShell 7 (pwsh.exe),
which the Veeam module on VBR 13 expects. Install PowerShell 7 machine-wide (MSI or winget)
first: Microsoft Store installs can't be run by SYSTEM.
Safe to re-run to change settings or update the script.
.EXAMPLE
.\Install-VeeamCollector.ps1 -ApiUrl https://backups.example.com -Tenant "Acme Corp"
(prompts for the API key)
#>
#Requires -RunAsAdministrator
[CmdletBinding()]
param(
[Parameter(Mandatory)][string]$ApiUrl,
[Parameter(Mandatory)][string]$Tenant,
[string]$SourceSystem = 'Veeam B&R',
[ValidateRange(5, 1440)][int]$IntervalMinutes = 15,
[ValidateRange(1, 720)][int]$LookbackHours = 48,
[string]$InstallDir = (Join-Path $env:ProgramFiles 'BackupDashboard'),
# Defaults to the machine-wide PowerShell 7 install
[string]$PwshPath
)
$ErrorActionPreference = 'Stop'
$TaskName = 'Backup Dashboard - Veeam collector'
$DataDir = Join-Path $env:ProgramData 'BackupDashboard'
$ConfigPath = Join-Path $DataDir 'veeam-collector.json'
# Find PowerShell 7 before changing anything, so a missing install fails cleanly
if (-not $PwshPath) {
$PwshPath = Join-Path $env:ProgramFiles 'PowerShell\7\pwsh.exe'
if (-not (Test-Path $PwshPath)) {
$onPath = Get-Command pwsh.exe -ErrorAction SilentlyContinue
if ($onPath) { $PwshPath = $onPath.Source }
}
}
if (-not (Test-Path $PwshPath)) {
throw "PowerShell 7 not found at '$PwshPath'. Install it machine-wide (e.g. winget install Microsoft.PowerShell) or pass -PwshPath."
}
if ($PwshPath -match '\\WindowsApps\\') {
throw "'$PwshPath' is a Microsoft Store install, which the SYSTEM account can't run. Install the MSI or winget version instead."
}
$pwshVersion = & $PwshPath -NoProfile -NonInteractive -Command '$PSVersionTable.PSVersion.ToString()'
$pwshMajor = & $PwshPath -NoProfile -NonInteractive -Command '$PSVersionTable.PSVersion.Major'
if ($LASTEXITCODE -or ($pwshMajor -as [int]) -lt 7) {
throw "'$PwshPath' isn't a working PowerShell 7 (reported '$pwshVersion')."
}
if ($ApiUrl -notmatch '^https://') {
Write-Warning 'ApiUrl is not HTTPS: the API key would be sent in clear text.'
}
# Read the key as a SecureString so it isn't echoed to the console or kept in shell history
$secure = Read-Host -AsSecureString 'API key (from: python manage.py create-key <name>)'
$bstr = [Runtime.InteropServices.Marshal]::SecureStringToBSTR($secure)
try { $apiKey = [Runtime.InteropServices.Marshal]::PtrToStringBSTR($bstr) }
finally { [Runtime.InteropServices.Marshal]::ZeroFreeBSTR($bstr) }
if (-not $apiKey.StartsWith('bdk_')) { throw "That doesn't look like a dashboard API key (expected 'bdk_...')." }
# Config folder: SYSTEM + Administrators only (SIDs so this works on non-English Windows)
New-Item -ItemType Directory -Path $DataDir -Force | Out-Null
& icacls.exe $DataDir /inheritance:r /grant:r '*S-1-5-18:(OI)(CI)F' '*S-1-5-32-544:(OI)(CI)F' | Out-Null
if ($LASTEXITCODE) { throw "icacls failed on $DataDir" }
[ordered]@{
ApiUrl = $ApiUrl.TrimEnd('/')
ApiKey = $apiKey
Tenant = $Tenant
SourceSystem = $SourceSystem
LookbackHours = $LookbackHours
} | ConvertTo-Json | Set-Content -Path $ConfigPath -Encoding UTF8
New-Item -ItemType Directory -Path $InstallDir -Force | Out-Null
$scriptPath = Join-Path $InstallDir 'Send-VeeamBackupStatus.ps1'
Copy-Item -Path (Join-Path $PSScriptRoot 'Send-VeeamBackupStatus.ps1') -Destination $scriptPath -Force
Unblock-File -Path $scriptPath
$action = New-ScheduledTaskAction -Execute $PwshPath `
-Argument "-NoProfile -NonInteractive -ExecutionPolicy Bypass -File `"$scriptPath`""
$trigger = New-ScheduledTaskTrigger -Once -At (Get-Date).AddMinutes(1) `
-RepetitionInterval (New-TimeSpan -Minutes $IntervalMinutes)
$principal = New-ScheduledTaskPrincipal -UserId 'S-1-5-18' -LogonType ServiceAccount -RunLevel Highest
$settings = New-ScheduledTaskSettingsSet -ExecutionTimeLimit (New-TimeSpan -Minutes 30) `
-MultipleInstances IgnoreNew -StartWhenAvailable
Register-ScheduledTask -TaskName $TaskName -Action $action -Trigger $trigger -Principal $principal `
-Settings $settings -Description 'Sends Veeam job results to the Backup Dashboard' -Force | Out-Null
Write-Host "Installed. Config: $ConfigPath"
Write-Host "Task '$TaskName' runs every $IntervalMinutes minutes as SYSTEM, using PowerShell $pwshVersion ($PwshPath)."
Write-Host ''
Write-Host 'Next steps:'
Write-Host " 1. Preview what will be sent: & '$PwshPath' -File '$scriptPath' -DryRun"
Write-Host " 2. Backfill 30 days of history: & '$PwshPath' -File '$scriptPath' -LookbackHours 720"
Write-Host " 3. Check the log: Get-Content '$DataDir\veeam-collector.log' -Tail 20"
+81
View File
@@ -0,0 +1,81 @@
# Veeam Backup & Replication collector
A PowerShell script that runs on each client's VBR server and reports job results to the dashboard.
It covers jobs returned by `Get-VBRJob`: VM backup, replication and backup copy jobs.
## Install (on the VBR server, as an administrator)
Requires PowerShell 7 installed machine-wide (`winget install Microsoft.PowerShell` or the MSI),
which VBR 13's Veeam module expects. The Microsoft Store version won't work, because the SYSTEM
account can't run it. The installer checks for this before changing anything.
1. On the dashboard host, create a key for this server:
```bash
python manage.py create-key veeam-acme
```
2. Copy this folder to the VBR server and run, from PowerShell 7 (`pwsh`):
```powershell
.\Install-VeeamCollector.ps1 -ApiUrl https://backups.example.com -Tenant "Acme Corp"
```
It prompts for the key, then:
- writes `C:\ProgramData\BackupDashboard\veeam-collector.json` (readable only by SYSTEM and Administrators)
- copies the script to `C:\Program Files\BackupDashboard\`
- registers the scheduled task **Backup Dashboard - Veeam collector** (SYSTEM, every 15 minutes,
run by `C:\Program Files\PowerShell\7\pwsh.exe`; use `-PwshPath` if it's installed elsewhere)
3. Preview the payloads, then backfill 30 days so the history view is populated straight away
(from `pwsh`):
```powershell
& 'C:\Program Files\BackupDashboard\Send-VeeamBackupStatus.ps1' -DryRun
& 'C:\Program Files\BackupDashboard\Send-VeeamBackupStatus.ps1' -LookbackHours 720
```
Re-run the installer to change settings or update the script. To remove the collector,
delete the scheduled task and both `BackupDashboard` folders.
## What it sends
| Dashboard | Veeam |
|---|---|
| Company | `Tenant` from the config |
| Job | Job name. The job's GUID is the ID, so renaming a job in Veeam doesn't duplicate it |
| Status | Session result (Success / Warning / Failed). Any state other than Stopped is **Running**. A session that stopped with no result (e.g. cancelled) is reported as **Warning** |
| Started / finished | Session `CreationTime` / `EndTime` |
| Next scheduled | Job's next run. Sent as empty for disabled jobs and chained ("run after") jobs, so those are never flagged Overdue |
| Message | Session reason, each VM or object that didn't succeed, and warning/error lines from the session log |
| Size | `BackupStats.BackupSize` |
**Backup copy jobs (VBR 12+):** Veeam records their sessions under hidden per-source "worker" jobs.
The collector files each worker session under its copy job (via the worker's `PolicyJobId`), so a
copy job gets one run per source job it copies, with the source in `copied_from`.
Each run covers the last 48 hours (`LookbackHours`), so a few missed runs (server rebooting, API
down) fill in automatically. Sessions already delivered are recorded in
`veeam-collector.state.json`. Running sessions are re-sent until they finish.
## Troubleshooting
- Log: `C:\ProgramData\BackupDashboard\veeam-collector.log` (rotates at 5 MB).
- Exit codes: `0` OK, `1` some jobs failed to send (they're retried next run), `2` setup failed
(config, Veeam module, or Veeam connection).
- `API key rejected`: the key was revoked or mistyped. Re-run the installer with a new key.
- A job shows **Unknown** on the dashboard: the collector sent the job but matched no sessions to it.
Run `.\Get-VeeamCollectorDiagnostics.ps1` (read-only). It lists sessions whose job ID matches no
job, plus the type and link properties of the jobs left without sessions.
- `Get-VBRBackupSession` loads every session in the VBR database. On a server with years of history,
this can make each run take a minute or more. The task's 30-minute limit and "don't start if
already running" setting keep that safe.
## Testing without Veeam
`tests/MockVeeam.ps1` defines stand-in `Get-VBRJob` / `Get-VBRBackupSession` cmdlets with a mix of
success, warning, failed, running and disabled jobs:
```powershell
. ./tests/MockVeeam.ps1
./Send-VeeamBackupStatus.ps1 -ConfigPath ./my-test-config.json -DryRun
```
## Not covered yet
- Veeam Agent jobs managed by VBR (`Get-VBRComputerBackupJob`), tape jobs, and SureBackup.
- Veeam Backup for Microsoft 365, which has its own PowerShell module.
+335
View File
@@ -0,0 +1,335 @@
<#
.SYNOPSIS
Reports Veeam Backup & Replication job results to the Backup Dashboard.
.DESCRIPTION
Runs on the VBR server, normally as a SYSTEM scheduled task every 15 minutes
(see Install-VeeamCollector.ps1). For every backup, replica and backup copy job it sends:
- the job's next scheduled run (drives the dashboard's Overdue flag), and
- every session from the last LookbackHours that the dashboard hasn't already received
in its final state. Running sessions are re-sent each time until they finish.
Already-reported sessions are tracked in a state file, and the ingest API is idempotent,
so a failed or repeated run is harmless: anything not confirmed is retried next time.
Compatible with Windows PowerShell 5.1 and PowerShell 7.
.PARAMETER ConfigPath
JSON config with ApiUrl, ApiKey and Tenant (optional: SourceSystem, LookbackHours).
.PARAMETER LookbackHours
Overrides the config value. Use 720 once to backfill 30 days of history.
.PARAMETER DryRun
Print the payloads instead of sending them. Doesn't touch the state or log files.
.EXAMPLE
.\Send-VeeamBackupStatus.ps1 -DryRun
.EXAMPLE
.\Send-VeeamBackupStatus.ps1 -LookbackHours 720
#>
[CmdletBinding()]
param(
[string]$ConfigPath = (Join-Path $env:ProgramData 'BackupDashboard\veeam-collector.json'),
[int]$LookbackHours = 0,
[switch]$DryRun
)
$ErrorActionPreference = 'Stop'
$DataDir = Split-Path -Parent $ConfigPath
$LogPath = Join-Path $DataDir 'veeam-collector.log'
$StatePath = Join-Path $DataDir 'veeam-collector.state.json'
$MaxLogBytes = 5MB
$Invariant = [Globalization.CultureInfo]::InvariantCulture
# ---------- Helpers ----------
function Write-Log {
param([string]$Message, [string]$Level = 'INFO')
$line = '{0} [{1}] {2}' -f (Get-Date).ToString('yyyy-MM-dd HH:mm:ss'), $Level, $Message
Write-Host $line
if ($DryRun) { return }
if ((Test-Path $LogPath) -and (Get-Item $LogPath).Length -gt $MaxLogBytes) {
Move-Item $LogPath "$LogPath.1" -Force
}
Add-Content -Path $LogPath -Value $line -Encoding UTF8
}
function Get-ErrorText($ErrorRecord) {
# Windows PowerShell puts the HTTP response body (e.g. the API's 422 explanation) here
if ($ErrorRecord.ErrorDetails -and $ErrorRecord.ErrorDetails.Message) {
return "$($ErrorRecord.Exception.Message) $($ErrorRecord.ErrorDetails.Message)"
}
return $ErrorRecord.Exception.Message
}
# The API needs an explicit UTC offset. Veeam returns local times, often with Kind=Unspecified,
# which DateTimeOffset treats as local, exactly what we want.
function ConvertTo-IsoTime($Value) {
if ($null -eq $Value) { return $null }
$dt = [datetime]$Value
if ($dt.Year -le 1900) { return $null } # Veeam's "not set" value (e.g. EndTime while running)
return ([DateTimeOffset]$dt).ToString("yyyy-MM-dd'T'HH:mm:sszzz", $Invariant)
}
function Import-Veeam {
if (Get-Command Get-VBRJob -ErrorAction SilentlyContinue) { return } # already loaded
if (Get-Module -ListAvailable -Name Veeam.Backup.PowerShell) {
Import-Module Veeam.Backup.PowerShell -WarningAction SilentlyContinue # VBR 11+
} else {
Add-PSSnapin VeeamPSSnapin # VBR 10 and earlier
}
}
function Get-NextRun($Job) {
try {
if (-not $Job.IsScheduleEnabled) { return $null }
$raw = $Job.ScheduleOptions.NextRun
if (-not $raw) { return $null } # e.g. chained jobs that run "after" another job
if ($raw -is [datetime]) { return ConvertTo-IsoTime $raw }
# Older versions return a string formatted with the server's culture, so parse with it
$parsed = [datetime]::MinValue
if ([datetime]::TryParse([string]$raw, [Globalization.CultureInfo]::CurrentCulture,
[Globalization.DateTimeStyles]::None, [ref]$parsed)) {
return ConvertTo-IsoTime $parsed
}
Write-Log "Couldn't parse next run '$raw' for job '$($Job.Name)'" 'WARN'
} catch {
Write-Log "Couldn't read schedule for job '$($Job.Name)': $(Get-ErrorText $_)" 'WARN'
}
return $null
}
function Get-RunStatus($Session) {
if ([string]$Session.State -ne 'Stopped') { return 'running' }
switch ([string]$Session.Result) {
'Success' { return 'success' }
'Warning' { return 'warning' }
'Failed' { return 'failed' }
# Stopped without a result (e.g. cancelled): surface it rather than hide it
default { return 'warning' }
}
}
# Builds a readable summary: the session's reason, any task (VM/object) that didn't succeed,
# and warning/error lines from the session log.
function Get-RunMessage($Session, [string]$Status, $Tasks) {
$lines = New-Object 'System.Collections.Generic.List[string]'
try {
$reason = [string]$Session.Info.Reason
if ($reason.Trim()) { $lines.Add($reason.Trim()) }
} catch { }
foreach ($task in $Tasks) {
$taskStatus = [string]$task.Status
if ($taskStatus -and $taskStatus -ne 'Success' -and $taskStatus -ne 'InProgress' -and $taskStatus -ne 'Pending') {
$taskReason = ''
try { $taskReason = ([string]$task.Info.Reason).Trim() } catch { }
$line = "$($task.Name): $taskStatus"
if ($taskReason) { $line += " - $taskReason" }
$lines.Add($line)
}
}
if ($Status -eq 'warning' -or $Status -eq 'failed') {
try {
foreach ($record in $Session.Logger.GetLog().UpdatedRecords) {
$recordStatus = [string]$record.Status
if ($recordStatus -eq 'EWarning' -or $recordStatus -eq 'EFailed') {
$title = ([string]$record.Title).Trim()
if ($title -and -not $lines.Contains($title)) { $lines.Add($title) }
}
}
} catch { }
}
if ($lines.Count -eq 0) {
switch ($Status) {
'success' { return 'Job completed successfully.' }
'running' { return '' }
default { return "Job finished with result '$($Session.Result)'." }
}
}
return ($lines -join "`n")
}
# VBR 12+ backup copy jobs: Get-VBRJob returns the copy (policy) job, but Get-VBRBackupSession
# returns only the sessions of its hidden worker jobs, one per source job, never the policy's own.
# Each worker job records its policy in PolicyJobId, so those sessions can be filed under it.
function Get-PolicyJobId($Session, [hashtable]$KnownJobIds) {
try {
$policyId = [string]$Session.GetJob().PolicyJobId
if ($policyId -and $KnownJobIds.ContainsKey($policyId)) { return $policyId }
} catch { }
return ''
}
function ConvertTo-Run($Session, $Job) {
$status = Get-RunStatus $Session
$tasks = @()
try { $tasks = @($Session.GetTaskSessions()) } catch { }
$details = [ordered]@{
job_type = [string]$Job.JobType
vbr_server = $env:COMPUTERNAME
objects = $tasks.Count
}
$failedTasks = @($tasks | Where-Object { [string]$_.Status -eq 'Failed' }).Count
$warningTasks = @($tasks | Where-Object { [string]$_.Status -eq 'Warning' }).Count
if ($failedTasks) { $details.objects_failed = $failedTasks }
if ($warningTasks) { $details.objects_warning = $warningTasks }
try { if ($Session.IsRetryMode) { $details.retry = $true } } catch { }
# Worker session of a backup copy job: say which source job it copied
# (worker sessions are named "<copy job>\<source job>")
if ($Session.JobId.ToString() -ne $Job.Id.ToString()) {
$details.copied_from = ([string]$Session.JobName -split '\\')[-1]
}
$run = [ordered]@{
external_id = $Session.Id.ToString()
status = $status
started_at = ConvertTo-IsoTime $Session.CreationTime
message = Get-RunMessage $Session $status $tasks
details = $details
}
if ($status -ne 'running') {
$finished = ConvertTo-IsoTime $Session.EndTime
if ($finished) { $run.finished_at = $finished } # otherwise the API uses receipt time
try {
$size = [long]$Session.BackupStats.BackupSize
if ($size -gt 0) { $run.size_bytes = $size }
$dataSize = [long]$Session.BackupStats.DataSize
if ($dataSize -gt 0) { $details.data_size_bytes = $dataSize }
} catch { }
}
return $run
}
function Send-Report($Body) {
$json = $Body | ConvertTo-Json -Depth 6 -Compress
if ($DryRun) {
Write-Host $json
return
}
# Send bytes: Windows PowerShell would otherwise encode a string body as Latin-1,
# mangling non-ASCII job names (e.g. en dashes).
$params = @{
Uri = "$($Config.ApiUrl.TrimEnd('/'))/api/v1/ingest"
Method = 'Post'
Headers = @{ Authorization = "Bearer $($Config.ApiKey)" }
ContentType = 'application/json; charset=utf-8'
Body = [Text.Encoding]::UTF8.GetBytes($json)
TimeoutSec = 30
}
Invoke-RestMethod @params | Out-Null
}
# ---------- Main ----------
try {
$Config = Get-Content -Path $ConfigPath -Raw | ConvertFrom-Json
foreach ($key in 'ApiUrl', 'ApiKey', 'Tenant') {
if (-not $Config.$key) { throw "Config $ConfigPath is missing '$key'" }
}
$SourceSystem = if ($Config.SourceSystem) { [string]$Config.SourceSystem } else { 'Veeam B&R' }
if ($LookbackHours -le 0) {
$LookbackHours = if ($Config.LookbackHours) { [int]$Config.LookbackHours } else { 48 }
}
# Windows PowerShell 5.1 may default to TLS 1.0
[Net.ServicePointManager]::SecurityProtocol = [Net.ServicePointManager]::SecurityProtocol -bor [Net.SecurityProtocolType]::Tls12
Import-Veeam
# Session ID -> start time (Unix seconds), for sessions already delivered in their final state.
# Numbers rather than ISO strings: PowerShell 7's ConvertFrom-Json turns date strings into
# DateTimes and loses the offset.
$sent = @{}
if (Test-Path $StatePath) {
$saved = Get-Content -Path $StatePath -Raw | ConvertFrom-Json
if ($saved) { foreach ($p in $saved.PSObject.Properties) { $sent[$p.Name] = [long]$p.Value } }
}
$since = (Get-Date).AddHours(-$LookbackHours)
$jobs = @(Get-VBRJob -WarningAction SilentlyContinue)
$jobIds = @{}
foreach ($j in $jobs) { $jobIds[$j.Id.ToString()] = $true }
# Get-VBRBackupSession returns every session in the database, so fetch once and bucket by job
$sessionsByJob = @{}
$policyByWorker = @{} # worker job ID -> policy job ID ('' if it isn't a worker of a listed job)
foreach ($s in Get-VBRBackupSession -WarningAction SilentlyContinue) {
if ($s.CreationTime -lt $since) { continue }
$key = $s.JobId.ToString()
if (-not $jobIds.ContainsKey($key)) {
if (-not $policyByWorker.ContainsKey($key)) { $policyByWorker[$key] = Get-PolicyJobId $s $jobIds }
if (-not $policyByWorker[$key]) { continue }
$key = $policyByWorker[$key]
}
if (-not $sessionsByJob.ContainsKey($key)) { $sessionsByJob[$key] = New-Object 'System.Collections.Generic.List[object]' }
$sessionsByJob[$key].Add($s)
}
} catch {
Write-Log "Collector setup failed: $(Get-ErrorText $_)" 'ERROR'
exit 2
}
$failures = 0
$runsSent = 0
foreach ($job in $jobs) {
$jobInfo = [ordered]@{
external_id = $job.Id.ToString()
name = $job.Name
# Always sent (null when unscheduled/disabled) so the dashboard's value tracks Veeam's
next_run_at = Get-NextRun $job
}
$pending = @()
if ($sessionsByJob.ContainsKey($jobInfo.external_id)) {
$pending = @($sessionsByJob[$jobInfo.external_id] |
Where-Object { -not $sent.ContainsKey($_.Id.ToString()) } |
Sort-Object CreationTime)
}
try {
if ($pending.Count -eq 0) {
# Keep next_run_at fresh even when nothing new ran
Send-Report ([ordered]@{ tenant = $Config.Tenant; source_system = $SourceSystem; job = $jobInfo })
}
foreach ($session in $pending) {
$run = ConvertTo-Run $session $job
Send-Report ([ordered]@{ tenant = $Config.Tenant; source_system = $SourceSystem; job = $jobInfo; run = $run })
$runsSent++
if ($run.status -ne 'running') {
$sent[$run.external_id] = ([DateTimeOffset][datetime]$session.CreationTime).ToUnixTimeSeconds()
}
}
} catch {
$failures++
Write-Log "Failed to report job '$($job.Name)': $(Get-ErrorText $_)" 'ERROR'
$httpStatus = 0
try { $httpStatus = [int]$_.Exception.Response.StatusCode } catch { }
if ($httpStatus -eq 401) {
Write-Log 'API key rejected; check ApiKey in the config. Stopping.' 'ERROR'
break
}
}
}
if (-not $DryRun) {
# Forget sessions that have aged out of the lookback window (plus a day of slack)
$cutoff = [DateTimeOffset]::Now.AddHours(-$LookbackHours - 24).ToUnixTimeSeconds()
$keep = [ordered]@{}
foreach ($id in $sent.Keys) {
if ($sent[$id] -ge $cutoff) { $keep[$id] = $sent[$id] }
}
try {
$keep | ConvertTo-Json -Compress | Set-Content -Path $StatePath -Encoding UTF8
} catch {
Write-Log "Couldn't save state file: $(Get-ErrorText $_)" 'WARN'
}
}
Write-Log ("Reported {0} jobs, {1} runs, {2} failures" -f $jobs.Count, $runsSent, $failures)
if ($failures) { exit 1 }
+83
View File
@@ -0,0 +1,83 @@
# Stand-ins for the Veeam cmdlets, shaped like the real objects' properties the collector reads.
# Dot-source this before running the collector to try it without a VBR server:
#
# . ./tests/MockVeeam.ps1
# ./Send-VeeamBackupStatus.ps1 -ConfigPath ./tests/config.json -DryRun
# Fixed to the top of the hour, and IDs derived from names/times, so repeated runs see the same
# jobs and sessions, like a real server
$now = Get-Date -Minute 0 -Second 0 -Millisecond 0
function New-StableGuid([string]$Seed) {
[guid]::new([Security.Cryptography.MD5]::Create().ComputeHash([Text.Encoding]::UTF8.GetBytes($Seed)))
}
$notSet = Get-Date '1900-01-01'
function New-MockJob($Name, $Type, $NextRun, [bool]$Enabled = $true) {
[pscustomobject]@{
Id = New-StableGuid $Name; Name = $Name; JobType = $Type
IsScheduleEnabled = $Enabled
ScheduleOptions = [pscustomobject]@{ NextRun = $NextRun }
}
}
function New-MockSession($Job, $Start, $End, $State, $Result, $Reason = '', $Tasks = @(), $LogRecords = @(), $Size = 0) {
$s = [pscustomobject]@{
Id = New-StableGuid "$($Job.Name)|$($Start.Ticks)"; JobId = $Job.Id; JobName = $Job.Name
CreationTime = $Start; EndTime = $End; State = $State; Result = $Result
IsRetryMode = $false
Info = [pscustomobject]@{ Reason = $Reason }
BackupStats = [pscustomobject]@{ BackupSize = $Size; DataSize = $Size * 3 }
Logger = [pscustomobject]@{ Records = $LogRecords }
Tasks = $Tasks
}
$s.Logger | Add-Member ScriptMethod GetLog { [pscustomobject]@{ UpdatedRecords = $this.Records } }
$s | Add-Member ScriptMethod GetTaskSessions { $this.Tasks }
$s
}
function New-MockTask($Name, $Status, $Reason = '') {
[pscustomobject]@{ Name = $Name; Status = $Status; Info = [pscustomobject]@{ Reason = $Reason } }
}
$global:MockJobs = @(
(New-MockJob 'File Server – Nightly' 'Backup' $now.Date.AddDays(1).AddHours(22)),
# Older VBR versions return NextRun as a culture-formatted string
(New-MockJob 'SQL01 – Logs' 'Backup' $now.AddMinutes(40).ToString()),
(New-MockJob 'Offsite Copy' 'BackupSync' $null),
(New-MockJob 'Old Exchange' 'Backup' $now.AddDays(1) $false),
# VBR 12+ backup copy job (immediate mode, no schedule): the "policy" job Get-VBRJob returns
(New-MockJob 'Cloud Copy' 'SimpleBackupCopyPolicy' $null)
)
$fs, $sql, $copy, $old, $cloud = $global:MockJobs
# Its hidden worker job, one per source job. Get-VBRJob doesn't return it, and Get-VBRBackupSession
# returns only the worker's sessions. Shape taken from a real VBR 13 server.
$cloudWorker = [pscustomobject]@{
Id = New-StableGuid 'Cloud Copy\File Server – Nightly'; Name = 'Cloud Copy\File Server – Nightly'
JobType = 'SimpleBackupCopyWorker'; PolicyJobId = $cloud.Id
}
function New-MockWorkerSession($Start, $End, $Result) {
$s = New-MockSession $cloudWorker $Start $End 'Stopped' $Result -Size 40GB
$s | Add-Member ScriptMethod GetJob { $cloudWorker }
$s
}
$global:MockSessions = @(
(New-MockSession $fs $now.AddDays(-1).AddHours(-2) $now.AddDays(-1).AddHours(-1) 'Stopped' 'Success' -Size 128GB -Tasks @(
(New-MockTask 'FS01' 'Success'), (New-MockTask 'FS02' 'Success'))),
(New-MockSession $fs $now.AddMinutes(-20) $notSet 'Working' 'None' -Tasks @((New-MockTask 'FS01' 'InProgress'))),
(New-MockSession $sql $now.AddHours(-1) $now.AddMinutes(-55) 'Stopped' 'Warning' -Size 2GB `
-Tasks @((New-MockTask 'SQL01' 'Warning' 'Failed to truncate transaction logs for database "ERP_PROD"'))),
(New-MockSession $copy $now.AddHours(-5) $now.AddHours(-4) 'Stopped' 'Failed' 'Error: Unable to connect to repository NAS02' `
-LogRecords @(
[pscustomobject]@{ Title = 'Preparing backup chain'; Status = 'ESucceeded' },
[pscustomobject]@{ Title = 'Network path not found (0x80070035)'; Status = 'EFailed' })),
(New-MockSession $old $now.AddDays(-10) $now.AddDays(-10).AddHours(1) 'Stopped' 'Success'), # outside 48h lookback
(New-MockWorkerSession $now.AddDays(-1).AddMinutes(-55) $now.AddDays(-1).AddMinutes(-30) 'Success'),
(New-MockWorkerSession $now.AddMinutes(-55) $now.AddMinutes(-35) 'Warning'),
# A session from a job the collector doesn't know (e.g. deleted): must be ignored, not crash
(New-MockSession ([pscustomobject]@{ Id = New-StableGuid 'gone'; Name = 'Deleted job' }) $now.AddHours(-2) $now.AddHours(-1) 'Stopped' 'Success')
)
function Get-VBRJob { [CmdletBinding()] param() $global:MockJobs }
function Get-VBRBackupSession { [CmdletBinding()] param() $global:MockSessions }
+23
View File
@@ -0,0 +1,23 @@
[Unit]
Description=Backup Dashboard API
After=network.target
[Service]
User=backupdash
Group=backupdash
WorkingDirectory=/opt/backup-dashboard/backend
# Settings live in this file so manage.py reads exactly the same ones (see deploy/backup-dashboard.env)
EnvironmentFile=/etc/backup-dashboard/backup-dashboard.env
# Bound to localhost: only nginx can reach it, so nginx's auth can't be bypassed from outside.
# One worker is plenty at this scale and keeps SQLite writes simple.
ExecStart=/opt/backup-dashboard/venv/bin/uvicorn --factory app.main:create_app --host 127.0.0.1 --port 8000 --proxy-headers
Restart=on-failure
StateDirectory=backup-dashboard
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
PrivateTmp=true
[Install]
WantedBy=multi-user.target
+4
View File
@@ -0,0 +1,4 @@
# Install as /etc/backup-dashboard/backup-dashboard.env
# Read by both the API service and manage.py, so they always use the same database.
BACKUP_DASH_DB=/var/lib/backup-dashboard/backup-dashboard.db
BACKUP_DASH_RETENTION_DAYS=90
+52
View File
@@ -0,0 +1,52 @@
server {
listen 80;
server_name backups.example.com;
# Basic auth sends the password with every request — never serve this over plain HTTP
return 301 https://$host$request_uri;
}
server {
listen 443 ssl;
http2 on;
server_name backups.example.com;
# Paths as created by `certbot certonly --nginx -d backups.example.com`
ssl_certificate /etc/letsencrypt/live/backups.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/backups.example.com/privkey.pem;
root /var/www/backup-dashboard/public;
index index.html;
# Strict CSP works because there is no inline JS/CSS
add_header Content-Security-Policy "default-src 'self'; object-src 'none'; base-uri 'self'; frame-ancestors 'none'" always;
add_header X-Content-Type-Options "nosniff" always;
add_header Referrer-Policy "no-referrer" always;
add_header Strict-Transport-Security "max-age=31536000" always;
# Dashboard users. Create with: htpasswd -B -c /etc/nginx/backup-dashboard.htpasswd <user>
# To move to Microsoft 365 sign-in later, replace these two lines with an auth_request to
# oauth2-proxy (provider=entra-id); the app itself doesn't need to change.
auth_basic "Backup Monitor";
auth_basic_user_file /etc/nginx/backup-dashboard.htpasswd;
location / {
try_files $uri $uri/ /index.html;
}
location /api/ {
proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
# Collectors authenticate with their own API key (Authorization: Bearer …), checked by the app
location = /api/v1/ingest {
auth_basic off;
client_max_body_size 256k;
proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
+341
View File
@@ -0,0 +1,341 @@
:root {
color-scheme: light dark;
--bg: #f6f7f9;
--surface: #ffffff;
--surface-alt: #f0f2f5;
--border: #d9dde3;
--text: #1f2328;
--muted: #656d76;
--accent: #0969da;
/* Status palette: each has fg/bg/border so badges stay legible in both themes */
--success-fg: #116329; --success-bg: #dafbe1; --success-bd: #4ac26b;
--warning-fg: #7d4e00; --warning-bg: #fff8c5; --warning-bd: #d4a72c;
--failed-fg: #a40e26; --failed-bg: #ffebe9; --failed-bd: #ff8182;
--unknown-fg: #57606a; --unknown-bg: #eaeef2; --unknown-bd: #afb8c1;
--running-fg: #0550ae; --running-bg: #ddf4ff; --running-bd: #54aeff;
--radius: 8px;
--shadow: 0 1px 2px rgb(0 0 0 / 6%), 0 4px 12px rgb(0 0 0 / 4%);
font-family: system-ui, -apple-system, "Segoe UI", Roboto, sans-serif;
font-size: 15px;
line-height: 1.45;
}
@media (prefers-color-scheme: dark) {
:root {
--bg: #0d1117;
--surface: #161b22;
--surface-alt: #1c2128;
--border: #30363d;
--text: #e6edf3;
--muted: #8d96a0;
--accent: #4493f8;
--success-fg: #6fdd8b; --success-bg: #12261e; --success-bd: #2ea043;
--warning-fg: #f2cc60; --warning-bg: #272115; --warning-bd: #9e6a03;
--failed-fg: #ffa198; --failed-bg: #2d1214; --failed-bd: #da3633;
--unknown-fg: #9da7b3; --unknown-bg: #21262d; --unknown-bd: #484f58;
--running-fg: #79c0ff; --running-bg: #0c2d4b; --running-bd: #1f6feb;
--shadow: none;
}
}
*, *::before, *::after { box-sizing: border-box; }
body {
margin: 0;
background: var(--bg);
color: var(--text);
}
.muted { color: var(--muted); }
.visually-hidden {
position: absolute !important;
width: 1px; height: 1px;
overflow: hidden;
clip: rect(0 0 0 0);
white-space: nowrap;
}
/* ---------- Layout ---------- */
.topbar {
display: flex;
align-items: center;
justify-content: space-between;
gap: 1rem;
flex-wrap: wrap;
padding: 0.75rem 1.5rem;
background: var(--surface);
border-bottom: 1px solid var(--border);
}
.topbar__title { margin: 0; font-size: 1.25rem; }
.topbar__meta { display: flex; align-items: center; gap: 0.75rem; font-size: 0.875rem; }
.container {
max-width: 1400px;
margin: 0 auto;
padding: 1.25rem 1.5rem 3rem;
}
/* ---------- Controls ---------- */
.btn {
font: inherit;
padding: 0.4rem 0.9rem;
border: 1px solid var(--border);
border-radius: 6px;
background: var(--surface-alt);
color: var(--text);
cursor: pointer;
}
.btn:hover { border-color: var(--muted); }
.btn:disabled { opacity: 0.6; cursor: progress; }
.btn--icon { padding: 0.25rem 0.55rem; line-height: 1; }
.input {
font: inherit;
width: min(100%, 360px);
padding: 0.45rem 0.75rem;
border: 1px solid var(--border);
border-radius: 6px;
background: var(--surface);
color: var(--text);
}
:focus-visible { outline: 2px solid var(--accent); outline-offset: 2px; }
.toolbar { margin: 1rem 0; }
/* ---------- Banner ---------- */
.banner {
padding: 0.75rem 1rem;
border-radius: var(--radius);
margin-bottom: 1rem;
border: 1px solid;
}
.banner--error { background: var(--failed-bg); color: var(--failed-fg); border-color: var(--failed-bd); }
/* ---------- Summary cards ---------- */
.summary {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(150px, 1fr));
gap: 0.75rem;
}
.summary-card {
display: flex;
flex-direction: column;
align-items: flex-start;
gap: 0.15rem;
padding: 0.9rem 1rem;
font: inherit;
text-align: left;
color: var(--text);
background: var(--surface);
border: 1px solid var(--border);
border-left-width: 4px;
border-radius: var(--radius);
box-shadow: var(--shadow);
cursor: pointer;
}
.summary-card[aria-pressed="true"] { outline: 2px solid var(--accent); outline-offset: -1px; }
.summary-card__count { font-size: 1.75rem; font-weight: 600; font-variant-numeric: tabular-nums; }
.summary-card__label { font-size: 0.85rem; color: var(--muted); }
.summary-card--failed { border-left-color: var(--failed-bd); }
.summary-card--warning { border-left-color: var(--warning-bd); }
.summary-card--running { border-left-color: var(--running-bd); }
.summary-card--success { border-left-color: var(--success-bd); }
.summary-card--overdue { border-left-color: var(--unknown-bd); }
/* ---------- Table ---------- */
.table-wrap {
overflow-x: auto;
background: var(--surface);
border: 1px solid var(--border);
border-radius: var(--radius);
box-shadow: var(--shadow);
}
.jobs { width: 100%; border-collapse: collapse; }
.jobs th, .jobs td {
padding: 0.65rem 1rem;
text-align: left;
border-bottom: 1px solid var(--border);
white-space: nowrap;
}
.jobs tbody tr:last-child td { border-bottom: none; }
.jobs tbody tr:hover { background: var(--surface-alt); }
.jobs th { background: var(--surface-alt); font-size: 0.8rem; text-transform: uppercase; letter-spacing: 0.03em; }
.sort {
all: unset;
cursor: pointer;
color: var(--muted);
font-weight: 600;
}
.sort::after { content: " ↕"; opacity: 0.4; }
th[aria-sort="ascending"] .sort::after { content: " ↑"; opacity: 1; }
th[aria-sort="descending"] .sort::after { content: " ↓"; opacity: 1; }
th[aria-sort="ascending"] .sort,
th[aria-sort="descending"] .sort { color: var(--text); }
.time { display: flex; flex-direction: column; }
.time__rel { font-size: 0.8rem; color: var(--muted); }
.empty { padding: 2rem; text-align: center; color: var(--muted); margin: 0; }
/* ---------- Status badges ---------- */
.badge {
display: inline-flex;
align-items: center;
gap: 0.35rem;
padding: 0.2rem 0.65rem;
font: inherit;
font-size: 0.85rem;
font-weight: 600;
border: 1px solid;
border-radius: 999px;
}
button.badge { cursor: pointer; }
button.badge:hover { filter: brightness(0.95); }
/* Icon + text, not colour alone, so it's readable for colour-blind users */
.badge__icon { font-size: 0.75rem; }
.badge--success { color: var(--success-fg); background: var(--success-bg); border-color: var(--success-bd); }
.badge--warning { color: var(--warning-fg); background: var(--warning-bg); border-color: var(--warning-bd); }
.badge--failed { color: var(--failed-fg); background: var(--failed-bg); border-color: var(--failed-bd); }
.badge--unknown { color: var(--unknown-fg); background: var(--unknown-bg); border-color: var(--unknown-bd); }
.badge--running { color: var(--running-fg); background: var(--running-bg); border-color: var(--running-bd); }
@media (prefers-reduced-motion: no-preference) {
.badge--running .badge__icon { animation: spin 2s linear infinite; }
}
@keyframes spin { to { transform: rotate(360deg); } }
.tag-overdue {
margin-left: 0.5rem;
padding: 0.05rem 0.45rem;
font-size: 0.75rem;
font-weight: 600;
border-radius: 4px;
color: var(--failed-fg);
background: var(--failed-bg);
}
/* ---------- Modal ---------- */
.modal {
width: min(720px, calc(100vw - 2rem));
max-height: calc(100vh - 4rem);
padding: 0;
color: var(--text);
background: var(--surface);
border: 1px solid var(--border);
border-radius: 12px;
}
.modal::backdrop { background: rgb(0 0 0 / 45%); }
.modal__header {
display: flex;
justify-content: space-between;
align-items: flex-start;
gap: 1rem;
padding: 1rem 1.25rem;
border-bottom: 1px solid var(--border);
}
.modal__title { margin: 0; font-size: 1.15rem; }
.modal__header p { margin: 0.15rem 0 0; }
.modal__body { padding: 1rem 1.25rem 1.25rem; }
.detail-grid {
display: grid;
grid-template-columns: max-content 1fr;
gap: 0.4rem 1.25rem;
margin: 1rem 0;
}
.detail-grid dt { color: var(--muted); }
.detail-grid dd { margin: 0; overflow-wrap: anywhere; }
.detail-section-title { font-size: 0.85rem; text-transform: uppercase; color: var(--muted); margin: 1.25rem 0 0.5rem; }
.message {
margin: 0;
padding: 0.75rem 1rem;
max-height: 320px;
overflow: auto;
white-space: pre-wrap; /* preserve backup-software line breaks */
overflow-wrap: anywhere;
font-family: ui-monospace, SFMono-Regular, Consolas, monospace;
font-size: 0.85rem;
background: var(--surface-alt);
border: 1px solid var(--border);
border-left-width: 4px;
border-radius: 6px;
}
.message--failed { border-left-color: var(--failed-bd); }
.message--warning { border-left-color: var(--warning-bd); }
.message--success { border-left-color: var(--success-bd); }
.message--running { border-left-color: var(--running-bd); }
.detail-status { display: flex; align-items: center; gap: 0.6rem; flex-wrap: wrap; }
.link {
all: unset;
color: var(--accent);
cursor: pointer;
text-decoration: underline;
}
/* ---------- Run history ---------- */
.history-stats { margin: 0 0 0.6rem; font-size: 0.9rem; }
/* One cell per day, oldest → today. Colour + tooltip; the runs table below is the accessible detail. */
.day-strip {
display: grid;
grid-template-columns: repeat(30, 1fr);
gap: 3px;
}
.day {
height: 28px;
padding: 0;
border: 1px solid var(--border);
border-radius: 3px;
background: var(--surface-alt);
cursor: pointer;
}
.day:disabled { cursor: default; opacity: 0.6; }
.day:not(:disabled):hover { filter: brightness(0.9); }
.day[aria-pressed="true"] { outline: 2px solid var(--accent); outline-offset: 1px; }
.day--success { background: var(--success-bd); border-color: var(--success-bd); }
.day--warning { background: var(--warning-bd); border-color: var(--warning-bd); }
.day--failed { background: var(--failed-bd); border-color: var(--failed-bd); }
.day--running { background: var(--running-bd); border-color: var(--running-bd); }
.day--unknown { background: var(--unknown-bd); border-color: var(--unknown-bd); }
.day-strip__ticks {
display: flex;
justify-content: space-between;
margin-top: 0.25rem;
font-size: 0.75rem;
}
.history-filter { margin: 0.75rem 0 0; font-size: 0.85rem; }
.runs-wrap {
margin-top: 0.75rem;
max-height: 300px;
overflow: auto;
border: 1px solid var(--border);
border-radius: 6px;
}
.runs { width: 100%; border-collapse: collapse; font-size: 0.85rem; }
.runs th, .runs td {
padding: 0.4rem 0.75rem;
text-align: left;
white-space: nowrap;
border-bottom: 1px solid var(--border);
}
.runs th {
position: sticky;
top: 0;
background: var(--surface-alt);
color: var(--muted);
font-size: 0.75rem;
text-transform: uppercase;
}
.runs tbody tr:last-child td { border-bottom: none; }
.runs tr.is-selected td { background: color-mix(in srgb, var(--accent) 16%, transparent); }
.runs .badge { font-size: 0.75rem; padding: 0.1rem 0.5rem; }
+85
View File
@@ -0,0 +1,85 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Backup Monitor</title>
<link rel="stylesheet" href="/css/styles.css">
<!-- type="module" is deferred by default; no inline JS so a strict CSP works -->
<script type="module" src="/js/app.js"></script>
</head>
<body>
<header class="topbar">
<h1 class="topbar__title">Backup Monitor</h1>
<div class="topbar__meta">
<span id="last-updated" class="muted" aria-live="polite">Loading…</span>
<button id="refresh-btn" class="btn" type="button">Refresh</button>
</div>
</header>
<main class="container">
<div id="error-banner" class="banner banner--error" role="alert" hidden></div>
<!-- Summary cards double as status filters -->
<section class="summary" aria-label="Status summary and filters">
<button type="button" class="summary-card" data-filter="all" aria-pressed="true">
<span class="summary-card__count" data-count="all">–</span>
<span class="summary-card__label">All jobs</span>
</button>
<button type="button" class="summary-card summary-card--failed" data-filter="failed" aria-pressed="false">
<span class="summary-card__count" data-count="failed">–</span>
<span class="summary-card__label">Failed</span>
</button>
<button type="button" class="summary-card summary-card--warning" data-filter="warning" aria-pressed="false">
<span class="summary-card__count" data-count="warning">–</span>
<span class="summary-card__label">Warning</span>
</button>
<button type="button" class="summary-card summary-card--running" data-filter="running" aria-pressed="false">
<span class="summary-card__count" data-count="running">–</span>
<span class="summary-card__label">Running</span>
</button>
<button type="button" class="summary-card summary-card--success" data-filter="success" aria-pressed="false">
<span class="summary-card__count" data-count="success">–</span>
<span class="summary-card__label">Success</span>
</button>
<button type="button" class="summary-card summary-card--overdue" data-filter="overdue" aria-pressed="false">
<span class="summary-card__count" data-count="overdue">–</span>
<span class="summary-card__label">Overdue</span>
</button>
</section>
<div class="toolbar">
<label for="search" class="visually-hidden">Search jobs</label>
<input id="search" class="input" type="search" placeholder="Search company or job…" autocomplete="off">
</div>
<div class="table-wrap">
<table class="jobs">
<thead>
<tr>
<th scope="col" aria-sort="none"><button type="button" class="sort" data-sort="company">Company</button></th>
<th scope="col" aria-sort="none"><button type="button" class="sort" data-sort="name">Backup Job</button></th>
<th scope="col" aria-sort="ascending"><button type="button" class="sort" data-sort="status">Status</button></th>
<th scope="col" aria-sort="none"><button type="button" class="sort" data-sort="lastRun">Last Run</button></th>
<th scope="col" aria-sort="none"><button type="button" class="sort" data-sort="nextRun">Next Scheduled</button></th>
</tr>
</thead>
<tbody id="jobs-body"></tbody>
</table>
<p id="empty-state" class="empty" hidden>No backup jobs match the current filter.</p>
</div>
</main>
<!-- Native <dialog>: built-in focus trap, Esc-to-close, and top-layer rendering -->
<dialog id="detail-dialog" class="modal" aria-labelledby="detail-title">
<div class="modal__header">
<div>
<h2 id="detail-title" class="modal__title"></h2>
<p id="detail-subtitle" class="muted"></p>
</div>
<button type="button" class="btn btn--icon" data-close aria-label="Close">✕</button>
</div>
<div id="detail-body" class="modal__body"></div>
</dialog>
</body>
</html>
+61
View File
@@ -0,0 +1,61 @@
import { CONFIG } from './config.js';
export class ApiError extends Error {
constructor(message, status = null) {
super(message);
this.name = 'ApiError';
this.status = status;
}
}
async function request(path) {
const controller = new AbortController();
const timer = setTimeout(() => controller.abort(), CONFIG.requestTimeoutMs);
try {
const res = await fetch(`${CONFIG.apiBaseUrl}${path}`, {
headers: { Accept: 'application/json' },
credentials: 'same-origin', // ready for cookie/session auth behind nginx
signal: controller.signal,
});
// 401 means the nginx/SSO session lapsed; a reload sends the user back through sign-in
if (res.status === 401) throw new ApiError('Your session has expired. Reload the page to sign in again', 401);
if (!res.ok) throw new ApiError(`Server returned HTTP ${res.status}`, res.status);
return await res.json();
} catch (err) {
if (err.name === 'AbortError') throw new ApiError('Request timed out');
if (err instanceof ApiError) throw err;
throw new ApiError(`Network error: ${err.message}`);
} finally {
clearTimeout(timer);
}
}
const httpApi = {
/** GET /jobs → { generated_at, jobs: Job[] } */
async listJobs() {
const data = await request('/jobs');
if (!Array.isArray(data?.jobs)) throw new ApiError('Unexpected response shape from /jobs');
return data;
},
/** GET /jobs/:id → JobDetail */
getJob(id) {
return request(`/jobs/${encodeURIComponent(id)}`);
},
/** GET /jobs/:id/runs?days=N → { job_id, since, runs: RunSummary[] } (newest first, no messages) */
listRuns(id, days) {
return request(`/jobs/${encodeURIComponent(id)}/runs?days=${encodeURIComponent(days)}`);
},
/** GET /runs/:id → Run (with message and details) */
getRun(id) {
return request(`/runs/${encodeURIComponent(id)}`);
},
};
// Mock is loaded lazily so it never ships in the production request path.
export async function getApi() {
if (CONFIG.useMock) {
const { mockApi } = await import('./mock.js');
return mockApi;
}
return httpApi;
}
+527
View File
@@ -0,0 +1,527 @@
import { CONFIG } from './config.js';
import { getApi } from './api.js';
import {
parseDate, formatDateTime, formatTime, formatRelative, formatDuration, formatBytes,
} from './format.js';
const api = await getApi();
// ---------- Domain ----------
// rank drives default sort: worst first
const STATUS = {
failed: { label: 'Failed', icon: '✕', rank: 0 },
warning: { label: 'Warning', icon: '!', rank: 1 },
unknown: { label: 'Unknown', icon: '?', rank: 2 },
running: { label: 'Running', icon: '↻', rank: 3 },
success: { label: 'Success', icon: '✓', rank: 4 },
};
// Anything the backend sends that we don't recognise becomes "unknown" rather than breaking the UI
function normalizeStatus(raw) {
const s = String(raw ?? '').toLowerCase();
return Object.hasOwn(STATUS, s) ? s : 'unknown';
}
function normalizeJob(j) {
return {
id: String(j.id),
company: j.company_name ?? '',
name: j.job_name ?? '',
status: normalizeStatus(j.status),
message: j.message ?? '',
runningSince: parseDate(j.running_since),
// When the most recent run *finished*; a job that's running keeps showing its previous finish
lastRun: parseDate(j.last_run_at),
nextRun: parseDate(j.next_run_at),
};
}
// A running job isn't silently dead, so it's never overdue (a long run can pass its next slot)
const isOverdue = (job, now = Date.now()) =>
job.status !== 'running' && !!job.nextRun && job.nextRun.getTime() + CONFIG.overdueGraceMinutes * 60_000 < now;
// ---------- State ----------
const state = {
jobs: [],
filter: 'all',
search: '',
sort: { key: 'status', dir: 1 },
lastSuccess: null,
};
// ---------- DOM ----------
const $ = (sel) => document.querySelector(sel);
const dom = {
tbody: $('#jobs-body'),
empty: $('#empty-state'),
search: $('#search'),
refreshBtn: $('#refresh-btn'),
lastUpdated: $('#last-updated'),
errorBanner: $('#error-banner'),
dialog: $('#detail-dialog'),
detailTitle: $('#detail-title'),
detailSubtitle: $('#detail-subtitle'),
detailBody: $('#detail-body'),
};
// Tiny element builder. All text goes through append() → text nodes, so API data can never inject HTML.
function el(tag, props = {}, ...children) {
const node = document.createElement(tag);
for (const [k, v] of Object.entries(props)) {
if (v == null || v === false) continue;
if (k === 'class') node.className = v;
else if (k === 'dataset') Object.assign(node.dataset, v);
else node.setAttribute(k, v === true ? '' : v);
}
node.append(...children.flat().filter((c) => c != null && c !== false));
return node;
}
// ---------- Rendering ----------
function statusBadge(status, { asButton = false, opensDialog = false, dataset = null, title = null } = {}) {
const { label, icon } = STATUS[status];
const content = [el('span', { class: 'badge__icon', 'aria-hidden': 'true' }, icon), label];
return asButton
? el('button', {
type: 'button',
class: `badge badge--${status}`,
'aria-haspopup': opensDialog ? 'dialog' : null,
'aria-label': `${label} – view details`,
title,
dataset,
}, content)
: el('span', { class: `badge badge--${status}`, title }, content);
}
const runningTitle = (since) => (since ? `Running since ${formatDateTime(since)}` : null);
function timeCell(date, extra = null) {
if (!date) return el('td', {}, el('span', { class: 'muted' }, 'Never'));
return el('td', {},
el('time', { class: 'time', datetime: date.toISOString(), title: date.toString() },
el('span', {}, formatDateTime(date), extra),
el('span', { class: 'time__rel' }, formatRelative(date))));
}
function renderRow(job) {
const overdueTag = isOverdue(job) ? el('span', { class: 'tag-overdue' }, 'Overdue') : null;
return el('tr', {},
el('td', {}, job.company),
el('td', {}, job.name),
el('td', {}, statusBadge(job.status, {
asButton: true, opensDialog: true, dataset: { jobId: job.id }, title: runningTitle(job.runningSince),
})),
timeCell(job.lastRun),
timeCell(job.nextRun, overdueTag));
}
const collator = new Intl.Collator(undefined, { sensitivity: 'base', numeric: true });
function cmpDate(a, b) {
if (!a && !b) return 0;
if (!a) return 1;
if (!b) return -1;
return a - b;
}
const SORTERS = {
company: (a, b) => collator.compare(a.company, b.company),
name: (a, b) => collator.compare(a.name, b.name),
status: (a, b) => STATUS[a.status].rank - STATUS[b.status].rank,
lastRun: (a, b) => cmpDate(a.lastRun, b.lastRun),
nextRun: (a, b) => cmpDate(a.nextRun, b.nextRun),
};
function visibleJobs() {
const q = state.search.trim().toLowerCase();
const now = Date.now();
const { key, dir } = state.sort;
return state.jobs
.filter((j) => {
if (state.filter === 'overdue') return isOverdue(j, now);
return state.filter === 'all' || j.status === state.filter;
})
.filter((j) => !q || j.company.toLowerCase().includes(q) || j.name.toLowerCase().includes(q))
// Stable tie-breakers keep row order from jumping around on each refresh
.sort((a, b) => SORTERS[key](a, b) * dir || SORTERS.company(a, b) || SORTERS.name(a, b));
}
function renderSummary() {
const now = Date.now();
const counts = { all: state.jobs.length, failed: 0, warning: 0, running: 0, success: 0, overdue: 0 };
for (const j of state.jobs) {
if (j.status in counts) counts[j.status]++;
if (isOverdue(j, now)) counts.overdue++;
}
for (const [k, v] of Object.entries(counts)) {
const node = document.querySelector(`[data-count="${k}"]`);
if (node) node.textContent = v;
}
document.querySelectorAll('.summary-card').forEach((card) => {
card.setAttribute('aria-pressed', String(card.dataset.filter === state.filter));
});
}
function renderSortHeaders() {
document.querySelectorAll('th[aria-sort]').forEach((th) => {
const key = th.querySelector('.sort').dataset.sort;
th.setAttribute('aria-sort',
key === state.sort.key ? (state.sort.dir === 1 ? 'ascending' : 'descending') : 'none');
});
}
function render() {
const rows = visibleJobs();
dom.tbody.replaceChildren(...rows.map(renderRow));
dom.empty.hidden = rows.length > 0;
renderSummary();
renderSortHeaders();
}
// ---------- Data loading ----------
let inFlight = null;
let pollTimer = null;
function refresh() {
if (inFlight) return inFlight; // de-dupe overlapping manual + scheduled refreshes
dom.refreshBtn.disabled = true;
inFlight = (async () => {
try {
const data = await api.listJobs();
state.jobs = data.jobs.map(normalizeJob);
state.lastSuccess = new Date();
dom.errorBanner.hidden = true;
dom.lastUpdated.textContent = `Updated ${formatTime(state.lastSuccess)}`;
} catch (err) {
// Keep showing stale data, but make it obvious it's stale
dom.errorBanner.textContent = state.lastSuccess
? `Couldn't refresh (${err.message}). Showing data from ${formatTime(state.lastSuccess)}.`
: `Couldn't load backup status (${err.message}).`;
dom.errorBanner.hidden = false;
dom.lastUpdated.textContent = 'Update failed';
console.error(err);
} finally {
render();
dom.refreshBtn.disabled = false;
inFlight = null;
}
})();
return inFlight;
}
function schedulePoll() {
clearTimeout(pollTimer);
pollTimer = setTimeout(async () => {
await refresh();
schedulePoll();
}, CONFIG.refreshIntervalMs);
}
// Don't poll from background tabs (e.g. a wall display that's been switched away); catch up on return
document.addEventListener('visibilitychange', () => {
if (document.hidden) clearTimeout(pollTimer);
else refresh().then(schedulePoll);
});
// ---------- Detail modal ----------
// Every async result checks reqId first, so a slow response for job A can never paint over job B.
const modal = {
reqId: 0,
job: null,
detail: null,
runs: null, // null = loading, [] = none
runsError: null,
dayFilter: null, // local YYYY-MM-DD, or null for all days
selectedRunId: null, // null = the job's latest finished run (from the detail call)
runMessage: null, // { status, text, startedAt } for the selected run
runMessageError: null,
};
// Colour of a day cell: the worst thing that happened that day
const DAY_SEVERITY = ['failed', 'warning', 'unknown', 'running', 'success'];
const pad2 = (n) => String(n).padStart(2, '0');
const localDayKey = (d) => `${d.getFullYear()}-${pad2(d.getMonth() + 1)}-${pad2(d.getDate())}`;
const dayLabelFmt = new Intl.DateTimeFormat(undefined, { weekday: 'short', month: 'short', day: 'numeric' });
const dayTickFmt = new Intl.DateTimeFormat(undefined, { month: 'short', day: 'numeric' });
function normalizeRun(r) {
return {
id: String(r.id),
status: normalizeStatus(r.status),
startedAt: parseDate(r.started_at),
finishedAt: parseDate(r.finished_at),
duration: r.duration_seconds,
size: r.size_bytes,
};
}
function buildDays(runs, count) {
const today = new Date();
today.setHours(0, 0, 0, 0);
const days = [];
for (let i = count - 1; i >= 0; i--) {
const date = new Date(today);
date.setDate(date.getDate() - i);
days.push({ key: localDayKey(date), date, counts: {}, total: 0 });
}
const byKey = new Map(days.map((d) => [d.key, d]));
for (const run of runs) {
const day = byKey.get(localDayKey(run.startedAt));
if (!day) continue;
day.counts[run.status] = (day.counts[run.status] ?? 0) + 1;
day.total++;
}
for (const day of days) day.status = day.total ? DAY_SEVERITY.find((s) => day.counts[s]) : 'none';
return days;
}
function describeCounts(counts) {
return DAY_SEVERITY.filter((s) => counts[s])
.map((s) => `${counts[s]} ${STATUS[s].label.toLowerCase()}`)
.join(', ');
}
function detailRow(label, value) {
if (value == null || value === '') return [];
return [el('dt', {}, label), el('dd', {}, value)];
}
function renderSummarySection() {
const { job, detail } = modal;
const status = detail ? normalizeStatus(detail.status) : job.status;
const runningSince = detail ? parseDate(detail.running_since) : job.runningSince;
const lastStatus = detail?.last_status ? normalizeStatus(detail.last_status) : null;
dom.detailTitle.textContent = detail?.job_name ?? job.name;
dom.detailSubtitle.textContent = detail?.company_name ?? job.company;
return el('section', {},
el('div', { class: 'detail-status' },
statusBadge(status),
runningSince ? el('span', { class: 'muted' }, `since ${formatDateTime(runningSince)}`) : null),
el('dl', { class: 'detail-grid' },
// While a job is running, its "last run" is the previous one — say how that went
status === 'running' && lastStatus ? detailRow('Previous result', STATUS[lastStatus].label) : [],
detailRow('Last finished', formatDateTime(detail ? parseDate(detail.last_run_at) : job.lastRun)),
detailRow('Next scheduled', formatDateTime(detail ? parseDate(detail.next_run_at) : job.nextRun)),
detailRow('Duration', formatDuration(detail?.duration_seconds)),
detailRow('Size', formatBytes(detail?.size_bytes)),
detailRow('Source', detail?.source_system)));
}
function renderHistorySection() {
const section = el('section', { id: 'detail-history' },
el('h3', { class: 'detail-section-title' }, `Last ${CONFIG.historyDays} days`));
if (modal.runsError) {
section.append(el('p', { class: 'banner banner--error' }, `Couldn't load run history: ${modal.runsError}`));
return section;
}
if (!modal.runs) {
section.append(el('p', { class: 'muted' }, 'Loading history…'));
return section;
}
const days = buildDays(modal.runs, CONFIG.historyDays);
const finished = modal.runs.filter((r) => r.finishedAt);
const ok = finished.filter((r) => r.status === 'success').length;
const totals = {};
for (const r of modal.runs) totals[r.status] = (totals[r.status] ?? 0) + 1;
section.append(
el('p', { class: 'history-stats' },
modal.runs.length
? `${modal.runs.length} runs · ${finished.length ? Math.round((ok / finished.length) * 100) : 0}% successful · ${describeCounts(totals)}`
: 'No runs recorded in this period.'),
el('div', { class: 'day-strip', role: 'group', 'aria-label': 'Daily results, oldest to newest. Select a day to filter runs.' },
days.map((d) => {
const label = `${dayLabelFmt.format(d.date)}: ${d.total ? describeCounts(d.counts) : 'no runs'}`;
return el('button', {
type: 'button',
class: `day day--${d.status}`,
title: label,
'aria-label': label,
'aria-pressed': String(modal.dayFilter === d.key),
disabled: !d.total,
dataset: { day: d.key },
});
})),
el('div', { class: 'day-strip__ticks muted', 'aria-hidden': 'true' },
el('span', {}, dayTickFmt.format(days[0].date)),
el('span', {}, 'Today')),
);
const rows = modal.dayFilter
? modal.runs.filter((r) => localDayKey(r.startedAt) === modal.dayFilter)
: modal.runs;
if (!rows.length) return section;
if (modal.dayFilter) {
const date = days.find((d) => d.key === modal.dayFilter)?.date;
section.append(el('p', { class: 'muted history-filter' },
`Showing ${dayLabelFmt.format(date)} · `,
el('button', { type: 'button', class: 'link', dataset: { day: modal.dayFilter } }, 'Show all days')));
}
section.append(el('div', { class: 'runs-wrap' },
el('table', { class: 'runs' },
el('thead', {}, el('tr', {},
['Status', 'Started', 'Finished', 'Duration', 'Size'].map((h) => el('th', { scope: 'col' }, h)))),
el('tbody', {}, rows.map((r) => el('tr', { class: r.id === modal.selectedRunId ? 'is-selected' : null },
el('td', {}, statusBadge(r.status, { asButton: true, dataset: { runId: r.id } })),
el('td', {}, formatDateTime(r.startedAt)),
el('td', {}, r.finishedAt ? formatDateTime(r.finishedAt) : el('span', { class: 'muted' }, 'In progress')),
el('td', {}, formatDuration(r.duration) ?? '—'),
el('td', {}, formatBytes(r.size) ?? '—')))))));
return section;
}
function renderMessageSection() {
const { detail, job } = modal;
let status;
let text;
let heading = 'Message';
if (modal.selectedRunId) {
const run = modal.runs?.find((r) => r.id === modal.selectedRunId);
heading = `Message · run started ${formatDateTime(run?.startedAt)}`;
status = run?.status ?? 'unknown';
text = modal.runMessageError
? `Couldn't load this run: ${modal.runMessageError}`
: modal.runMessage?.text ?? 'Loading…';
} else {
status = detail?.last_status ? normalizeStatus(detail.last_status) : job.status;
text = detail?.message ?? job.message;
}
const extras = !modal.selectedRunId && detail?.details && typeof detail.details === 'object'
? Object.entries(detail.details).flatMap(([k, v]) =>
detailRow(k.replace(/_/g, ' '), typeof v === 'object' ? JSON.stringify(v) : String(v)))
: modal.runMessage?.extras ?? [];
return el('section', { id: 'detail-message' },
el('h3', { class: 'detail-section-title' }, heading),
el('pre', { class: `message message--${status}` }, text || 'No message reported.'),
extras.length ? el('h3', { class: 'detail-section-title' }, 'Additional details') : null,
extras.length ? el('dl', { class: 'detail-grid' }, extras) : null);
}
function renderModal({ loading = false, error = null } = {}) {
dom.detailBody.replaceChildren(...[
renderSummarySection(),
loading ? el('p', { class: 'muted' }, 'Loading full details…') : null,
error ? el('p', { class: 'banner banner--error' }, `Couldn't load full details: ${error}`) : null,
renderMessageSection(),
renderHistorySection(),
].filter(Boolean));
}
// Re-render just the history section, keeping the runs table's scroll position
function renderHistory() {
const wrap = dom.detailBody.querySelector('.runs-wrap');
const scrollTop = wrap?.scrollTop ?? 0;
dom.detailBody.querySelector('#detail-history')?.replaceWith(renderHistorySection());
const newWrap = dom.detailBody.querySelector('.runs-wrap');
if (newWrap) newWrap.scrollTop = scrollTop;
}
function renderMessage() {
dom.detailBody.querySelector('#detail-message')?.replaceWith(renderMessageSection());
}
async function openDetail(jobId) {
const job = state.jobs.find((j) => j.id === jobId);
if (!job) return;
const reqId = ++modal.reqId;
Object.assign(modal, {
job, detail: null, runs: null, runsError: null, dayFilter: null,
selectedRunId: null, runMessage: null, runMessageError: null,
});
renderModal({ loading: true }); // show what we have immediately
if (!dom.dialog.open) dom.dialog.showModal();
const current = () => reqId === modal.reqId && dom.dialog.open;
const detailP = api.getJob(jobId).then(
(detail) => { if (current()) { modal.detail = detail; renderModal(); } },
(err) => { if (current()) renderModal({ error: err.message }); },
);
const runsP = api.listRuns(jobId, CONFIG.historyDays).then(
(data) => {
if (!current()) return;
modal.runs = (data.runs ?? []).map(normalizeRun).filter((r) => r.startedAt);
renderHistory();
},
(err) => { if (current()) { modal.runsError = err.message; renderHistory(); } },
);
await Promise.all([detailP, runsP]);
}
async function selectRun(runId) {
const reqId = modal.reqId;
modal.selectedRunId = runId;
modal.runMessage = null;
modal.runMessageError = null;
renderHistory();
renderMessage();
try {
const run = await api.getRun(runId);
if (reqId !== modal.reqId || modal.selectedRunId !== runId) return;
modal.runMessage = {
text: run.message,
extras: Object.entries(run.details ?? {}).flatMap(([k, v]) =>
detailRow(k.replace(/_/g, ' '), typeof v === 'object' ? JSON.stringify(v) : String(v))),
};
} catch (err) {
if (reqId !== modal.reqId || modal.selectedRunId !== runId) return;
modal.runMessageError = err.message;
}
renderMessage();
dom.detailBody.querySelector('#detail-message')?.scrollIntoView({ block: 'nearest', behavior: 'smooth' });
}
// ---------- Events ----------
dom.tbody.addEventListener('click', (e) => {
const btn = e.target.closest('button[data-job-id]');
if (btn) openDetail(btn.dataset.jobId);
});
dom.dialog.addEventListener('click', (e) => {
// Click on the backdrop (the dialog element itself, outside its content) or the close button
if (e.target === dom.dialog || e.target.closest('[data-close]')) dom.dialog.close();
});
dom.detailBody.addEventListener('click', (e) => {
const day = e.target.closest('button[data-day]');
if (day) {
modal.dayFilter = modal.dayFilter === day.dataset.day ? null : day.dataset.day;
renderHistory();
return;
}
const run = e.target.closest('button[data-run-id]');
if (run) selectRun(run.dataset.runId);
});
document.querySelector('.summary').addEventListener('click', (e) => {
const card = e.target.closest('.summary-card');
if (!card) return;
state.filter = card.dataset.filter;
render();
});
document.querySelector('.jobs thead').addEventListener('click', (e) => {
const btn = e.target.closest('.sort');
if (!btn) return;
const key = btn.dataset.sort;
state.sort = state.sort.key === key ? { key, dir: -state.sort.dir } : { key, dir: 1 };
render();
});
dom.search.addEventListener('input', () => {
state.search = dom.search.value;
render();
});
dom.refreshBtn.addEventListener('click', () => refresh().then(schedulePoll));
// ---------- Boot ----------
refresh().then(schedulePoll);
+15
View File
@@ -0,0 +1,15 @@
// Central runtime config. Kept as a static file so ops can change it without a build.
const params = new URLSearchParams(window.location.search);
export const CONFIG = Object.freeze({
apiBaseUrl: '/api/v1',
refreshIntervalMs: 60_000,
requestTimeoutMs: 15_000,
// A job whose next scheduled run is this far in the past is flagged "Overdue".
// Catches the silent failure mode: a job stops reporting and stays green forever.
overdueGraceMinutes: 30,
// Run history shown in the job modal (the API caps this at 30)
historyDays: 30,
// ?mock=1 → use in-browser sample data, no backend needed
useMock: params.has('mock'),
});
+44
View File
@@ -0,0 +1,44 @@
const dateTimeFmt = new Intl.DateTimeFormat(undefined, { dateStyle: 'medium', timeStyle: 'short' });
const timeFmt = new Intl.DateTimeFormat(undefined, { timeStyle: 'medium' });
const relFmt = new Intl.RelativeTimeFormat(undefined, { numeric: 'auto' });
const REL_UNITS = [
['year', 31_536_000], ['month', 2_592_000], ['week', 604_800],
['day', 86_400], ['hour', 3_600], ['minute', 60],
];
export function parseDate(value) {
if (!value) return null;
const d = new Date(value);
return Number.isNaN(d.getTime()) ? null : d;
}
export const formatDateTime = (d) => (d ? dateTimeFmt.format(d) : '—');
export const formatTime = (d) => timeFmt.format(d);
export function formatRelative(d, now = Date.now()) {
if (!d) return '';
const diffSec = Math.round((d.getTime() - now) / 1000);
for (const [unit, secs] of REL_UNITS) {
if (Math.abs(diffSec) >= secs) return relFmt.format(Math.round(diffSec / secs), unit);
}
return relFmt.format(diffSec, 'second');
}
export function formatDuration(totalSec) {
if (totalSec == null || Number.isNaN(Number(totalSec))) return null;
const s = Math.max(0, Math.round(totalSec));
const h = Math.floor(s / 3600);
const m = Math.floor((s % 3600) / 60);
const sec = s % 60;
return [h && `${h}h`, (h || m) && `${m}m`, `${sec}s`].filter(Boolean).join(' ');
}
export function formatBytes(bytes) {
if (bytes == null || Number.isNaN(Number(bytes))) return null;
const units = ['B', 'KB', 'MB', 'GB', 'TB', 'PB'];
let n = Number(bytes);
let i = 0;
while (n >= 1024 && i < units.length - 1) { n /= 1024; i++; }
return `${n.toFixed(i === 0 ? 0 : 1)} ${units[i]}`;
}
+129
View File
@@ -0,0 +1,129 @@
// Sample data matching the API contract. Timestamps are relative to page load so it always looks "live".
const now = Date.now();
const MIN = 60_000;
const HR = 60 * MIN;
const DAY = 24 * HR;
const iso = (offsetMs) => new Date(now + offsetMs).toISOString();
// every_h drives the generated run history; it isn't part of the API.
const JOBS = [
{
id: 1, company_name: 'Acme Corp', job_name: 'File Server – Nightly', status: 'success', every_h: 24,
last_run_at: iso(-6 * HR), next_run_at: iso(18 * HR),
message: 'Job completed successfully. 1,284 files processed.',
source_system: 'Veeam B&R', duration_seconds: 2840, size_bytes: 128_849_018_880,
},
{
id: 2, company_name: 'Acme Corp', job_name: 'SQL01 – Transaction Logs', status: 'warning', every_h: 1,
last_run_at: iso(-25 * MIN), next_run_at: iso(35 * MIN),
message: 'Processing SQL01 Warning:\nFailed to truncate transaction logs for database "ERP_PROD".\nLog backup completed but truncation was skipped.',
source_system: 'Veeam B&R', duration_seconds: 312, size_bytes: 2_147_483_648,
details: { retry_count: 0, repository: 'REPO-ONSITE-01' },
},
{
// Running now; "last run" is still the previous completion
id: 7, company_name: 'Acme Corp', job_name: 'Hyper-V Hosts – Daily', status: 'running', every_h: 24,
running_since: iso(-12 * MIN), last_status: 'success',
last_run_at: iso(-DAY + 40 * MIN), next_run_at: iso(DAY - 12 * MIN),
message: 'Completed successfully. 6 VMs processed.',
source_system: 'Veeam B&R', duration_seconds: 3120, size_bytes: 412_316_860_416,
},
{
id: 3, company_name: 'Bluewater Dental', job_name: 'Workstations – Daily', status: 'failed', every_h: 24,
last_run_at: iso(-3 * HR), next_run_at: iso(21 * HR),
message: 'Error: Unable to connect to backup repository \\\\NAS02\\Backups.\nThe network path was not found. (0x80070035)\nJob terminated after 3 retries.',
source_system: 'Datto', duration_seconds: 95,
details: { retry_count: 3, target: '\\\\NAS02\\Backups' },
},
{
id: 4, company_name: 'Bluewater Dental', job_name: 'Imaging Server – Hourly', status: 'success', every_h: 1,
last_run_at: iso(-50 * MIN), next_run_at: iso(10 * MIN),
message: 'Completed.', source_system: 'Datto', duration_seconds: 420, size_bytes: 8_589_934_592,
},
{
// Green status but next run is in the past → should show as Overdue
id: 5, company_name: 'Northfield Logistics', job_name: 'Exchange – Nightly', status: 'success', every_h: 24,
last_run_at: iso(-30 * HR), next_run_at: iso(-6 * HR),
message: 'Completed successfully.', source_system: 'Acronis', duration_seconds: 5400,
},
{
// Never run / unrecognised status → rendered as Unknown
id: 6, company_name: 'Northfield Logistics', job_name: 'New NAS – Weekly', status: null,
last_run_at: null, next_run_at: iso(2 * DAY),
message: '', source_system: 'Synology Hyper Backup',
},
];
const MESSAGES = {
success: 'Job completed successfully.',
warning: 'Completed with warnings: 1 file skipped (file in use).',
failed: 'Error: Target repository unreachable. Job terminated.',
};
// Deterministic pseudo-random so history doesn't reshuffle on every modal open
function rng(seed) {
let x = seed;
return () => ((x = (x * 1_103_515_245 + 12_345) % 2_147_483_648) / 2_147_483_648);
}
const RUNS = new Map(); // job id → runs, newest first
for (const job of JOBS) {
const runs = [];
if (job.running_since) {
runs.push({ id: `${job.id}-r`, status: 'running', started_at: job.running_since, finished_at: null, message: 'In progress…' });
}
if (job.last_run_at && job.every_h) {
const rand = rng(job.id * 7919);
const dur = (job.duration_seconds ?? 600) * 1000;
let end = Date.parse(job.last_run_at);
for (let i = 0; end - dur > now - 30 * DAY; i++, end -= job.every_h * HR) {
const r = rand();
const status = i === 0 ? (job.last_status ?? job.status) : r < 0.03 ? 'failed' : r < 0.1 ? 'warning' : 'success';
const thisDur = dur * (0.8 + rand() * 0.4);
runs.push({
id: `${job.id}-${i}`,
status,
started_at: new Date(end - thisDur).toISOString(),
finished_at: new Date(end).toISOString(),
duration_seconds: Math.round(thisDur / 1000),
size_bytes: status === 'failed' ? null : job.size_bytes,
message: i === 0 ? job.message : MESSAGES[status],
details: i === 0 ? job.details ?? {} : {},
});
}
}
RUNS.set(String(job.id), runs);
}
const delay = (ms) => new Promise((r) => setTimeout(r, ms));
const summary = ({ details, every_h, last_status, duration_seconds, size_bytes, ...rest }) => rest;
export const mockApi = {
async listJobs() {
await delay(250);
return { generated_at: new Date().toISOString(), jobs: JOBS.map(summary) };
},
async getJob(id) {
await delay(200);
const job = JOBS.find((j) => String(j.id) === String(id));
if (!job) throw new Error('Job not found');
const { every_h, ...rest } = job;
return { last_status: job.status === 'running' ? null : job.status, ...rest, details: job.details ?? {} };
},
async listRuns(id, days) {
await delay(300);
const since = now - days * DAY;
const runs = (RUNS.get(String(id)) ?? [])
.filter((r) => Date.parse(r.started_at) >= since)
.map(({ message, details, ...r }) => r);
return { job_id: id, since: new Date(since).toISOString(), runs };
},
async getRun(id) {
await delay(150);
for (const runs of RUNS.values()) {
const run = runs.find((r) => r.id === id);
if (run) return { details: {}, ...run };
}
throw new Error('Run not found');
},
};