Author SHA1 Message Date
kyleandClaude Opus 5.5 2fa9eea274 Replace basic auth with Microsoft 365 sign-in
nginx now checks every dashboard and API request with oauth2-proxy
(auth_request), which signs users in with Microsoft Entra ID. Who may
sign in is controlled in Entra: a single-tenant app registration with
"Assignment required", so only assigned users/groups get through.

- deploy/oauth2-proxy.cfg (entra-id provider, auth-only mode, secure
  cookie, trusted proxy = localhost) and deploy/oauth2-proxy.service
- nginx: pages redirect to Microsoft sign-in and back to the same URL;
  API calls get a plain 401 (the page already asks to reload);
  /api/v1/ingest stays key-only for collectors; /oauth2/auth is
  internal; the signed-in user is passed to the API in headers nginx
  sets (so browsers can't spoof them); dashboard files are served with
  Cache-Control: no-cache so updates show up at once
- GET /api/v1/me; the top bar shows the signed-in user and a Sign out
  that also signs out of Microsoft (otherwise SSO signs straight back in)
- README: Entra and server setup steps

Tested locally with oauth2-proxy 7.15.5 and nginx 1.31 running the
deploy configs (TLS, self-signed): redirects to the tenant's sign-in
with the return URL kept, 401s, ingest bypass, sign-out allow-list,
cookie flags, security headers. The signed-in path was tested with a
stand-in for oauth2-proxy's auth answer; a real Entra sign-in wasn't.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-08 11:31:35 -06:00
kyleandClaude Opus 5.5 8d0e6dca71 Move code shared by the collectors into one module
The Veeam, MARS, WSB and VB365 collectors each carried identical copies
of logging, ISO time conversion, property lookup, posting to the API,
config loading, TLS setup, the sent-runs state file and the 401 check,
and their installers repeated the same key prompt, locked-down config,
file copy and scheduled-task registration. These now live in
collectors/common/BackupDashboard.Collector.psm1:

- Initialize-Collector / Test-RunSent / Send-JobReport /
  Test-AuthRejected / Complete-Collector for collector runs
- Write-Log, Get-ErrorText, ConvertTo-IsoTime, Get-FirstValue
- Install-CollectorTask for the shared half of each installer

Collectors find the module next to themselves (installed) or in
..\common (repo); installers copy it into Program Files beside the
collector. The diagnostic scripts stay standalone single files.

No behavior change: a harness ran each collector against its mock
(12 scenarios: dry run, two real runs against a stub API, state file,
logs, exit codes, rejected key, en-GB, backfill) before and after, and
all 84 outputs are identical. Installers, run with stand-ins for the
Windows-only commands, write identical configs and tasks; the only
differences are the module being copied and the summary line naming
the shell's path.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-08 11:09:01 -06:00
kyleandClaude Opus 5.5 118a7f978d Fix VB365 collector against a real server's object model
Checked with Get-Vb365CollectorDiagnostics.ps1 output from a VB365 8.4
server (GDC-VEEAM-04, 11 organizations, Windows PowerShell 5.1). Times
(session CreationTime/EndTime, job NextRun) are local, and property
names matched. Two fixes:

- Session log items carry their status as a Title prefix ("[Warning]
  ...") with no separate status property. Match on the prefix and strip
  it, instead of matching words in the text, which would have pulled in
  lines like "[Success] Processing site: .../error-reports"
- Retries are separate sessions (MainSessionId -> the retried session,
  which has JobWillBeRetried = True). Mark them in details (retry_of,
  will_retry) and say "Veeam will retry this job." on a pending retry

Mock rebuilt on the real shapes, including a warning + retry pair.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-08 10:47:40 -06:00
kyleandClaude Opus 5.5 3f407ba76e Add Veeam Backup for Microsoft 365 collector
PowerShell script for the VB365 server, run hourly as a SYSTEM scheduled
task (Windows PowerShell 5.1 by default, -PwshPath for PowerShell 7).
One dashboard job per VB365 backup job and backup copy job.

- Sessions from Get-VBOJobSession (and Get-VBOCopyJobSession where
  available) within the lookback window; status from session Status,
  message from the session log's warning/error lines, size from
  Statistics.TransferredData ("1.5 GB" -> bytes)
- One VB365 server can protect several Microsoft 365 organizations, so
  the dashboard company is chosen per job: OrganizationTenants mapping,
  else Tenant, else the organization's name
- Next run from the job's NextRun; empty for disabled jobs
- Get-Vb365CollectorDiagnostics.ps1 dumps jobs and sessions, showing
  each DateTime's Kind (local vs UTC); installer adapted from MARS

Not yet run against a real server.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-08 10:31:43 -06:00
kyleandClaude Opus 5.5 dd3e2fe885 Read Veeam next run from the live schedule
On VBR 13 a job's ScheduleOptions property is a stale snapshot: NextRun
is empty (and its LatestRunLocal years old), so every Veeam job showed
no next run and could never be flagged Overdue. GetScheduleOptions() /
Get-VBRJobScheduleOptions return the live schedule ("2026-10-09
12:15:00 AM"). Use GetScheduleOptions(), falling back to the property.

Immediate-mode backup copy jobs (IsContinuous) have a live NextRun too,
but it's a stale date months in the past, which would show as Overdue;
keep reporting no next run for them.

Parsing tries the current culture, then invariant; VBR 13's format
parsed correctly under en-CA, en-US, fr-CA and en-GB. Mock updated to
VBR 13's schedule shape.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-08 09:53:55 -06:00
kyleandClaude Opus 5.5 51b97c4556 Diagnose Veeam schedules; show "Not scheduled" for no next run
Veeam jobs on a VBR 13 server show no next run on the dashboard. The
diagnostics script gains a section dumping each job's schedule
properties (ScheduleOptions, GetScheduleOptions(),
Get-VBRJobScheduleOptions) and the current culture, to find where VBR 13
keeps the next run.

The dashboard showed "Never" for a missing next run, which reads as
"this job will never run"; it now says "Not scheduled" (e.g. immediate
mode copy jobs).

Co-Authored-By: Claude Opus 5.5 <[email protected]>
2026-10-08 09:42:01 -06:00
31 changed files with 1327 additions and 515 deletions
+56 -9
View File
@@ -5,14 +5,15 @@ Proof of concept backup dashboard, intended to display backup jobs from multiple
``` ```
collectors ──POST /api/v1/ingest (API key)──▶ ┌──────────────┐ collectors ──POST /api/v1/ingest (API key)──▶ ┌──────────────┐
│ FastAPI + │ │ FastAPI + │
browser ──nginx (basic auth, TLS)──▶ /api/* ─▶│ SQLite │ browser ──nginx (TLS)──▶ /api/* ─────────────▶│ SQLite │
└─ static files: public/ └──────────────┘ │ └─ static files: public/ └──────────────┘
└─ auth_request ─▶ oauth2-proxy ─▶ Microsoft Entra ID (Microsoft 365 sign-in)
``` ```
- `public/`: static frontend (no build step). Add `?mock=1` to the URL to use built-in sample data. - `public/`: static frontend (no build step). Add `?mock=1` to the URL to use built-in sample data.
- `backend/`: FastAPI API backed by a single SQLite file. - `backend/`: FastAPI API backed by a single SQLite file.
- `deploy/`: nginx site and systemd unit. - `deploy/`: nginx site and systemd unit.
- `collectors/`: scripts that report results from backup products ([Veeam B&R](collectors/veeam/README.md), [Azure Backup / MARS](collectors/mars/README.md), [Windows Server Backup](collectors/wsb/README.md)). - `collectors/`: PowerShell scripts that report results from backup products ([Veeam B&R](collectors/veeam/README.md), [Azure Backup / MARS](collectors/mars/README.md), [Windows Server Backup](collectors/wsb/README.md), [Veeam Backup for M365](collectors/vb365/README.md)). Code they share (config, logging, posting, state, installing) is in `collectors/common/BackupDashboard.Collector.psm1`.
## Data model ## Data model
@@ -72,12 +73,58 @@ The OpenAPI schema is at `/api/v1/openapi.json`.
## Authentication ## Authentication
- **Dashboard users:** nginx basic auth over HTTPS (`deploy/nginx.conf`). Every user can see every tenant. - **Dashboard users:** Microsoft 365 sign-in. nginx checks every request with
*Planned:* Microsoft 365 sign-in through [oauth2-proxy](https://oauth2-proxy.github.io/oauth2-proxy/) [oauth2-proxy](https://oauth2-proxy.github.io/oauth2-proxy/), which signs users in with Microsoft
with the Entra ID provider and nginx `auth_request`. That's an nginx-only change. The frontend Entra ID. Who may sign in is controlled in Entra (users or a group assigned to the app). Every
already tells the user to reload when a session expires (HTTP 401). signed-in user can see every tenant. Sessions last 12 hours; signing in again is usually silent.
The dashboard shows who's signed in; **Sign out** also signs out of Microsoft in that browser,
otherwise the next visit would sign straight back in.
- **Collectors:** per-collector bearer keys, checked by the app. Only a SHA-256 hash is stored. - **Collectors:** per-collector bearer keys, checked by the app. Only a SHA-256 hash is stored.
To revoke one, run `python manage.py revoke-key NAME`. To revoke one, run `python manage.py revoke-key NAME`. `/api/v1/ingest` is the only path that
doesn't need Microsoft sign-in.
### Microsoft 365 sign-in setup
**In the Microsoft Entra admin center** (entra.microsoft.com), as an admin of your tenant:
1. *App registrations → New registration*. Name: `Backup Dashboard`. Supported account types:
**Accounts in this organizational directory only** (single tenant). Redirect URI: platform
**Web**, `https://<your dashboard host>/oauth2/callback`. Register.
2. On its *Overview*, note the **Application (client) ID** and **Directory (tenant) ID**.
3. *Certificates & secrets → New client secret*. Copy the secret's **Value** now (it's only shown
once). Note its expiry date: sign-in stops working when it expires, so set a reminder to
create a new one before then.
4. *Enterprise applications → Backup Dashboard → Properties*: set **Assignment required** to
**Yes**. Then *Users and groups → Add user/group* and add the people (or a group) who should
see the dashboard. Everyone else in the tenant is refused by Microsoft before reaching it.
No API permissions beyond the default `User.Read` are needed.
**On the dashboard server:**
1. Install oauth2-proxy 7.8 or newer from its
[releases](https://github.com/oauth2-proxy/oauth2-proxy/releases) as `/usr/local/bin/oauth2-proxy`,
and create its user: `sudo useradd --system --no-create-home oauth2-proxy`.
2. Install the config and the client secret, readable only by root and oauth2-proxy:
```bash
sudo install -d -m 750 -g oauth2-proxy /etc/oauth2-proxy
sudo install -m 640 -g oauth2-proxy deploy/oauth2-proxy.cfg /etc/oauth2-proxy/backup-dashboard.cfg
sudo sh -c 'umask 027; printf %s "<client secret value>" > /etc/oauth2-proxy/client-secret' && sudo chgrp oauth2-proxy /etc/oauth2-proxy/client-secret
```
In `/etc/oauth2-proxy/backup-dashboard.cfg`, fill in `<TENANT_ID>`, `<APPLICATION_CLIENT_ID>`,
`<DASHBOARD_HOST>`, and `<COOKIE_SECRET>`, generated with:
`openssl rand -base64 32 | tr -- '+/' '-_'`
3. Install `deploy/oauth2-proxy.service` into `/etc/systemd/system/`, then
`sudo systemctl daemon-reload && sudo systemctl enable --now oauth2-proxy`.
Check it with `journalctl -u oauth2-proxy -n 20`: it should say "OAuthProxy configured for
Microsoft Entra ID".
4. Install the new `deploy/nginx.conf` (keeping your server name and certificate paths), then
`sudo nginx -t && sudo systemctl reload nginx`. Basic auth and its htpasswd file are no longer
used.
**Troubleshooting:** sign-in problems show in `journalctl -u oauth2-proxy`. Microsoft's own
errors (wrong redirect URI, user not assigned, expired secret) appear on its sign-in page with an
`AADSTS…` code.
## Development ## Development
@@ -96,7 +143,7 @@ BACKUP_DASH_SERVE_STATIC=1 uvicorn --factory app.main:create_app --reload
1. Copy the repo to `/opt/backup-dashboard`, create a venv at `/opt/backup-dashboard/venv`, and run `pip install -r backend/requirements.txt`. 1. Copy the repo to `/opt/backup-dashboard`, create a venv at `/opt/backup-dashboard/venv`, and run `pip install -r backend/requirements.txt`.
2. Copy `deploy/backup-dashboard.env` to `/etc/backup-dashboard/backup-dashboard.env`. Both the service and `manage.py` read their settings from it. 2. Copy `deploy/backup-dashboard.env` to `/etc/backup-dashboard/backup-dashboard.env`. Both the service and `manage.py` read their settings from it.
3. Create a `backupdash` system user, then install `deploy/backup-dashboard-api.service` into `/etc/systemd/system/` and enable it. The database is created on first start. 3. Create a `backupdash` system user, then install `deploy/backup-dashboard-api.service` into `/etc/systemd/system/` and enable it. The database is created on first start.
4. Copy `public/` to `/var/www/backup-dashboard/public`, install `deploy/nginx.conf`, get a certificate with certbot, and create users with `htpasswd -B`. 4. Copy `public/` to `/var/www/backup-dashboard/public`, install `deploy/nginx.conf`, get a certificate with certbot, and set up Microsoft 365 sign-in (see [Microsoft 365 sign-in setup](#microsoft-365-sign-in-setup)).
5. Run `manage.py` as the service user. It finds the database through the env file: 5. Run `manage.py` as the service user. It finds the database through the env file:
```bash ```bash
cd /opt/backup-dashboard/backend cd /opt/backup-dashboard/backend
+2 -2
View File
@@ -1,7 +1,7 @@
"""Collector (machine) authentication. """Collector (machine) authentication.
Dashboard *users* are authenticated by nginx in front of the app (basic auth today, Dashboard *users* are authenticated by nginx in front of the app (Microsoft 365 sign-in via
oauth2-proxy + Microsoft Entra ID later), so nothing here deals with people. oauth2-proxy), so nothing here deals with people.
""" """
import hashlib import hashlib
+10
View File
@@ -25,6 +25,16 @@ def health(conn: sqlite3.Connection = Depends(db.get_db)):
return {"status": "ok"} return {"status": "ok"}
@router.get("/me")
def me(request: Request):
# Set by nginx from oauth2-proxy's sign-in check (nginx overwrites anything the browser sends).
# Display only: access control happens in nginx. Empty in dev or without Microsoft sign-in.
return {
"email": request.headers.get("x-forwarded-email") or None,
"name": request.headers.get("x-forwarded-preferred-username") or request.headers.get("x-forwarded-user") or None,
}
@router.get("/jobs") @router.get("/jobs")
def list_jobs(conn: sqlite3.Connection = Depends(db.get_db)): def list_jobs(conn: sqlite3.Connection = Depends(db.get_db)):
return {"generated_at": to_db(utcnow()), "jobs": repo.list_jobs(conn)} return {"generated_at": to_db(utcnow()), "jobs": repo.list_jobs(conn)}
+6
View File
@@ -122,3 +122,9 @@ def test_tenant_name_is_case_insensitive(client):
body["tenant"] = "ACME CORP" body["tenant"] = "ACME CORP"
client.post("/api/v1/ingest", json=body) client.post("/api/v1/ingest", json=body)
assert len(client.get("/api/v1/jobs").json()["jobs"]) == 1 assert len(client.get("/api/v1/jobs").json()["jobs"]) == 1
def test_me_reports_signed_in_user_from_proxy_headers(client):
assert client.get("/api/v1/me").json() == {"email": None, "name": None}
r = client.get("/api/v1/me", headers={"X-Forwarded-Email": "[email protected]", "X-Forwarded-User": "abc-123"})
assert r.json() == {"email": "[email protected]", "name": "abc-123"}
@@ -0,0 +1,270 @@
<#
Shared code for the Backup Dashboard collectors: config and state files, logging, posting to
the ingest API, and installing a collector as a scheduled task.
Each collector imports this from next to itself (installed: C:\Program Files\BackupDashboard)
or from ..\common (in the repo). Collectors on the same server share one installed copy, so
keep these functions backward compatible.
Works on Windows PowerShell 5.1 and PowerShell 7.
#>
$script:Config = $null
$script:DryRun = $false
$script:LogPath = $null
$script:StatePath = $null
$script:LookbackHours = 48
$script:Sent = @{}
$script:RunsSent = 0
$script:MaxLogBytes = 5MB
$script:Invariant = [Globalization.CultureInfo]::InvariantCulture
# ---------- Logging and values ----------
function Write-Log {
param([string]$Message, [string]$Level = 'INFO')
$line = '{0} [{1}] {2}' -f (Get-Date).ToString('yyyy-MM-dd HH:mm:ss'), $Level, $Message
Write-Host $line
if ($script:DryRun -or -not $script:LogPath) { return }
if ((Test-Path $script:LogPath) -and (Get-Item $script:LogPath).Length -gt $script:MaxLogBytes) {
Move-Item $script:LogPath "$($script:LogPath).1" -Force
}
Add-Content -Path $script:LogPath -Value $line -Encoding UTF8
}
function Get-ErrorText($ErrorRecord) {
# Windows PowerShell puts the HTTP response body (e.g. the API's 422 explanation) here
if ($ErrorRecord.ErrorDetails -and $ErrorRecord.ErrorDetails.Message) {
return "$($ErrorRecord.Exception.Message) $($ErrorRecord.ErrorDetails.Message)"
}
return $ErrorRecord.Exception.Message
}
# The API needs an explicit UTC offset. DateTimeOffset keeps a Local/Unspecified time as local and a
# Utc time as +00:00, so convert UTC-valued times to Kind Utc before calling this.
function ConvertTo-IsoTime($Value) {
if ($null -eq $Value) { return $null }
$dt = [datetime]$Value
if ($dt.Year -le 1900) { return $null } # "not set" (DateTime.MinValue, MARS's 1601-01-01, ...)
return ([DateTimeOffset]$dt).ToString("yyyy-MM-dd'T'HH:mm:sszzz", $script:Invariant)
}
# First property of $Object, out of $Names, that exists and isn't empty. Backup products' object
# models aren't fully documented, so reads go through this rather than assuming one spelling.
function Get-FirstValue($Object, [string[]]$Names) {
if ($null -eq $Object) { return $null }
foreach ($name in $Names) {
try {
$prop = $Object.PSObject.Properties[$name]
if (-not $prop) { continue }
$value = $prop.Value
# Check emptiness by type: casting to [string] would make a one-item list of objects ''
if ($null -eq $value) { continue }
if ($value -is [string] -and $value -eq '') { continue }
if ($value -is [Collections.ICollection] -and $value.Count -eq 0) { continue }
return $value
} catch { }
}
return $null
}
# ---------- Collector run ----------
<#
Loads the config ($ConfigPath, JSON) and the record of runs already sent, and sets up logging
and TLS. Log and state files sit next to the config: <Name>-collector.log / .state.json.
Returns the config object; its LookbackHours is resolved (the parameter, else the config, else
48). Throws if the config can't be read or lacks one of $RequiredKeys.
#>
function Initialize-Collector {
param(
[Parameter(Mandatory)][string]$Name,
[Parameter(Mandatory)][string]$ConfigPath,
[int]$LookbackHours = 0,
[switch]$DryRun,
[string[]]$RequiredKeys = @('ApiUrl', 'ApiKey', 'Tenant')
)
# Logging first, so a bad config can still be logged
$dataDir = Split-Path -Parent $ConfigPath
$script:LogPath = Join-Path $dataDir "$Name-collector.log"
$script:StatePath = Join-Path $dataDir "$Name-collector.state.json"
$script:DryRun = [bool]$DryRun
$script:RunsSent = 0
$script:Sent = @{}
$config = Get-Content -Path $ConfigPath -Raw | ConvertFrom-Json
foreach ($key in $RequiredKeys) {
if (-not $config.$key) { throw "Config $ConfigPath is missing '$key'" }
}
if ($LookbackHours -le 0) {
$LookbackHours = if ($config.LookbackHours) { [int]$config.LookbackHours } else { 48 }
}
$config | Add-Member -NotePropertyName LookbackHours -NotePropertyValue $LookbackHours -Force
$script:LookbackHours = $LookbackHours
$script:Config = $config
# Windows PowerShell 5.1 may default to TLS 1.0
[Net.ServicePointManager]::SecurityProtocol = [Net.ServicePointManager]::SecurityProtocol -bor [Net.SecurityProtocolType]::Tls12
# Run key -> start time (Unix seconds), for runs already delivered in their final state.
# Numbers rather than ISO strings: PowerShell 7's ConvertFrom-Json turns date strings into
# DateTimes and loses the offset.
if (Test-Path $script:StatePath) {
$saved = Get-Content -Path $script:StatePath -Raw | ConvertFrom-Json
if ($saved) { foreach ($p in $saved.PSObject.Properties) { $script:Sent[$p.Name] = [long]$p.Value } }
}
return $config
}
function Test-RunSent([string]$Key) {
return $script:Sent.ContainsKey($Key)
}
<#
Posts one report to the ingest API: the job alone (to refresh next_run_at) or the job with a
run. A run that has finished is then recorded as sent under $SentKey, so later collector runs
skip it; a running one isn't, so it's re-sent until it finishes. Throws on an HTTP error.
#>
function Send-JobReport {
param(
[Parameter(Mandatory)][string]$Tenant,
[Parameter(Mandatory)][string]$SourceSystem,
[Parameter(Mandatory)]$Job,
$Run,
[string]$SentKey,
$StartedAt
)
$body = [ordered]@{ tenant = $Tenant; source_system = $SourceSystem; job = $Job }
if ($Run) { $body.run = $Run }
$json = $body | ConvertTo-Json -Depth 6 -Compress
if ($script:DryRun) {
Write-Host $json
} else {
# Send bytes: Windows PowerShell would otherwise encode a string body as Latin-1,
# mangling non-ASCII job names (e.g. en dashes)
$params = @{
Uri = "$($script:Config.ApiUrl.TrimEnd('/'))/api/v1/ingest"
Method = 'Post'
Headers = @{ Authorization = "Bearer $($script:Config.ApiKey)" }
ContentType = 'application/json; charset=utf-8'
Body = [Text.Encoding]::UTF8.GetBytes($json)
TimeoutSec = 30
}
Invoke-RestMethod @params | Out-Null
}
if ($Run) {
$script:RunsSent++
if ($Run.status -ne 'running' -and $SentKey) {
$script:Sent[$SentKey] = ([DateTimeOffset][datetime]$StartedAt).ToUnixTimeSeconds()
}
}
}
# True when a failed Send-JobReport was the API rejecting the key: no point trying more jobs
function Test-AuthRejected($ErrorRecord) {
$status = 0
try { $status = [int]$ErrorRecord.Exception.Response.StatusCode } catch { }
return $status -eq 401
}
<#
Saves the sent-runs record (dropping runs older than the lookback window plus a day) and logs
the summary line. Doesn't exit: the collector does that with the returned exit code.
#>
function Complete-Collector {
param([int]$JobCount = -1, [int]$Failures = 0)
if (-not $script:DryRun) {
$cutoff = [DateTimeOffset]::Now.AddHours(-$script:LookbackHours - 24).ToUnixTimeSeconds()
$keep = [ordered]@{}
foreach ($key in $script:Sent.Keys) {
if ($script:Sent[$key] -ge $cutoff) { $keep[$key] = $script:Sent[$key] }
}
try {
$keep | ConvertTo-Json -Compress | Set-Content -Path $script:StatePath -Encoding UTF8
} catch {
Write-Log "Couldn't save state file: $(Get-ErrorText $_)" 'WARN'
}
}
if ($JobCount -ge 0) {
Write-Log ("Reported {0} jobs, {1} runs, {2} failures" -f $JobCount, $script:RunsSent, $Failures)
} else {
Write-Log ("Reported {0} runs, {1} failures" -f $script:RunsSent, $Failures)
}
if ($Failures) { return 1 }
return 0
}
# ---------- Installing ----------
<#
The part of every collector installer after its own checks: prompt for the API key, write the
config to ProgramData\BackupDashboard (locked to SYSTEM and Administrators, since it holds the
key), copy the collector script and this module to $InstallDir (Program Files: only admins can
modify what runs as SYSTEM), and register a scheduled task running it as SYSTEM.
#>
function Install-CollectorTask {
param(
[Parameter(Mandatory)][string]$Name, # 'veeam' -> veeam-collector.json / .log
[Parameter(Mandatory)][string]$ScriptPath, # the collector script to install
[Parameter(Mandatory)][string]$TaskName,
[Parameter(Mandatory)][string]$TaskDescription,
[Parameter(Mandatory)][string]$Shell, # powershell.exe or pwsh.exe, full path
[Parameter(Mandatory)][string]$ApiUrl,
[Parameter(Mandatory)][int]$IntervalMinutes,
[Parameter(Mandatory)][string]$InstallDir,
[System.Collections.IDictionary]$Settings = @{} # extra config: Tenant, LookbackHours, ...
)
$dataDir = Join-Path $env:ProgramData 'BackupDashboard'
$configPath = Join-Path $dataDir "$Name-collector.json"
if ($ApiUrl -notmatch '^https://') {
Write-Warning 'ApiUrl is not HTTPS: the API key would be sent in clear text.'
}
# Read the key as a SecureString so it isn't echoed to the console or kept in shell history
$secure = Read-Host -AsSecureString 'API key (from: python manage.py create-key <name>)'
$bstr = [Runtime.InteropServices.Marshal]::SecureStringToBSTR($secure)
try { $apiKey = [Runtime.InteropServices.Marshal]::PtrToStringBSTR($bstr) }
finally { [Runtime.InteropServices.Marshal]::ZeroFreeBSTR($bstr) }
if (-not $apiKey.StartsWith('bdk_')) { throw "That doesn't look like a dashboard API key (expected 'bdk_...')." }
# Config folder: SYSTEM + Administrators only (SIDs so this works on non-English Windows)
New-Item -ItemType Directory -Path $dataDir -Force | Out-Null
& icacls.exe $dataDir /inheritance:r /grant:r '*S-1-5-18:(OI)(CI)F' '*S-1-5-32-544:(OI)(CI)F' | Out-Null
if ($LASTEXITCODE) { throw "icacls failed on $dataDir" }
$config = [ordered]@{ ApiUrl = $ApiUrl.TrimEnd('/'); ApiKey = $apiKey }
foreach ($key in $Settings.Keys) { $config[$key] = $Settings[$key] }
$config | ConvertTo-Json -Depth 4 | Set-Content -Path $configPath -Encoding UTF8
# The collector and this module, side by side (the collector looks for the module next to itself)
New-Item -ItemType Directory -Path $InstallDir -Force | Out-Null
$installedScript = Join-Path $InstallDir (Split-Path -Leaf $ScriptPath)
foreach ($file in $ScriptPath, $PSCommandPath) {
$target = Join-Path $InstallDir (Split-Path -Leaf $file)
Copy-Item -Path $file -Destination $target -Force
Unblock-File -Path $target
}
$action = New-ScheduledTaskAction -Execute $Shell `
-Argument "-NoProfile -NonInteractive -ExecutionPolicy Bypass -File `"$installedScript`""
$trigger = New-ScheduledTaskTrigger -Once -At (Get-Date).AddMinutes(1) `
-RepetitionInterval (New-TimeSpan -Minutes $IntervalMinutes)
$principal = New-ScheduledTaskPrincipal -UserId 'S-1-5-18' -LogonType ServiceAccount -RunLevel Highest
$taskSettings = New-ScheduledTaskSettingsSet -ExecutionTimeLimit (New-TimeSpan -Minutes 30) `
-MultipleInstances IgnoreNew -StartWhenAvailable
Register-ScheduledTask -TaskName $TaskName -Action $action -Trigger $trigger -Principal $principal `
-Settings $taskSettings -Description $TaskDescription -Force | Out-Null
Write-Host "Installed. Config: $configPath"
Write-Host "Task '$TaskName' runs every $IntervalMinutes minutes as SYSTEM, using $Shell."
Write-Host ''
Write-Host 'Next steps:'
Write-Host " 1. Preview what will be sent: & '$Shell' -File '$installedScript' -DryRun"
Write-Host " 2. Backfill 30 days of history: & '$Shell' -File '$installedScript' -LookbackHours 720"
Write-Host " 3. Check the log: Get-Content '$dataDir\$Name-collector.log' -Tail 20"
}
Export-ModuleMember -Function Write-Log, Get-ErrorText, ConvertTo-IsoTime, Get-FirstValue,
Initialize-Collector, Test-RunSent, Send-JobReport, Test-AuthRejected, Complete-Collector,
Install-CollectorTask
+14 -49
View File
@@ -29,9 +29,6 @@ param(
) )
$ErrorActionPreference = 'Stop' $ErrorActionPreference = 'Stop'
$TaskName = 'Backup Dashboard - MARS collector'
$DataDir = Join-Path $env:ProgramData 'BackupDashboard'
$ConfigPath = Join-Path $DataDir 'mars-collector.json'
$WindowsPowerShell = Join-Path $env:SystemRoot 'System32\WindowsPowerShell\v1.0\powershell.exe' $WindowsPowerShell = Join-Path $env:SystemRoot 'System32\WindowsPowerShell\v1.0\powershell.exe'
# Check the agent is here before changing anything # Check the agent is here before changing anything
@@ -40,51 +37,19 @@ if (-not (Get-Module -ListAvailable -Name MSOnlineBackup) -and -not (Test-Path $
throw 'The MARS agent (MSOnlineBackup PowerShell module) is not installed on this server.' throw 'The MARS agent (MSOnlineBackup PowerShell module) is not installed on this server.'
} }
if ($ApiUrl -notmatch '^https://') { # Shared installer code: in the repo it's in ..\common, so copy the whole collectors folder
Write-Warning 'ApiUrl is not HTTPS: the API key would be sent in clear text.' $sharedModule = @(
(Join-Path $PSScriptRoot 'BackupDashboard.Collector.psm1'),
(Join-Path (Join-Path (Join-Path $PSScriptRoot '..') 'common') 'BackupDashboard.Collector.psm1')
) | Where-Object { Test-Path $_ } | Select-Object -First 1
if (-not $sharedModule) {
throw "BackupDashboard.Collector.psm1 not found. Copy the whole collectors folder, including common."
} }
Import-Module $sharedModule -Force
# Read the key as a SecureString so it isn't echoed to the console or kept in shell history $settings = [ordered]@{ Tenant = $Tenant; SourceSystem = $SourceSystem; LookbackHours = $LookbackHours }
$secure = Read-Host -AsSecureString 'API key (from: python manage.py create-key <name>)' if ($JobName) { $settings.JobName = $JobName }
$bstr = [Runtime.InteropServices.Marshal]::SecureStringToBSTR($secure) Install-CollectorTask -Name 'mars' -ScriptPath (Join-Path $PSScriptRoot 'Send-MarsBackupStatus.ps1') `
try { $apiKey = [Runtime.InteropServices.Marshal]::PtrToStringBSTR($bstr) } -TaskName 'Backup Dashboard - MARS collector' `
finally { [Runtime.InteropServices.Marshal]::ZeroFreeBSTR($bstr) } -TaskDescription 'Sends Azure Backup (MARS) results to the Backup Dashboard' `
if (-not $apiKey.StartsWith('bdk_')) { throw "That doesn't look like a dashboard API key (expected 'bdk_...')." } -Shell $WindowsPowerShell -ApiUrl $ApiUrl -IntervalMinutes $IntervalMinutes -InstallDir $InstallDir -Settings $settings
# Config folder: SYSTEM + Administrators only (SIDs so this works on non-English Windows)
New-Item -ItemType Directory -Path $DataDir -Force | Out-Null
& icacls.exe $DataDir /inheritance:r /grant:r '*S-1-5-18:(OI)(CI)F' '*S-1-5-32-544:(OI)(CI)F' | Out-Null
if ($LASTEXITCODE) { throw "icacls failed on $DataDir" }
$config = [ordered]@{
ApiUrl = $ApiUrl.TrimEnd('/')
ApiKey = $apiKey
Tenant = $Tenant
SourceSystem = $SourceSystem
LookbackHours = $LookbackHours
}
if ($JobName) { $config.JobName = $JobName }
$config | ConvertTo-Json | Set-Content -Path $ConfigPath -Encoding UTF8
New-Item -ItemType Directory -Path $InstallDir -Force | Out-Null
$scriptPath = Join-Path $InstallDir 'Send-MarsBackupStatus.ps1'
Copy-Item -Path (Join-Path $PSScriptRoot 'Send-MarsBackupStatus.ps1') -Destination $scriptPath -Force
Unblock-File -Path $scriptPath
$action = New-ScheduledTaskAction -Execute $WindowsPowerShell `
-Argument "-NoProfile -NonInteractive -ExecutionPolicy Bypass -File `"$scriptPath`""
$trigger = New-ScheduledTaskTrigger -Once -At (Get-Date).AddMinutes(1) `
-RepetitionInterval (New-TimeSpan -Minutes $IntervalMinutes)
$principal = New-ScheduledTaskPrincipal -UserId 'S-1-5-18' -LogonType ServiceAccount -RunLevel Highest
$settings = New-ScheduledTaskSettingsSet -ExecutionTimeLimit (New-TimeSpan -Minutes 30) `
-MultipleInstances IgnoreNew -StartWhenAvailable
Register-ScheduledTask -TaskName $TaskName -Action $action -Trigger $trigger -Principal $principal `
-Settings $settings -Description 'Sends Azure Backup (MARS) results to the Backup Dashboard' -Force | Out-Null
Write-Host "Installed. Config: $ConfigPath"
Write-Host "Task '$TaskName' runs every $IntervalMinutes minutes as SYSTEM, using Windows PowerShell."
Write-Host ''
Write-Host 'Next steps:'
Write-Host " 1. Preview what will be sent: & '$WindowsPowerShell' -File '$scriptPath' -DryRun"
Write-Host " 2. Backfill 30 days of history: & '$WindowsPowerShell' -File '$scriptPath' -LookbackHours 720"
Write-Host " 3. Check the log: Get-Content '$DataDir\mars-collector.log' -Tail 20"
+5 -3
View File
@@ -18,7 +18,8 @@ targets and which every supported Windows Server has.
```bash ```bash
python manage.py create-key mars-acme-fs01 python manage.py create-key mars-acme-fs01
``` ```
2. Copy this folder to the server and run: 2. Copy the whole `collectors` folder to the server (the installer needs `common`
as well as this folder), then from `collectors\mars` run:
```powershell ```powershell
.\Install-MarsCollector.ps1 -ApiUrl https://backups.example.com -Tenant "Acme Corp" .\Install-MarsCollector.ps1 -ApiUrl https://backups.example.com -Tenant "Acme Corp"
``` ```
@@ -33,8 +34,9 @@ targets and which every supported Windows Server has.
powershell.exe -File 'C:\Program Files\BackupDashboard\Send-MarsBackupStatus.ps1' -LookbackHours 720 powershell.exe -File 'C:\Program Files\BackupDashboard\Send-MarsBackupStatus.ps1' -LookbackHours 720
``` ```
It can be installed alongside the Veeam collector: it uses its own config, state, log and task. It can be installed alongside the other collectors: it uses its own config, state, log and task.
Re-run the installer to change settings or update the script. Re-run the installer to change settings or update the script. It also installs
`BackupDashboard.Collector.psm1`, the code shared by all collectors, next to the script.
## What it sends ## What it sends
+18 -111
View File
@@ -38,63 +38,22 @@ param(
$ErrorActionPreference = 'Stop' $ErrorActionPreference = 'Stop'
$DataDir = Split-Path -Parent $ConfigPath # Shared collector code: installed next to this script; in the repo, in ..\common
$LogPath = Join-Path $DataDir 'mars-collector.log' $sharedModule = @(
$StatePath = Join-Path $DataDir 'mars-collector.state.json' (Join-Path $PSScriptRoot 'BackupDashboard.Collector.psm1'),
$MaxLogBytes = 5MB (Join-Path (Join-Path (Join-Path $PSScriptRoot '..') 'common') 'BackupDashboard.Collector.psm1')
) | Where-Object { Test-Path $_ } | Select-Object -First 1
if (-not $sharedModule) {
Write-Host "BackupDashboard.Collector.psm1 not found next to $PSCommandPath; re-run the installer."
exit 2
}
Import-Module $sharedModule -Force
# MARS keeps a limited job history; this comfortably covers 30 days at 3 backups a day # MARS keeps a limited job history; this comfortably covers 30 days at 3 backups a day
$MaxPreviousJobs = 200 $MaxPreviousJobs = 200
$Invariant = [Globalization.CultureInfo]::InvariantCulture
# ---------- Helpers ---------- # ---------- Helpers ----------
function Write-Log {
param([string]$Message, [string]$Level = 'INFO')
$line = '{0} [{1}] {2}' -f (Get-Date).ToString('yyyy-MM-dd HH:mm:ss'), $Level, $Message
Write-Host $line
if ($DryRun) { return }
if ((Test-Path $LogPath) -and (Get-Item $LogPath).Length -gt $MaxLogBytes) {
Move-Item $LogPath "$LogPath.1" -Force
}
Add-Content -Path $LogPath -Value $line -Encoding UTF8
}
function Get-ErrorText($ErrorRecord) {
# Windows PowerShell puts the HTTP response body (e.g. the API's 422 explanation) here
if ($ErrorRecord.ErrorDetails -and $ErrorRecord.ErrorDetails.Message) {
return "$($ErrorRecord.Exception.Message) $($ErrorRecord.ErrorDetails.Message)"
}
return $ErrorRecord.Exception.Message
}
# The API needs an explicit UTC offset. DateTimeOffset keeps a Local/Unspecified time as local and a
# Utc time as +00:00.
function ConvertTo-IsoTime($Value) {
if ($null -eq $Value) { return $null }
$dt = [datetime]$Value
if ($dt.Year -le 1900) { return $null } # "not set" (MARS uses 1601-01-01)
return ([DateTimeOffset]$dt).ToString("yyyy-MM-dd'T'HH:mm:sszzz", $Invariant)
}
# First property of $Object, out of $Names, that exists and isn't empty. MARS object property names
# aren't documented, so reads go through this rather than assuming one spelling.
function Get-FirstValue($Object, [string[]]$Names) {
if ($null -eq $Object) { return $null }
foreach ($name in $Names) {
try {
$prop = $Object.PSObject.Properties[$name]
if (-not $prop) { continue }
$value = $prop.Value
# Check emptiness by type: casting to [string] would make a one-item list of objects ''
if ($null -eq $value) { continue }
if ($value -is [string] -and $value -eq '') { continue }
if ($value -is [Collections.ICollection] -and $value.Count -eq 0) { continue }
return $value
} catch { }
}
return $null
}
function Import-Mars { function Import-Mars {
if (Get-Command Get-OBJob -ErrorAction SilentlyContinue) { return } # already loaded if (Get-Command Get-OBJob -ErrorAction SilentlyContinue) { return } # already loaded
if (Get-Module -ListAvailable -Name MSOnlineBackup) { if (Get-Module -ListAvailable -Name MSOnlineBackup) {
@@ -305,49 +264,16 @@ function Test-IsBackupJob($Job) {
return (-not $type) -or $type -match 'Backup' return (-not $type) -or $type -match 'Backup'
} }
function Send-Report($Body) {
$json = $Body | ConvertTo-Json -Depth 6 -Compress
if ($DryRun) {
Write-Host $json
return
}
# Send bytes: Windows PowerShell would otherwise encode a string body as Latin-1
$params = @{
Uri = "$($Config.ApiUrl.TrimEnd('/'))/api/v1/ingest"
Method = 'Post'
Headers = @{ Authorization = "Bearer $($Config.ApiKey)" }
ContentType = 'application/json; charset=utf-8'
Body = [Text.Encoding]::UTF8.GetBytes($json)
TimeoutSec = 30
}
Invoke-RestMethod @params | Out-Null
}
# ---------- Main ---------- # ---------- Main ----------
try { try {
$Config = Get-Content -Path $ConfigPath -Raw | ConvertFrom-Json $Config = Initialize-Collector -Name 'mars' -ConfigPath $ConfigPath -LookbackHours $LookbackHours -DryRun:$DryRun
foreach ($key in 'ApiUrl', 'ApiKey', 'Tenant') {
if (-not $Config.$key) { throw "Config $ConfigPath is missing '$key'" }
}
$SourceSystem = if ($Config.SourceSystem) { [string]$Config.SourceSystem } else { 'Azure Backup (MARS)' } $SourceSystem = if ($Config.SourceSystem) { [string]$Config.SourceSystem } else { 'Azure Backup (MARS)' }
$JobName = if ($Config.JobName) { [string]$Config.JobName } else { "$env:COMPUTERNAME - Azure Backup" } $JobName = if ($Config.JobName) { [string]$Config.JobName } else { "$env:COMPUTERNAME - Azure Backup" }
if ($LookbackHours -le 0) { $LookbackHours = $Config.LookbackHours
$LookbackHours = if ($Config.LookbackHours) { [int]$Config.LookbackHours } else { 48 }
}
# Windows PowerShell 5.1 may default to TLS 1.0
[Net.ServicePointManager]::SecurityProtocol = [Net.ServicePointManager]::SecurityProtocol -bor [Net.SecurityProtocolType]::Tls12
Import-Mars Import-Mars
# Job ID -> start time (Unix seconds), for jobs already delivered in their final state
$sent = @{}
if (Test-Path $StatePath) {
$saved = Get-Content -Path $StatePath -Raw | ConvertFrom-Json
if ($saved) { foreach ($p in $saved.PSObject.Properties) { $sent[$p.Name] = [long]$p.Value } }
}
$sinceUtc = (Get-Date).ToUniversalTime().AddHours(-$LookbackHours) $sinceUtc = (Get-Date).ToUniversalTime().AddHours(-$LookbackHours)
# Get-OBJob with no arguments lists jobs in progress; -Previous lists finished ones # Get-OBJob with no arguments lists jobs in progress; -Previous lists finished ones
$jobs = @(@(Get-OBJob) + @(Get-OBJob -Previous $MaxPreviousJobs) | Where-Object { $jobs = @(@(Get-OBJob) + @(Get-OBJob -Previous $MaxPreviousJobs) | Where-Object {
@@ -360,7 +286,7 @@ try {
$byId[[string]$job.JobId] = $job # a job in both lists: the finished entry (later) wins $byId[[string]$job.JobId] = $job # a job in both lists: the finished entry (later) wins
} }
$pending = @($byId.Values | $pending = @($byId.Values |
Where-Object { -not $sent.ContainsKey([string]$_.JobId) } | Where-Object { -not (Test-RunSent ([string]$_.JobId)) } |
Sort-Object { Get-JobTimeUtc $_ 'StartTime' }) Sort-Object { Get-JobTimeUtc $_ 'StartTime' })
$jobInfo = [ordered]@{ $jobInfo = [ordered]@{
@@ -376,38 +302,19 @@ try {
} }
$failures = 0 $failures = 0
$runsSent = 0
try { try {
if ($pending.Count -eq 0) { if ($pending.Count -eq 0) {
# Keep next_run_at fresh even when nothing new ran # Keep next_run_at fresh even when nothing new ran
Send-Report ([ordered]@{ tenant = $Config.Tenant; source_system = $SourceSystem; job = $jobInfo }) Send-JobReport -Tenant $Config.Tenant -SourceSystem $SourceSystem -Job $jobInfo
} }
foreach ($job in $pending) { foreach ($job in $pending) {
$run = ConvertTo-Run $job $run = ConvertTo-Run $job
Send-Report ([ordered]@{ tenant = $Config.Tenant; source_system = $SourceSystem; job = $jobInfo; run = $run }) Send-JobReport -Tenant $Config.Tenant -SourceSystem $SourceSystem -Job $jobInfo -Run $run -SentKey $run.external_id -StartedAt ((Get-JobTimeUtc $job 'StartTime'))
$runsSent++
if ($run.status -ne 'running') {
$sent[$run.external_id] = ([DateTimeOffset](Get-JobTimeUtc $job 'StartTime')).ToUnixTimeSeconds()
}
} }
} catch { } catch {
$failures++ $failures++
Write-Log "Failed to report: $(Get-ErrorText $_)" 'ERROR' Write-Log "Failed to report: $(Get-ErrorText $_)" 'ERROR'
} }
if (-not $DryRun) { $exitCode = Complete-Collector -Failures $failures
# Forget jobs that have aged out of the lookback window (plus a day of slack) if ($exitCode) { exit $exitCode }
$cutoff = [DateTimeOffset]::Now.AddHours(-$LookbackHours - 24).ToUnixTimeSeconds()
$keep = [ordered]@{}
foreach ($id in $sent.Keys) {
if ($sent[$id] -ge $cutoff) { $keep[$id] = $sent[$id] }
}
try {
$keep | ConvertTo-Json -Compress | Set-Content -Path $StatePath -Encoding UTF8
} catch {
Write-Log "Couldn't save state file: $(Get-ErrorText $_)" 'WARN'
}
}
Write-Log ("Reported {0} runs, {1} failures" -f $runsSent, $failures)
if ($failures) { exit 1 }
@@ -0,0 +1,119 @@
<#
.SYNOPSIS
Read-only dump of what the Veeam Backup for Microsoft 365 cmdlets return, for troubleshooting
the collector.
.DESCRIPTION
The collector reads properties of Get-VBOJob / Get-VBOJobSession results whose names aren't
fully documented. This prints those objects' real structure (property names, types, values and
each date's Kind, a few levels deep) for a few jobs and their latest sessions. Nothing is
changed. Output is also saved to a text file.
.EXAMPLE
.\Get-Vb365CollectorDiagnostics.ps1
#>
[CmdletBinding()]
param(
[int]$Jobs = 3,
[int]$SessionsPerJob = 2,
[string]$OutFile = (Join-Path $env:TEMP 'vb365-collector-diagnostics.txt')
)
$ErrorActionPreference = 'Stop'
$MaxDepth = 4
if (-not (Get-Command Get-VBOJob -ErrorAction SilentlyContinue)) {
if (Get-Module -ListAvailable -Name Veeam.Archiver.PowerShell) {
Import-Module Veeam.Archiver.PowerShell
} else {
Import-Module (Join-Path $env:ProgramFiles 'Veeam\Backup365\Veeam.Archiver.PowerShell\Veeam.Archiver.PowerShell.psd1')
}
}
function Test-Scalar($Value) {
return $null -eq $Value -or $Value -is [string] -or $Value -is [ValueType] -or $Value -is [guid]
}
# Recursive "name (type) = value" listing. Collections show their first 3 items.
function Show-Object($Value, [string]$Indent = ' ', [int]$Depth = 0) {
if (Test-Scalar $Value) { return }
if ($Depth -ge $MaxDepth) { "$Indent..."; return }
if ($Value -is [Collections.IEnumerable]) {
$items = @($Value)
"$Indent[$($items.Count) item(s)]"
$i = 0
foreach ($item in $items | Select-Object -First 3) {
"$Indent[$i] ($($item.GetType().FullName))$(if (Test-Scalar $item) { " = $item" })"
Show-Object $item "$Indent " ($Depth + 1)
$i++
}
return
}
foreach ($p in $Value.PSObject.Properties) {
try { $v = $p.Value } catch { "$Indent$($p.Name) = <error: $($_.Exception.Message)>"; continue }
$type = if ($null -eq $v) { 'null' } else { $v.GetType().FullName }
if (Test-Scalar $v) {
$kind = if ($v -is [datetime]) { " [Kind=$($v.Kind)]" } else { '' }
"$Indent$($p.Name) ($type) = $v$kind"
} else {
$text = [string]$v
if ($text.Length -gt 100) { $text = $text.Substring(0, 97) + '...' }
"$Indent$($p.Name) ($type) = $text"
Show-Object $v "$Indent " ($Depth + 1)
}
}
}
function Show-Section([string]$Title, [scriptblock]$Get) {
"== $Title"
try {
$result = @(& $Get)
if ($result.Count -eq 0 -or ($result.Count -eq 1 -and $null -eq $result[0])) { ' (nothing returned)' }
$n = 0
foreach ($item in $result) {
if ($null -eq $item) { continue }
" [$n] ($($item.GetType().FullName))"
Show-Object $item ' '
$n++
}
} catch {
" error: $($_.Exception.Message)"
}
''
}
$report = & {
$module = Get-Module Veeam.Archiver.PowerShell
"Generated $(Get-Date -Format o) on $env:COMPUTERNAME, PowerShell $($PSVersionTable.PSVersion), module $($module.Version)"
"Local time zone: $([TimeZoneInfo]::Local.Id), UTC offset now $([TimeZoneInfo]::Local.GetUtcOffset((Get-Date))), culture $([Globalization.CultureInfo]::CurrentCulture.Name)"
''
"== Organizations"
try {
Get-VBOOrganization | ForEach-Object { " $($_.Name) [$($_.Type)]" }
} catch { " error: $($_.Exception.Message)" }
''
$allJobs = @(Get-VBOJob)
"== Jobs: $($allJobs.Count) in total"
$allJobs | ForEach-Object { " $($_.Name) [$($_.Id)] enabled=$($_.IsEnabled) next=$($_.NextRun)" }
''
Show-Section "Get-VBOJob (first $Jobs, full detail)" { $allJobs | Select-Object -First $Jobs }
foreach ($j in $allJobs | Select-Object -First $Jobs) {
Show-Section "Get-VBOJobSession -Job '$($j.Name)' (latest $SessionsPerJob)" {
Get-VBOJobSession -Job $j | Sort-Object CreationTime -Descending | Select-Object -First $SessionsPerJob
}
}
if (Get-Command Get-VBOCopyJob -ErrorAction SilentlyContinue) {
Show-Section 'Get-VBOCopyJob (first 2)' { Get-VBOCopyJob | Select-Object -First 2 }
if (Get-Command Get-VBOCopyJobSession -ErrorAction SilentlyContinue) {
Show-Section 'Get-VBOCopyJobSession (latest 2)' {
Get-VBOCopyJobSession | Sort-Object CreationTime -Descending | Select-Object -First 2
}
}
} else {
'== Get-VBOCopyJob: not available'
}
} | Out-String -Width 300
$report
$report | Set-Content -Path $OutFile -Encoding UTF8
Write-Host "Saved to $OutFile"
@@ -0,0 +1,74 @@
<#
.SYNOPSIS
Installs the Veeam Backup for Microsoft 365 collector as a SYSTEM scheduled task.
.DESCRIPTION
- Copies Send-Vb365BackupStatus.ps1 to Program Files (only admins can modify it, which
matters because it runs as SYSTEM).
- Writes the config to ProgramData\BackupDashboard and restricts that folder to SYSTEM and
Administrators, since it contains the API key.
- Registers a scheduled task that runs every IntervalMinutes, under Windows PowerShell 5.1
(powershell.exe) unless -PwshPath points at PowerShell 7.
Safe to re-run to change settings or update the script. Can sit alongside the other collectors.
Dashboard company per job: -OrganizationTenants maps Microsoft 365 organization names to
companies; jobs of unmapped organizations use -Tenant, or the organization's name if no -Tenant.
.EXAMPLE
.\Install-Vb365Collector.ps1 -ApiUrl https://backups.example.com -Tenant "Acme Corp"
One client per VB365 server. Prompts for the API key.
.EXAMPLE
.\Install-Vb365Collector.ps1 -ApiUrl https://backups.example.com -OrganizationTenants @{
'acme.onmicrosoft.com' = 'Acme Corp'; 'bluewater.onmicrosoft.com' = 'Bluewater Dental' }
A shared VB365 server protecting several organizations.
#>
#Requires -RunAsAdministrator
[CmdletBinding()]
param(
[Parameter(Mandatory)][string]$ApiUrl,
[string]$Tenant,
[hashtable]$OrganizationTenants = @{},
[string]$SourceSystem = 'Veeam Backup for M365',
[ValidateRange(5, 1440)][int]$IntervalMinutes = 60,
[ValidateRange(1, 720)][int]$LookbackHours = 48,
[string]$InstallDir = (Join-Path $env:ProgramFiles 'BackupDashboard'),
# Run under PowerShell 7 instead of Windows PowerShell, e.g. 'C:\Program Files\PowerShell\7\pwsh.exe'
[string]$PwshPath
)
$ErrorActionPreference = 'Stop'
$Shell = Join-Path $env:SystemRoot 'System32\WindowsPowerShell\v1.0\powershell.exe'
if ($PwshPath) {
if (-not (Test-Path $PwshPath)) { throw "PowerShell 7 not found at '$PwshPath'." }
$Shell = $PwshPath
}
# Check VB365 is here before changing anything
$moduleFile = Join-Path $env:ProgramFiles 'Veeam\Backup365\Veeam.Archiver.PowerShell\Veeam.Archiver.PowerShell.psd1'
if (-not (Get-Module -ListAvailable -Name Veeam.Archiver.PowerShell) -and -not (Test-Path $moduleFile)) {
throw 'The Veeam Backup for Microsoft 365 PowerShell module (Veeam.Archiver.PowerShell) is not installed on this server.'
}
if (-not $Tenant -and $OrganizationTenants.Count -eq 0) {
Write-Warning 'No -Tenant or -OrganizationTenants: jobs will be listed under their Microsoft 365 organization name.'
}
# Shared installer code: in the repo it's in ..\common, so copy the whole collectors folder
$sharedModule = @(
(Join-Path $PSScriptRoot 'BackupDashboard.Collector.psm1'),
(Join-Path (Join-Path (Join-Path $PSScriptRoot '..') 'common') 'BackupDashboard.Collector.psm1')
) | Where-Object { Test-Path $_ } | Select-Object -First 1
if (-not $sharedModule) {
throw "BackupDashboard.Collector.psm1 not found. Copy the whole collectors folder, including common."
}
Import-Module $sharedModule -Force
$settings = [ordered]@{
SourceSystem = $SourceSystem
LookbackHours = $LookbackHours
OrganizationTenants = $OrganizationTenants
}
if ($Tenant) { $settings.Tenant = $Tenant }
Install-CollectorTask -Name 'vb365' -ScriptPath (Join-Path $PSScriptRoot 'Send-Vb365BackupStatus.ps1') `
-TaskName 'Backup Dashboard - VB365 collector' `
-TaskDescription 'Sends Veeam Backup for Microsoft 365 results to the Backup Dashboard' `
-Shell $Shell -ApiUrl $ApiUrl -IntervalMinutes $IntervalMinutes -InstallDir $InstallDir -Settings $settings
+88
View File
@@ -0,0 +1,88 @@
# Veeam Backup for Microsoft 365 collector
A PowerShell script that runs on the Veeam Backup for Microsoft 365 (VB365) server and reports its
backup jobs (and backup copy jobs) to the dashboard, one dashboard job per VB365 job.
> **Checked against:** VB365 8.4 under Windows PowerShell 5.1, a shared server protecting 11
> organizations: successful, warning and retried sessions. Not yet seen on a real server: a failed
> session, a session in progress, or a backup copy job. If one looks wrong, run
> `Get-Vb365CollectorDiagnostics.ps1` while it's in that state and send the output.
## Which company a job appears under
One VB365 server can protect several Microsoft 365 organizations, so the company is picked per job
from the job's organization:
1. `OrganizationTenants` maps an organization name (as `Get-VBOOrganization` shows it, e.g.
`acme.onmicrosoft.com`; not case-sensitive) to a dashboard company.
2. Otherwise `Tenant`, for a VB365 server dedicated to one client.
3. Otherwise the organization's own name.
Changing a job's company later creates a new dashboard job under the new company; delete the old
one with `python manage.py delete-job <id>`.
## Install (on the VB365 server, as an administrator)
1. On the dashboard host, create a key for this server:
```bash
python manage.py create-key vb365-01
```
2. Copy the whole `collectors` folder to the server (the installer needs `common`
as well as this folder), then from `collectors\vb365` run one of:
```powershell
# VB365 server for one client
.\Install-Vb365Collector.ps1 -ApiUrl https://backups.example.com -Tenant "Acme Corp"
# Shared VB365 server
.\Install-Vb365Collector.ps1 -ApiUrl https://backups.example.com -OrganizationTenants @{
'acme.onmicrosoft.com' = 'Acme Corp'
'bluewater.onmicrosoft.com' = 'Bluewater Dental'
}
```
It prompts for the key, then:
- writes `C:\ProgramData\BackupDashboard\vb365-collector.json` (readable only by SYSTEM and Administrators)
- copies the script to `C:\Program Files\BackupDashboard\`
- registers the scheduled task **Backup Dashboard - VB365 collector** (SYSTEM, every 60 minutes)
It runs under Windows PowerShell 5.1. If your VB365 version's module needs PowerShell 7, add
`-PwshPath 'C:\Program Files\PowerShell\7\pwsh.exe'`.
3. Preview the payloads, then backfill 30 days so the history view is populated straight away:
```powershell
powershell.exe -File 'C:\Program Files\BackupDashboard\Send-Vb365BackupStatus.ps1' -DryRun
powershell.exe -File 'C:\Program Files\BackupDashboard\Send-Vb365BackupStatus.ps1' -LookbackHours 720
```
It can be installed alongside the other collectors: it uses its own config, state, log and task.
Re-run the installer to change settings (including the organization mapping) or update the script. It also installs
`BackupDashboard.Collector.psm1`, the code shared by all collectors, next to the script.
## What it sends
| Dashboard | VB365 |
|---|---|
| Company | See above |
| Job | Each backup job and backup copy job. The job's GUID is the ID, so renaming it in VB365 doesn't duplicate it |
| Status | Session status: Success / Warning / Failed. Running or queued → **Running**. Stopped (cancelled) or unrecognised → **Warning** |
| Started / finished | Session `CreationTime` / `EndTime` |
| Next scheduled | Job's `NextRun`. Empty for disabled jobs |
| Message | Warning and error lines from the session log (items titled `[Warning] ...`, `[Failed] ...`), plus "Veeam will retry this job." when a retry is pending |
| Retries | VB365 runs a retry as a separate session, so it appears as its own run, marked `details.retry_of` (the session it retried). The retried run is marked `details.will_retry` |
| Size | Data transferred (`Statistics.TransferredData`) |
| Details | Organization, objects processed, job kind (`backup` / `copy`) |
Each run covers the last 48 hours (`LookbackHours`). Sessions already delivered are recorded in
`vb365-collector.state.json`.
## Troubleshooting
- Log: `C:\ProgramData\BackupDashboard\vb365-collector.log`. Exit codes: `0` OK, `1` some jobs
failed to send (retried next run), `2` setup failed (config or VB365 module).
- Wrong status, times or messages: run `.\Get-Vb365CollectorDiagnostics.ps1` (read-only). It prints
the real structure of jobs and sessions, including whether each date is local or UTC.
## Testing without VB365
```powershell
. ./tests/MockVb365.ps1
./Send-Vb365BackupStatus.ps1 -ConfigPath ./my-test-config.json -DryRun
```
+278
View File
@@ -0,0 +1,278 @@
<#
.SYNOPSIS
Reports Veeam Backup for Microsoft 365 (VB365) job results to the Backup Dashboard.
.DESCRIPTION
Runs on the VB365 server, normally as a SYSTEM scheduled task every 60 minutes
(see Install-Vb365Collector.ps1). For every backup job (and backup copy job) it sends:
- the job's next scheduled run, which drives the dashboard's Overdue flag, and
- every session from the last LookbackHours that the dashboard hasn't already received in
its final state. Running sessions are re-sent each time until they finish.
One VB365 server can protect several Microsoft 365 organizations, so the dashboard company is
chosen per job from its organization: OrganizationTenants in the config maps an organization
name (e.g. "contoso.onmicrosoft.com") to a company; otherwise Tenant is used; otherwise the
organization's own name.
Already-reported sessions are tracked in a state file, and the ingest API is idempotent, so a
failed or repeated run is harmless: anything not confirmed is retried next time.
.PARAMETER ConfigPath
JSON config with ApiUrl and ApiKey (optional: Tenant, OrganizationTenants, SourceSystem,
LookbackHours).
.PARAMETER LookbackHours
Overrides the config value. Use 720 once to backfill 30 days of history.
.PARAMETER DryRun
Print the payloads instead of sending them. Doesn't touch the state or log files.
.EXAMPLE
.\Send-Vb365BackupStatus.ps1 -DryRun
.EXAMPLE
.\Send-Vb365BackupStatus.ps1 -LookbackHours 720
#>
[CmdletBinding()]
param(
[string]$ConfigPath = (Join-Path $env:ProgramData 'BackupDashboard\vb365-collector.json'),
[int]$LookbackHours = 0,
[switch]$DryRun
)
$ErrorActionPreference = 'Stop'
# Shared collector code: installed next to this script; in the repo, in ..\common
$sharedModule = @(
(Join-Path $PSScriptRoot 'BackupDashboard.Collector.psm1'),
(Join-Path (Join-Path (Join-Path $PSScriptRoot '..') 'common') 'BackupDashboard.Collector.psm1')
) | Where-Object { Test-Path $_ } | Select-Object -First 1
if (-not $sharedModule) {
Write-Host "BackupDashboard.Collector.psm1 not found next to $PSCommandPath; re-run the installer."
exit 2
}
Import-Module $sharedModule -Force
$Invariant = [Globalization.CultureInfo]::InvariantCulture
# ---------- Helpers ----------
function Import-Vb365 {
if (Get-Command Get-VBOJob -ErrorAction SilentlyContinue) { return } # already loaded
if (Get-Module -ListAvailable -Name Veeam.Archiver.PowerShell) {
Import-Module Veeam.Archiver.PowerShell
return
}
# Fall back to the default install path
Import-Module (Join-Path $env:ProgramFiles 'Veeam\Backup365\Veeam.Archiver.PowerShell\Veeam.Archiver.PowerShell.psd1')
}
function Get-Time($Object, [string]$Field) {
$value = Get-FirstValue $Object @($Field)
if ($null -eq $value) { return $null }
$dt = [datetime]$value
if ($dt.Year -le 1900) { return $null } # e.g. EndTime while a session is running
return $dt
}
# "1.5 GB", "512 MB", "123456" -> bytes; $null when it can't be read
function ConvertTo-Bytes($Value) {
if ($null -eq $Value) { return $null }
if ($Value -is [ValueType]) { return [long]$Value }
$match = [regex]::Match([string]$Value, '^\s*([\d.,]+)\s*([KMGTP]?B)?\s*$', 'IgnoreCase')
if (-not $match.Success) { return $null }
$number = 0.0
if (-not [double]::TryParse(($match.Groups[1].Value -replace ',', '.'), [Globalization.NumberStyles]::Float, $Invariant, [ref]$number)) {
return $null
}
$power = @{ '' = 0; 'B' = 0; 'KB' = 1; 'MB' = 2; 'GB' = 3; 'TB' = 4; 'PB' = 5 }[$match.Groups[2].Value.ToUpperInvariant()]
return [long]($number * [math]::Pow(1024, $power))
}
# ---------- Jobs and tenants ----------
function Get-OrganizationName($Job) {
$org = Get-FirstValue $Job @('Organization')
if ($null -eq $org) {
# Copy jobs point at their source backup job
$org = Get-FirstValue (Get-FirstValue $Job @('BackupJob')) @('Organization')
}
if ($org -is [string]) { return $org }
$name = Get-FirstValue $org @('Name', 'OfficeName', 'DisplayName')
if ($name) { return [string]$name }
return $null
}
function Get-TenantForJob($Job) {
$org = Get-OrganizationName $Job
if ($org -and $Config.OrganizationTenants) {
$mapped = $Config.OrganizationTenants.PSObject.Properties[$org] # case-insensitive lookup
if ($mapped -and $mapped.Value) { return [string]$mapped.Value }
}
if ($Config.Tenant) { return [string]$Config.Tenant }
if ($org) { return $org }
return $env:COMPUTERNAME
}
function Get-NextRun($Job) {
try {
$enabled = Get-FirstValue $Job @('IsEnabled', 'Enabled')
if ($enabled -eq $false) { return $null }
return ConvertTo-IsoTime (Get-Time $Job 'NextRun')
} catch {
Write-Log "Couldn't read schedule for job '$($Job.Name)': $(Get-ErrorText $_)" 'WARN'
return $null
}
}
# ---------- Sessions ----------
function Get-RunStatus($Session) {
$status = [string](Get-FirstValue $Session @('Status', 'Result', 'State'))
if ($status -match 'Fail|Error') { return 'failed' }
if ($status -match 'Warn') { return 'warning' }
if ($status -match 'Succe') { return 'success' }
if ($status -match 'Run|Queue|Progress|Start|Wait') { return 'running' }
if (-not (Get-Time $Session 'EndTime')) { return 'running' }
# Stopped (cancelled), Disconnected, or a state we don't recognise: surface it rather than hide it
return 'warning'
}
function Test-EmptyGuid($Value) {
return (-not $Value) -or [string]$Value -eq [string][guid]::Empty
}
# Warning/error lines from the session log. Each log item's status is a prefix of its Title, e.g.
# "[Warning] Item was skipped" (there's no separate status property), so match on that rather than
# on words in the text, which would catch e.g. "[Success] Processing site: .../error-reports".
function Get-RunMessage($Session, [string]$Status) {
if ($Status -eq 'running') { return '' }
$lines = New-Object 'System.Collections.Generic.List[string]'
$sessionStatus = [string](Get-FirstValue $Session @('Status', 'Result', 'State'))
if ($Status -ne 'success') {
foreach ($entry in @(Get-FirstValue $Session @('Log', 'Logs') | Where-Object { $_ })) {
$title = ([string](Get-FirstValue $entry @('Title', 'Message', 'Text'))).Trim()
$prefix = [regex]::Match($title, '^\[(\w+)\]\s*')
if ($prefix.Success) {
if ($prefix.Groups[1].Value -notmatch 'Warn|Fail|Error') { continue }
$title = $title.Substring($prefix.Length)
} elseif ([string](Get-FirstValue $entry @('Status', 'Type', 'Severity')) -notmatch 'Warn|Fail|Error') {
continue
}
if ($title -and -not $lines.Contains($title)) { $lines.Add($title) }
}
}
if ($lines.Count -eq 0) {
if ($Status -eq 'success') { $lines.Add('Job completed successfully.') }
else { $lines.Add("Job finished with status '$sessionStatus'.") }
} elseif ($sessionStatus -and $sessionStatus -notmatch 'Fail|Warn') {
$lines.Insert(0, "Session status: $sessionStatus")
}
# A warning/failure VB365 is about to retry; the retry arrives as its own run
if ($Status -ne 'success' -and (Get-FirstValue $Session @('JobWillBeRetried')) -eq $true) {
$lines.Add('Veeam will retry this job.')
}
return ($lines -join "`n")
}
function ConvertTo-Run($Session, $Job, [string]$JobKind) {
$status = Get-RunStatus $Session
$details = [ordered]@{ vb365_server = $env:COMPUTERNAME; job_kind = $JobKind }
$org = Get-OrganizationName $Job
if ($org) { $details.organization = $org }
$sessionStatus = Get-FirstValue $Session @('Status', 'Result', 'State')
if ($sessionStatus) { $details.vb365_status = [string]$sessionStatus }
# Retries are separate sessions pointing at the session they retry
$mainSession = Get-FirstValue $Session @('MainSessionId')
if (-not (Test-EmptyGuid $mainSession)) { $details.retry_of = [string]$mainSession }
if ((Get-FirstValue $Session @('JobWillBeRetried')) -eq $true) { $details.will_retry = $true }
$run = [ordered]@{
external_id = [string](Get-FirstValue $Session @('Id', 'SessionId'))
status = $status
started_at = ConvertTo-IsoTime (Get-Time $Session 'CreationTime')
message = Get-RunMessage $Session $status
details = $details
}
if ($status -ne 'running') {
$finished = ConvertTo-IsoTime (Get-Time $Session 'EndTime')
if ($finished) { $run.finished_at = $finished } # otherwise the API uses receipt time
$stats = Get-FirstValue $Session @('Statistics')
$processed = Get-FirstValue $stats @('ProcessedObjects')
if ($null -ne $processed) { $details.processed_objects = $processed }
$transferred = ConvertTo-Bytes (Get-FirstValue $stats @('TransferredData', 'TransferredDataSize'))
if ($transferred -gt 0) { $run.size_bytes = $transferred }
}
return $run
}
# ---------- Main ----------
try {
$Config = Initialize-Collector -Name 'vb365' -ConfigPath $ConfigPath -LookbackHours $LookbackHours -DryRun:$DryRun -RequiredKeys 'ApiUrl', 'ApiKey'
$SourceSystem = if ($Config.SourceSystem) { [string]$Config.SourceSystem } else { 'Veeam Backup for M365' }
$LookbackHours = $Config.LookbackHours
Import-Vb365
$since = (Get-Date).AddHours(-$LookbackHours)
# (job, kind, sessions) for backup jobs, plus backup copy jobs where the cmdlets exist
$work = New-Object 'System.Collections.Generic.List[object]'
foreach ($job in @(Get-VBOJob)) {
$sessions = @(Get-VBOJobSession -Job $job | Where-Object { (Get-Time $_ 'CreationTime') -ge $since })
$work.Add([pscustomobject]@{ Job = $job; Kind = 'backup'; Sessions = $sessions })
}
if ((Get-Command Get-VBOCopyJob -ErrorAction SilentlyContinue) -and (Get-Command Get-VBOCopyJobSession -ErrorAction SilentlyContinue)) {
$copyJobs = @(Get-VBOCopyJob)
if ($copyJobs.Count) {
# Fetch once and match by job ID, since the per-job parameter name isn't consistent
$copySessions = @(Get-VBOCopyJobSession | Where-Object { (Get-Time $_ 'CreationTime') -ge $since })
foreach ($job in $copyJobs) {
$id = [string]$job.Id
$work.Add([pscustomobject]@{
Job = $job; Kind = 'copy'
Sessions = @($copySessions | Where-Object { [string](Get-FirstValue $_ @('JobId')) -eq $id })
})
}
}
}
} catch {
Write-Log "Collector setup failed: $(Get-ErrorText $_)" 'ERROR'
exit 2
}
$failures = 0
foreach ($item in $work) {
$job = $item.Job
$tenant = Get-TenantForJob $job
$jobInfo = [ordered]@{
external_id = [string]$job.Id
name = [string]$job.Name
# Always sent (null when disabled/unscheduled) so the dashboard's value tracks VB365's
next_run_at = Get-NextRun $job
}
$pending = @($item.Sessions |
Where-Object { $_ -and -not (Test-RunSent ([string](Get-FirstValue $_ @('Id', 'SessionId')))) } |
Sort-Object { Get-Time $_ 'CreationTime' })
try {
if ($pending.Count -eq 0) {
# Keep next_run_at fresh even when nothing new ran
Send-JobReport -Tenant $tenant -SourceSystem $SourceSystem -Job $jobInfo
}
foreach ($session in $pending) {
$run = ConvertTo-Run $session $job $item.Kind
Send-JobReport -Tenant $tenant -SourceSystem $SourceSystem -Job $jobInfo -Run $run -SentKey $run.external_id -StartedAt ((Get-Time $session 'CreationTime'))
}
} catch {
$failures++
Write-Log "Failed to report job '$($job.Name)': $(Get-ErrorText $_)" 'ERROR'
if (Test-AuthRejected $_) {
Write-Log 'API key rejected; check ApiKey in the config. Stopping.' 'ERROR'
break
}
}
}
$exitCode = Complete-Collector -JobCount $work.Count -Failures $failures
if ($exitCode) { exit $exitCode }
+78
View File
@@ -0,0 +1,78 @@
# Stand-ins for the Veeam Backup for Microsoft 365 cmdlets. Object shapes follow
# Get-Vb365CollectorDiagnostics.ps1 output from a real VB365 8.4 server: session and NextRun times
# are local, Statistics.TransferredData is a string ("187.2 MB"), log items carry their status as a
# Title prefix ("[Warning] ..."), and retries are separate sessions whose MainSessionId points at
# the session they retry (which has JobWillBeRetried = True).
#
# . ./tests/MockVb365.ps1
# ./Send-Vb365BackupStatus.ps1 -ConfigPath ./my-test-config.json -DryRun
# Fixed to the top of the hour, and IDs derived from names/times, so repeated runs see the same data
$now = Get-Date -Minute 0 -Second 0 -Millisecond 0
function New-StableGuid([string]$Seed) {
[guid]::new([Security.Cryptography.MD5]::Create().ComputeHash([Text.Encoding]::UTF8.GetBytes($Seed)))
}
function New-MockOrg($Name) { [pscustomobject]@{ Name = $Name; OfficeName = $Name; Type = 'Office365'; IsBackedUp = $true } }
$acme = New-MockOrg 'acme.onmicrosoft.com'
$blue = New-MockOrg 'bluewater.onmicrosoft.com'
function New-MockJob($Name, $Org, $NextRun, [bool]$Enabled = $true) {
[pscustomobject]@{
Id = New-StableGuid $Name; Name = $Name; Organization = $Org; JobType = 'Backup'
IsEnabled = $Enabled; NextRun = $NextRun; LastStatus = 'Success'
LastRun = $(if ($NextRun) { $NextRun.AddDays(-1).ToUniversalTime() }) # UTC, as on the real server
}
}
function New-LogItem($Status, $Text, $At) {
[pscustomobject]@{ Id = [guid]::NewGuid(); USN = 1; Title = "[$Status] $Text"; CreationTime = $At; EndTime = $At }
}
function New-MockSession($Job, $Start, $End, $Status, $Transferred = '0 B', [long]$Objects = 0, $Log = @(),
$MainSessionId = [guid]::Empty, [bool]$WillRetry = $false) {
$logItems = @((New-LogItem 'Success' "Job started at: $($Start.ToString('G'))" $Start)) + @($Log)
[pscustomobject]@{
JobId = $Job.Id; JobName = $Job.Name; Status = $Status
Statistics = [pscustomobject]@{ ProcessingRate = '1.6 MB/s (4 items/s)'; TransferredData = $Transferred; ProcessedObjects = $Objects; Bottleneck = 'Target' }
JobSessionConfigType = 'Incremental'
Id = New-StableGuid "$($Job.Name)|$($Start.Ticks)"
CreationTime = $Start
EndTime = $(if ($End) { $End } else { [datetime]::MinValue })
Progress = 49; MainSessionId = $MainSessionId; JobWillBeRetried = $WillRetry
Log = $logItems
}
}
$global:MockVboJobs = @(
(New-MockJob 'Acme 365 Backup' $acme $now.Date.AddHours(17).AddDays($(if ($now.Hour -ge 17) { 1 } else { 0 }))),
(New-MockJob 'Bluewater 365 Backup' $blue $now.Date.AddHours(17.75).AddDays($(if ($now.Hour -ge 17) { 1 } else { 0 }))),
(New-MockJob 'Bluewater Old Mailboxes' $blue $now.AddDays(1) $false) # disabled
)
$acmeJob, $blueJob, $oldJob = $global:MockVboJobs
$y = $now.Date.AddDays(-1) # yesterday
$blueWarn = New-MockSession $blueJob $y.AddHours(17.75) $y.AddHours(17.75).AddMinutes(3.5) 'Warning' '32.9 MB' 47 @(
(New-LogItem 'Success' 'Found 94 objects' $y.AddHours(17.75)),
# A success line mentioning "error" must not end up in the message
(New-LogItem 'Success' 'Processing site: https://bluewater.sharepoint.com/sites/error-reports' $y.AddHours(17.76)),
(New-LogItem 'Warning' 'Processing mailbox [email protected] finished with warning: Item "Q3.pst" exceeds the maximum size' $y.AddHours(17.78))
) -WillRetry $true
$global:MockVboSessions = @(
(New-MockSession $acmeJob $y.AddHours(17) $y.AddHours(17).AddMinutes(4.8) 'Success' '187.2 MB' 568 @(
(New-LogItem 'Success' 'Found 49 objects' $y.AddHours(17)), (New-LogItem 'Success' 'Connected to organization' $y.AddHours(17)))),
(New-MockSession $acmeJob $now.AddMinutes(-20) $null 'Running' '0 B' 12),
$blueWarn,
# Its retry 13 minutes later, which succeeded
(New-MockSession $blueJob $y.AddHours(17.75).AddMinutes(13.5) $y.AddHours(17.75).AddMinutes(16.8) 'Success' '0 B' 0 -MainSessionId $blueWarn.Id),
(New-MockSession $blueJob $now.AddHours(-30) $now.AddHours(-29.9) 'Failed' '0 B' 0 @(
(New-LogItem 'Failed' 'Failed to connect to organization: The remote server returned an error: (401) Unauthorized.' $now.AddHours(-30)))),
(New-MockSession $oldJob $now.AddDays(-5) $now.AddDays(-5).AddHours(1) 'Success') # outside 48h lookback
)
function Get-VBOJob { $global:MockVboJobs }
function Get-VBOJobSession { param($Job) $global:MockVboSessions | Where-Object { -not $Job -or $_.JobId -eq $Job.Id } }
function Get-VBOOrganization { $acme, $blue }
function Get-VBOCopyJob { } # none, as on the real server
function Get-VBOCopyJobSession { }
@@ -9,6 +9,7 @@
2. sessions whose JobId matches no job (the runs that are going missing) 2. sessions whose JobId matches no job (the runs that are going missing)
3. for jobs with no matching sessions, their type/link-related properties and methods 3. for jobs with no matching sessions, their type/link-related properties and methods
4. what other job cmdlets (e.g. Get-VBRBackupCopyJob) return 4. what other job cmdlets (e.g. Get-VBRBackupCopyJob) return
5. each job's schedule properties, which the collector reads for the next run
Nothing is changed in Veeam or the dashboard. Output is also saved to a text file. Nothing is changed in Veeam or the dashboard. Output is also saved to a text file.
.EXAMPLE .EXAMPLE
@@ -42,14 +43,14 @@ function Format-Value($Value) {
# Property and method names matching $LinkPattern, with property values. Every read is guarded: # Property and method names matching $LinkPattern, with property values. Every read is guarded:
# some Veeam properties throw when accessed. # some Veeam properties throw when accessed.
function Show-LinkMembers($Object) { function Show-LinkMembers($Object, [string]$Pattern = $LinkPattern, [string]$Indent = ' ') {
foreach ($p in $Object.PSObject.Properties | Where-Object { $_.Name -match $LinkPattern }) { foreach ($p in $Object.PSObject.Properties | Where-Object { $_.Name -match $Pattern }) {
$value = try { Format-Value $p.Value } catch { "<error: $($_.Exception.Message)>" } $value = try { Format-Value $p.Value } catch { "<error: $($_.Exception.Message)>" }
' {0} = {1}' -f $p.Name, $value '{0}{1} = {2}' -f $Indent, $p.Name, $value
} }
$methods = @($Object | Get-Member -MemberType Methods -ErrorAction SilentlyContinue | $methods = @($Object | Get-Member -MemberType Methods -ErrorAction SilentlyContinue |
Where-Object { $_.Name -match $LinkPattern } | ForEach-Object Name) Where-Object { $_.Name -match $Pattern } | ForEach-Object Name)
if ($methods) { ' methods: ' + ($methods -join ', ') } if ($methods) { $Indent + 'methods: ' + ($methods -join ', ') }
} }
function Get-Prop($Object, [string]$Name) { function Get-Prop($Object, [string]$Name) {
@@ -147,6 +148,39 @@ $report = & {
" ${cmd}: error: $($_.Exception.Message)" " ${cmd}: error: $($_.Exception.Message)"
} }
} }
''
# The collector uses IsScheduleEnabled and ScheduleOptions.NextRun; show what else is there in
# case this version keeps the next run somewhere different
"== 5. Schedules (culture: $([Globalization.CultureInfo]::CurrentCulture.Name))"
$SchedulePattern = 'Schedul|Next|Latest|Run|Enabled|Daily|Periodic|Monthly|Continuous|After|Option'
foreach ($j in $jobs) {
" $($j.Name) [$(Get-Prop $j 'JobType')]"
Show-LinkMembers $j $SchedulePattern ' '
foreach ($source in 'ScheduleOptions', 'GetScheduleOptions()', 'Get-VBRJobScheduleOptions') {
try {
$opts = switch ($source) {
'ScheduleOptions' { $j.ScheduleOptions }
'GetScheduleOptions()' { $j.GetScheduleOptions() }
'Get-VBRJobScheduleOptions' { Get-VBRJobScheduleOptions -Job $j -WarningAction SilentlyContinue }
}
if ($null -eq $opts) { " ${source}: <null>"; continue }
" ${source} ($($opts.GetType().FullName)):"
foreach ($p in $opts.PSObject.Properties) {
$value = try { $p.Value } catch { "<error: $($_.Exception.Message)>" }
$type = if ($null -eq $value) { 'null' } else { $value.GetType().Name }
if ($value -is [string] -or $value -is [ValueType] -or $null -eq $value) {
' {0} ({1}) = {2}' -f $p.Name, $type, (Format-Value $value)
} else {
' {0} ({1})' -f $p.Name, $type
}
}
} catch {
" ${source}: not available: $($_.Exception.Message)"
}
}
''
}
} | Out-String -Width 300 } | Out-String -Width 300
$report $report
+13 -47
View File
@@ -30,9 +30,6 @@ param(
) )
$ErrorActionPreference = 'Stop' $ErrorActionPreference = 'Stop'
$TaskName = 'Backup Dashboard - Veeam collector'
$DataDir = Join-Path $env:ProgramData 'BackupDashboard'
$ConfigPath = Join-Path $DataDir 'veeam-collector.json'
# Find PowerShell 7 before changing anything, so a missing install fails cleanly # Find PowerShell 7 before changing anything, so a missing install fails cleanly
if (-not $PwshPath) { if (-not $PwshPath) {
@@ -54,49 +51,18 @@ if ($LASTEXITCODE -or ($pwshMajor -as [int]) -lt 7) {
throw "'$PwshPath' isn't a working PowerShell 7 (reported '$pwshVersion')." throw "'$PwshPath' isn't a working PowerShell 7 (reported '$pwshVersion')."
} }
if ($ApiUrl -notmatch '^https://') { # Shared installer code: in the repo it's in ..\common, so copy the whole collectors folder
Write-Warning 'ApiUrl is not HTTPS: the API key would be sent in clear text.' $sharedModule = @(
(Join-Path $PSScriptRoot 'BackupDashboard.Collector.psm1'),
(Join-Path (Join-Path (Join-Path $PSScriptRoot '..') 'common') 'BackupDashboard.Collector.psm1')
) | Where-Object { Test-Path $_ } | Select-Object -First 1
if (-not $sharedModule) {
throw "BackupDashboard.Collector.psm1 not found. Copy the whole collectors folder, including common."
} }
Import-Module $sharedModule -Force
# Read the key as a SecureString so it isn't echoed to the console or kept in shell history $settings = [ordered]@{ Tenant = $Tenant; SourceSystem = $SourceSystem; LookbackHours = $LookbackHours }
$secure = Read-Host -AsSecureString 'API key (from: python manage.py create-key <name>)' Install-CollectorTask -Name 'veeam' -ScriptPath (Join-Path $PSScriptRoot 'Send-VeeamBackupStatus.ps1') `
$bstr = [Runtime.InteropServices.Marshal]::SecureStringToBSTR($secure) -TaskName 'Backup Dashboard - Veeam collector' `
try { $apiKey = [Runtime.InteropServices.Marshal]::PtrToStringBSTR($bstr) } -TaskDescription 'Sends Veeam job results to the Backup Dashboard' `
finally { [Runtime.InteropServices.Marshal]::ZeroFreeBSTR($bstr) } -Shell $PwshPath -ApiUrl $ApiUrl -IntervalMinutes $IntervalMinutes -InstallDir $InstallDir -Settings $settings
if (-not $apiKey.StartsWith('bdk_')) { throw "That doesn't look like a dashboard API key (expected 'bdk_...')." }
# Config folder: SYSTEM + Administrators only (SIDs so this works on non-English Windows)
New-Item -ItemType Directory -Path $DataDir -Force | Out-Null
& icacls.exe $DataDir /inheritance:r /grant:r '*S-1-5-18:(OI)(CI)F' '*S-1-5-32-544:(OI)(CI)F' | Out-Null
if ($LASTEXITCODE) { throw "icacls failed on $DataDir" }
[ordered]@{
ApiUrl = $ApiUrl.TrimEnd('/')
ApiKey = $apiKey
Tenant = $Tenant
SourceSystem = $SourceSystem
LookbackHours = $LookbackHours
} | ConvertTo-Json | Set-Content -Path $ConfigPath -Encoding UTF8
New-Item -ItemType Directory -Path $InstallDir -Force | Out-Null
$scriptPath = Join-Path $InstallDir 'Send-VeeamBackupStatus.ps1'
Copy-Item -Path (Join-Path $PSScriptRoot 'Send-VeeamBackupStatus.ps1') -Destination $scriptPath -Force
Unblock-File -Path $scriptPath
$action = New-ScheduledTaskAction -Execute $PwshPath `
-Argument "-NoProfile -NonInteractive -ExecutionPolicy Bypass -File `"$scriptPath`""
$trigger = New-ScheduledTaskTrigger -Once -At (Get-Date).AddMinutes(1) `
-RepetitionInterval (New-TimeSpan -Minutes $IntervalMinutes)
$principal = New-ScheduledTaskPrincipal -UserId 'S-1-5-18' -LogonType ServiceAccount -RunLevel Highest
$settings = New-ScheduledTaskSettingsSet -ExecutionTimeLimit (New-TimeSpan -Minutes 30) `
-MultipleInstances IgnoreNew -StartWhenAvailable
Register-ScheduledTask -TaskName $TaskName -Action $action -Trigger $trigger -Principal $principal `
-Settings $settings -Description 'Sends Veeam job results to the Backup Dashboard' -Force | Out-Null
Write-Host "Installed. Config: $ConfigPath"
Write-Host "Task '$TaskName' runs every $IntervalMinutes minutes as SYSTEM, using PowerShell $pwshVersion ($PwshPath)."
Write-Host ''
Write-Host 'Next steps:'
Write-Host " 1. Preview what will be sent: & '$PwshPath' -File '$scriptPath' -DryRun"
Write-Host " 2. Backfill 30 days of history: & '$PwshPath' -File '$scriptPath' -LookbackHours 720"
Write-Host " 3. Check the log: Get-Content '$DataDir\veeam-collector.log' -Tail 20"
+5 -3
View File
@@ -13,7 +13,8 @@ account can't run it. The installer checks for this before changing anything.
```bash ```bash
python manage.py create-key veeam-acme python manage.py create-key veeam-acme
``` ```
2. Copy this folder to the VBR server and run, from PowerShell 7 (`pwsh`): 2. Copy the whole `collectors` folder to the VBR server (the installer needs `common`
as well as this folder), then from `collectors\veeam` run, from PowerShell 7 (`pwsh`):
```powershell ```powershell
.\Install-VeeamCollector.ps1 -ApiUrl https://backups.example.com -Tenant "Acme Corp" .\Install-VeeamCollector.ps1 -ApiUrl https://backups.example.com -Tenant "Acme Corp"
``` ```
@@ -29,7 +30,8 @@ account can't run it. The installer checks for this before changing anything.
& 'C:\Program Files\BackupDashboard\Send-VeeamBackupStatus.ps1' -LookbackHours 720 & 'C:\Program Files\BackupDashboard\Send-VeeamBackupStatus.ps1' -LookbackHours 720
``` ```
Re-run the installer to change settings or update the script. To remove the collector, Re-run the installer to change settings or update the script. It also installs
`BackupDashboard.Collector.psm1`, the code shared by all collectors, next to the script. To remove the collector,
delete the scheduled task and both `BackupDashboard` folders. delete the scheduled task and both `BackupDashboard` folders.
## What it sends ## What it sends
@@ -40,7 +42,7 @@ delete the scheduled task and both `BackupDashboard` folders.
| Job | Job name. The job's GUID is the ID, so renaming a job in Veeam doesn't duplicate it | | Job | Job name. The job's GUID is the ID, so renaming a job in Veeam doesn't duplicate it |
| Status | Session result (Success / Warning / Failed). Any state other than Stopped is **Running**. A session that stopped with no result (e.g. cancelled) is reported as **Warning** | | Status | Session result (Success / Warning / Failed). Any state other than Stopped is **Running**. A session that stopped with no result (e.g. cancelled) is reported as **Warning** |
| Started / finished | Session `CreationTime` / `EndTime` | | Started / finished | Session `CreationTime` / `EndTime` |
| Next scheduled | Job's next run. Sent as empty for disabled jobs and chained ("run after") jobs, so those are never flagged Overdue | | Next scheduled | Job's next run, from the live schedule (`GetScheduleOptions()`). Sent as empty, so never flagged Overdue, for disabled jobs, chained ("run after") jobs, and immediate-mode backup copy jobs, which run whenever their source does |
| Message | Session reason, each VM or object that didn't succeed, and warning/error lines from the session log | | Message | Session reason, each VM or object that didn't succeed, and warning/error lines from the session log |
| Size | `BackupStats.BackupSize` | | Size | `BackupStats.BackupSize` |
+35 -104
View File
@@ -37,42 +37,19 @@ param(
$ErrorActionPreference = 'Stop' $ErrorActionPreference = 'Stop'
$DataDir = Split-Path -Parent $ConfigPath # Shared collector code: installed next to this script; in the repo, in ..\common
$LogPath = Join-Path $DataDir 'veeam-collector.log' $sharedModule = @(
$StatePath = Join-Path $DataDir 'veeam-collector.state.json' (Join-Path $PSScriptRoot 'BackupDashboard.Collector.psm1'),
$MaxLogBytes = 5MB (Join-Path (Join-Path (Join-Path $PSScriptRoot '..') 'common') 'BackupDashboard.Collector.psm1')
$Invariant = [Globalization.CultureInfo]::InvariantCulture ) | Where-Object { Test-Path $_ } | Select-Object -First 1
if (-not $sharedModule) {
Write-Host "BackupDashboard.Collector.psm1 not found next to $PSCommandPath; re-run the installer."
exit 2
}
Import-Module $sharedModule -Force
# ---------- Helpers ---------- # ---------- Helpers ----------
function Write-Log {
param([string]$Message, [string]$Level = 'INFO')
$line = '{0} [{1}] {2}' -f (Get-Date).ToString('yyyy-MM-dd HH:mm:ss'), $Level, $Message
Write-Host $line
if ($DryRun) { return }
if ((Test-Path $LogPath) -and (Get-Item $LogPath).Length -gt $MaxLogBytes) {
Move-Item $LogPath "$LogPath.1" -Force
}
Add-Content -Path $LogPath -Value $line -Encoding UTF8
}
function Get-ErrorText($ErrorRecord) {
# Windows PowerShell puts the HTTP response body (e.g. the API's 422 explanation) here
if ($ErrorRecord.ErrorDetails -and $ErrorRecord.ErrorDetails.Message) {
return "$($ErrorRecord.Exception.Message) $($ErrorRecord.ErrorDetails.Message)"
}
return $ErrorRecord.Exception.Message
}
# The API needs an explicit UTC offset. Veeam returns local times, often with Kind=Unspecified,
# which DateTimeOffset treats as local, exactly what we want.
function ConvertTo-IsoTime($Value) {
if ($null -eq $Value) { return $null }
$dt = [datetime]$Value
if ($dt.Year -le 1900) { return $null } # Veeam's "not set" value (e.g. EndTime while running)
return ([DateTimeOffset]$dt).ToString("yyyy-MM-dd'T'HH:mm:sszzz", $Invariant)
}
function Import-Veeam { function Import-Veeam {
if (Get-Command Get-VBRJob -ErrorAction SilentlyContinue) { return } # already loaded if (Get-Command Get-VBRJob -ErrorAction SilentlyContinue) { return } # already loaded
if (Get-Module -ListAvailable -Name Veeam.Backup.PowerShell) { if (Get-Module -ListAvailable -Name Veeam.Backup.PowerShell) {
@@ -85,14 +62,25 @@ function Import-Veeam {
function Get-NextRun($Job) { function Get-NextRun($Job) {
try { try {
if (-not $Job.IsScheduleEnabled) { return $null } if (-not $Job.IsScheduleEnabled) { return $null }
$raw = $Job.ScheduleOptions.NextRun # Immediate-mode backup copy jobs run whenever their source job does. Their schedule still
# carries a NextRun, but it's a stale date (months old on VBR 13) that would show as Overdue.
if ($Job.IsContinuous) { return $null }
# The ScheduleOptions property is a stale snapshot on VBR 13 (NextRun empty, last run years
# old); GetScheduleOptions() reads the live schedule. Fall back for versions without it.
$options = $null
try { $options = $Job.GetScheduleOptions() } catch { }
if (-not $options) { $options = $Job.ScheduleOptions }
$raw = $options.NextRun
if (-not $raw) { return $null } # e.g. chained jobs that run "after" another job if (-not $raw) { return $null } # e.g. chained jobs that run "after" another job
if ($raw -is [datetime]) { return ConvertTo-IsoTime $raw } if ($raw -is [datetime]) { return ConvertTo-IsoTime $raw }
# Older versions return a string formatted with the server's culture, so parse with it # A string formatted by VBR ("2026-10-09 12:15:00 AM" on VBR 13, en-CA). Try the server's
$parsed = [datetime]::MinValue # culture first, then the invariant one
if ([datetime]::TryParse([string]$raw, [Globalization.CultureInfo]::CurrentCulture, foreach ($culture in [Globalization.CultureInfo]::CurrentCulture, [Globalization.CultureInfo]::InvariantCulture) {
[Globalization.DateTimeStyles]::None, [ref]$parsed)) { $parsed = [datetime]::MinValue
return ConvertTo-IsoTime $parsed if ([datetime]::TryParse([string]$raw, $culture, [Globalization.DateTimeStyles]::None, [ref]$parsed)) {
return ConvertTo-IsoTime $parsed
}
} }
Write-Log "Couldn't parse next run '$raw' for job '$($Job.Name)'" 'WARN' Write-Log "Couldn't parse next run '$raw' for job '$($Job.Name)'" 'WARN'
} catch { } catch {
@@ -206,51 +194,15 @@ function ConvertTo-Run($Session, $Job) {
return $run return $run
} }
function Send-Report($Body) {
$json = $Body | ConvertTo-Json -Depth 6 -Compress
if ($DryRun) {
Write-Host $json
return
}
# Send bytes: Windows PowerShell would otherwise encode a string body as Latin-1,
# mangling non-ASCII job names (e.g. en dashes).
$params = @{
Uri = "$($Config.ApiUrl.TrimEnd('/'))/api/v1/ingest"
Method = 'Post'
Headers = @{ Authorization = "Bearer $($Config.ApiKey)" }
ContentType = 'application/json; charset=utf-8'
Body = [Text.Encoding]::UTF8.GetBytes($json)
TimeoutSec = 30
}
Invoke-RestMethod @params | Out-Null
}
# ---------- Main ---------- # ---------- Main ----------
try { try {
$Config = Get-Content -Path $ConfigPath -Raw | ConvertFrom-Json $Config = Initialize-Collector -Name 'veeam' -ConfigPath $ConfigPath -LookbackHours $LookbackHours -DryRun:$DryRun
foreach ($key in 'ApiUrl', 'ApiKey', 'Tenant') {
if (-not $Config.$key) { throw "Config $ConfigPath is missing '$key'" }
}
$SourceSystem = if ($Config.SourceSystem) { [string]$Config.SourceSystem } else { 'Veeam B&R' } $SourceSystem = if ($Config.SourceSystem) { [string]$Config.SourceSystem } else { 'Veeam B&R' }
if ($LookbackHours -le 0) { $LookbackHours = $Config.LookbackHours
$LookbackHours = if ($Config.LookbackHours) { [int]$Config.LookbackHours } else { 48 }
}
# Windows PowerShell 5.1 may default to TLS 1.0
[Net.ServicePointManager]::SecurityProtocol = [Net.ServicePointManager]::SecurityProtocol -bor [Net.SecurityProtocolType]::Tls12
Import-Veeam Import-Veeam
# Session ID -> start time (Unix seconds), for sessions already delivered in their final state.
# Numbers rather than ISO strings: PowerShell 7's ConvertFrom-Json turns date strings into
# DateTimes and loses the offset.
$sent = @{}
if (Test-Path $StatePath) {
$saved = Get-Content -Path $StatePath -Raw | ConvertFrom-Json
if ($saved) { foreach ($p in $saved.PSObject.Properties) { $sent[$p.Name] = [long]$p.Value } }
}
$since = (Get-Date).AddHours(-$LookbackHours) $since = (Get-Date).AddHours(-$LookbackHours)
$jobs = @(Get-VBRJob -WarningAction SilentlyContinue) $jobs = @(Get-VBRJob -WarningAction SilentlyContinue)
@@ -277,7 +229,6 @@ try {
} }
$failures = 0 $failures = 0
$runsSent = 0
foreach ($job in $jobs) { foreach ($job in $jobs) {
$jobInfo = [ordered]@{ $jobInfo = [ordered]@{
external_id = $job.Id.ToString() external_id = $job.Id.ToString()
@@ -288,48 +239,28 @@ foreach ($job in $jobs) {
$pending = @() $pending = @()
if ($sessionsByJob.ContainsKey($jobInfo.external_id)) { if ($sessionsByJob.ContainsKey($jobInfo.external_id)) {
$pending = @($sessionsByJob[$jobInfo.external_id] | $pending = @($sessionsByJob[$jobInfo.external_id] |
Where-Object { -not $sent.ContainsKey($_.Id.ToString()) } | Where-Object { -not (Test-RunSent ($_.Id.ToString())) } |
Sort-Object CreationTime) Sort-Object CreationTime)
} }
try { try {
if ($pending.Count -eq 0) { if ($pending.Count -eq 0) {
# Keep next_run_at fresh even when nothing new ran # Keep next_run_at fresh even when nothing new ran
Send-Report ([ordered]@{ tenant = $Config.Tenant; source_system = $SourceSystem; job = $jobInfo }) Send-JobReport -Tenant $Config.Tenant -SourceSystem $SourceSystem -Job $jobInfo
} }
foreach ($session in $pending) { foreach ($session in $pending) {
$run = ConvertTo-Run $session $job $run = ConvertTo-Run $session $job
Send-Report ([ordered]@{ tenant = $Config.Tenant; source_system = $SourceSystem; job = $jobInfo; run = $run }) Send-JobReport -Tenant $Config.Tenant -SourceSystem $SourceSystem -Job $jobInfo -Run $run -SentKey $run.external_id -StartedAt ([datetime]$session.CreationTime)
$runsSent++
if ($run.status -ne 'running') {
$sent[$run.external_id] = ([DateTimeOffset][datetime]$session.CreationTime).ToUnixTimeSeconds()
}
} }
} catch { } catch {
$failures++ $failures++
Write-Log "Failed to report job '$($job.Name)': $(Get-ErrorText $_)" 'ERROR' Write-Log "Failed to report job '$($job.Name)': $(Get-ErrorText $_)" 'ERROR'
$httpStatus = 0 if (Test-AuthRejected $_) {
try { $httpStatus = [int]$_.Exception.Response.StatusCode } catch { }
if ($httpStatus -eq 401) {
Write-Log 'API key rejected; check ApiKey in the config. Stopping.' 'ERROR' Write-Log 'API key rejected; check ApiKey in the config. Stopping.' 'ERROR'
break break
} }
} }
} }
if (-not $DryRun) { $exitCode = Complete-Collector -JobCount $jobs.Count -Failures $failures
# Forget sessions that have aged out of the lookback window (plus a day of slack) if ($exitCode) { exit $exitCode }
$cutoff = [DateTimeOffset]::Now.AddHours(-$LookbackHours - 24).ToUnixTimeSeconds()
$keep = [ordered]@{}
foreach ($id in $sent.Keys) {
if ($sent[$id] -ge $cutoff) { $keep[$id] = $sent[$id] }
}
try {
$keep | ConvertTo-Json -Compress | Set-Content -Path $StatePath -Encoding UTF8
} catch {
Write-Log "Couldn't save state file: $(Get-ErrorText $_)" 'WARN'
}
}
Write-Log ("Reported {0} jobs, {1} runs, {2} failures" -f $jobs.Count, $runsSent, $failures)
if ($failures) { exit 1 }
+17 -7
View File
@@ -12,12 +12,22 @@ function New-StableGuid([string]$Seed) {
} }
$notSet = Get-Date '1900-01-01' $notSet = Get-Date '1900-01-01'
function New-MockJob($Name, $Type, $NextRun, [bool]$Enabled = $true) { # As on VBR 13: the ScheduleOptions property is a stale snapshot with an empty NextRun, and the live
[pscustomobject]@{ # schedule (NextRun as a "2026-10-09 12:15:00 AM" string) comes from GetScheduleOptions().
# -Continuous: an immediate-mode copy job, whose live NextRun is a stale date in the past.
function New-MockJob($Name, $Type, $NextRun, [bool]$Enabled = $true, [switch]$Continuous) {
$job = [pscustomobject]@{
Id = New-StableGuid $Name; Name = $Name; JobType = $Type Id = New-StableGuid $Name; Name = $Name; JobType = $Type
IsScheduleEnabled = $Enabled IsScheduleEnabled = $Enabled
ScheduleOptions = [pscustomobject]@{ NextRun = $NextRun } IsContinuous = [bool]$Continuous
ScheduleOptions = [pscustomobject]@{ NextRun = '' }
LiveNextRun = $NextRun
} }
$job | Add-Member ScriptMethod GetScheduleOptions { [pscustomobject]@{ NextRun = $this.LiveNextRun } }
$job
}
function Format-VbrTime([datetime]$Value) {
$Value.ToString('yyyy-MM-dd h:mm:ss tt', [Globalization.CultureInfo]::InvariantCulture)
} }
function New-MockSession($Job, $Start, $End, $State, $Result, $Reason = '', $Tasks = @(), $LogRecords = @(), $Size = 0) { function New-MockSession($Job, $Start, $End, $State, $Result, $Reason = '', $Tasks = @(), $LogRecords = @(), $Size = 0) {
@@ -40,13 +50,13 @@ function New-MockTask($Name, $Status, $Reason = '') {
} }
$global:MockJobs = @( $global:MockJobs = @(
(New-MockJob 'File Server – Nightly' 'Backup' $now.Date.AddDays(1).AddHours(22)), (New-MockJob 'File Server – Nightly' 'Backup' (Format-VbrTime $now.Date.AddDays(1).AddMinutes(15))),
# Older VBR versions return NextRun as a culture-formatted string # Some versions return NextRun as a DateTime
(New-MockJob 'SQL01 – Logs' 'Backup' $now.AddMinutes(40).ToString()), (New-MockJob 'SQL01 – Logs' 'Backup' $now.AddMinutes(40)),
(New-MockJob 'Offsite Copy' 'BackupSync' $null), (New-MockJob 'Offsite Copy' 'BackupSync' $null),
(New-MockJob 'Old Exchange' 'Backup' $now.AddDays(1) $false), (New-MockJob 'Old Exchange' 'Backup' $now.AddDays(1) $false),
# VBR 12+ backup copy job (immediate mode, no schedule): the "policy" job Get-VBRJob returns # VBR 12+ backup copy job (immediate mode, no schedule): the "policy" job Get-VBRJob returns
(New-MockJob 'Cloud Copy' 'SimpleBackupCopyPolicy' $null) (New-MockJob 'Cloud Copy' 'SimpleBackupCopyPolicy' (Format-VbrTime $now.AddMonths(-6)) -Continuous)
) )
$fs, $sql, $copy, $old, $cloud = $global:MockJobs $fs, $sql, $copy, $old, $cloud = $global:MockJobs
+14 -49
View File
@@ -29,9 +29,6 @@ param(
) )
$ErrorActionPreference = 'Stop' $ErrorActionPreference = 'Stop'
$TaskName = 'Backup Dashboard - WSB collector'
$DataDir = Join-Path $env:ProgramData 'BackupDashboard'
$ConfigPath = Join-Path $DataDir 'wsb-collector.json'
$WindowsPowerShell = Join-Path $env:SystemRoot 'System32\WindowsPowerShell\v1.0\powershell.exe' $WindowsPowerShell = Join-Path $env:SystemRoot 'System32\WindowsPowerShell\v1.0\powershell.exe'
# Check Windows Server Backup is here before changing anything # Check Windows Server Backup is here before changing anything
@@ -39,51 +36,19 @@ if (-not (Get-Module -ListAvailable -Name WindowsServerBackup)) {
throw 'The WindowsServerBackup PowerShell module is not available. Install the Windows Server Backup feature (Install-WindowsFeature Windows-Server-Backup).' throw 'The WindowsServerBackup PowerShell module is not available. Install the Windows Server Backup feature (Install-WindowsFeature Windows-Server-Backup).'
} }
if ($ApiUrl -notmatch '^https://') { # Shared installer code: in the repo it's in ..\common, so copy the whole collectors folder
Write-Warning 'ApiUrl is not HTTPS: the API key would be sent in clear text.' $sharedModule = @(
(Join-Path $PSScriptRoot 'BackupDashboard.Collector.psm1'),
(Join-Path (Join-Path (Join-Path $PSScriptRoot '..') 'common') 'BackupDashboard.Collector.psm1')
) | Where-Object { Test-Path $_ } | Select-Object -First 1
if (-not $sharedModule) {
throw "BackupDashboard.Collector.psm1 not found. Copy the whole collectors folder, including common."
} }
Import-Module $sharedModule -Force
# Read the key as a SecureString so it isn't echoed to the console or kept in shell history $settings = [ordered]@{ Tenant = $Tenant; SourceSystem = $SourceSystem; LookbackHours = $LookbackHours }
$secure = Read-Host -AsSecureString 'API key (from: python manage.py create-key <name>)' if ($JobName) { $settings.JobName = $JobName }
$bstr = [Runtime.InteropServices.Marshal]::SecureStringToBSTR($secure) Install-CollectorTask -Name 'wsb' -ScriptPath (Join-Path $PSScriptRoot 'Send-WsbBackupStatus.ps1') `
try { $apiKey = [Runtime.InteropServices.Marshal]::PtrToStringBSTR($bstr) } -TaskName 'Backup Dashboard - WSB collector' `
finally { [Runtime.InteropServices.Marshal]::ZeroFreeBSTR($bstr) } -TaskDescription 'Sends Windows Server Backup results to the Backup Dashboard' `
if (-not $apiKey.StartsWith('bdk_')) { throw "That doesn't look like a dashboard API key (expected 'bdk_...')." } -Shell $WindowsPowerShell -ApiUrl $ApiUrl -IntervalMinutes $IntervalMinutes -InstallDir $InstallDir -Settings $settings
# Config folder: SYSTEM + Administrators only (SIDs so this works on non-English Windows)
New-Item -ItemType Directory -Path $DataDir -Force | Out-Null
& icacls.exe $DataDir /inheritance:r /grant:r '*S-1-5-18:(OI)(CI)F' '*S-1-5-32-544:(OI)(CI)F' | Out-Null
if ($LASTEXITCODE) { throw "icacls failed on $DataDir" }
$config = [ordered]@{
ApiUrl = $ApiUrl.TrimEnd('/')
ApiKey = $apiKey
Tenant = $Tenant
SourceSystem = $SourceSystem
LookbackHours = $LookbackHours
}
if ($JobName) { $config.JobName = $JobName }
$config | ConvertTo-Json | Set-Content -Path $ConfigPath -Encoding UTF8
New-Item -ItemType Directory -Path $InstallDir -Force | Out-Null
$scriptPath = Join-Path $InstallDir 'Send-WsbBackupStatus.ps1'
Copy-Item -Path (Join-Path $PSScriptRoot 'Send-WsbBackupStatus.ps1') -Destination $scriptPath -Force
Unblock-File -Path $scriptPath
$action = New-ScheduledTaskAction -Execute $WindowsPowerShell `
-Argument "-NoProfile -NonInteractive -ExecutionPolicy Bypass -File `"$scriptPath`""
$trigger = New-ScheduledTaskTrigger -Once -At (Get-Date).AddMinutes(1) `
-RepetitionInterval (New-TimeSpan -Minutes $IntervalMinutes)
$principal = New-ScheduledTaskPrincipal -UserId 'S-1-5-18' -LogonType ServiceAccount -RunLevel Highest
$settings = New-ScheduledTaskSettingsSet -ExecutionTimeLimit (New-TimeSpan -Minutes 30) `
-MultipleInstances IgnoreNew -StartWhenAvailable
Register-ScheduledTask -TaskName $TaskName -Action $action -Trigger $trigger -Principal $principal `
-Settings $settings -Description 'Sends Windows Server Backup results to the Backup Dashboard' -Force | Out-Null
Write-Host "Installed. Config: $ConfigPath"
Write-Host "Task '$TaskName' runs every $IntervalMinutes minutes as SYSTEM, using Windows PowerShell."
Write-Host ''
Write-Host 'Next steps:'
Write-Host " 1. Preview what will be sent: & '$WindowsPowerShell' -File '$scriptPath' -DryRun"
Write-Host " 2. Backfill 30 days of history: & '$WindowsPowerShell' -File '$scriptPath' -LookbackHours 720"
Write-Host " 3. Check the log: Get-Content '$DataDir\wsb-collector.log' -Tail 20"
+4 -2
View File
@@ -19,7 +19,8 @@ Requires the Windows Server Backup feature with its PowerShell module
```bash ```bash
python manage.py create-key wsb-acme-dc01 python manage.py create-key wsb-acme-dc01
``` ```
2. Copy this folder to the server and run: 2. Copy the whole `collectors` folder to the server (the installer needs `common`
as well as this folder), then from `collectors\wsb` run:
```powershell ```powershell
.\Install-WsbCollector.ps1 -ApiUrl https://backups.example.com -Tenant "Acme Corp" .\Install-WsbCollector.ps1 -ApiUrl https://backups.example.com -Tenant "Acme Corp"
``` ```
@@ -35,7 +36,8 @@ Requires the Windows Server Backup feature with its PowerShell module
``` ```
It can be installed alongside the other collectors: it uses its own config, state, log and task. It can be installed alongside the other collectors: it uses its own config, state, log and task.
Re-run the installer to change settings or update the script. Re-run the installer to change settings or update the script. It also installs
`BackupDashboard.Collector.psm1`, the code shared by all collectors, next to the script.
## What it sends ## What it sends
+18 -111
View File
@@ -40,64 +40,23 @@ param(
$ErrorActionPreference = 'Stop' $ErrorActionPreference = 'Stop'
$DataDir = Split-Path -Parent $ConfigPath # Shared collector code: installed next to this script; in the repo, in ..\common
$LogPath = Join-Path $DataDir 'wsb-collector.log' $sharedModule = @(
$StatePath = Join-Path $DataDir 'wsb-collector.state.json' (Join-Path $PSScriptRoot 'BackupDashboard.Collector.psm1'),
$MaxLogBytes = 5MB (Join-Path (Join-Path (Join-Path $PSScriptRoot '..') 'common') 'BackupDashboard.Collector.psm1')
) | Where-Object { Test-Path $_ } | Select-Object -First 1
if (-not $sharedModule) {
Write-Host "BackupDashboard.Collector.psm1 not found next to $PSCommandPath; re-run the installer."
exit 2
}
Import-Module $sharedModule -Force
# Get-WBJob -Previous accepts at most 100. That covers a 30-day backfill at up to 3 backups a day; # Get-WBJob -Previous accepts at most 100. That covers a 30-day backfill at up to 3 backups a day;
# servers backing up more often only get their latest 100 runs backfilled. # servers backing up more often only get their latest 100 runs backfilled.
$MaxPreviousJobs = 100 $MaxPreviousJobs = 100
$Invariant = [Globalization.CultureInfo]::InvariantCulture
# ---------- Helpers ---------- # ---------- Helpers ----------
function Write-Log {
param([string]$Message, [string]$Level = 'INFO')
$line = '{0} [{1}] {2}' -f (Get-Date).ToString('yyyy-MM-dd HH:mm:ss'), $Level, $Message
Write-Host $line
if ($DryRun) { return }
if ((Test-Path $LogPath) -and (Get-Item $LogPath).Length -gt $MaxLogBytes) {
Move-Item $LogPath "$LogPath.1" -Force
}
Add-Content -Path $LogPath -Value $line -Encoding UTF8
}
function Get-ErrorText($ErrorRecord) {
# Windows PowerShell puts the HTTP response body (e.g. the API's 422 explanation) here
if ($ErrorRecord.ErrorDetails -and $ErrorRecord.ErrorDetails.Message) {
return "$($ErrorRecord.Exception.Message) $($ErrorRecord.ErrorDetails.Message)"
}
return $ErrorRecord.Exception.Message
}
# The API needs an explicit UTC offset. DateTimeOffset keeps a Local/Unspecified time as local and a
# Utc time as +00:00.
function ConvertTo-IsoTime($Value) {
if ($null -eq $Value) { return $null }
$dt = [datetime]$Value
if ($dt.Year -le 1900) { return $null } # "not set" (DateTime.MinValue and friends)
return ([DateTimeOffset]$dt).ToString("yyyy-MM-dd'T'HH:mm:sszzz", $Invariant)
}
# First property of $Object, out of $Names, that exists and isn't empty. The WSB object model isn't
# fully documented, so reads go through this rather than assuming one spelling.
function Get-FirstValue($Object, [string[]]$Names) {
if ($null -eq $Object) { return $null }
foreach ($name in $Names) {
try {
$prop = $Object.PSObject.Properties[$name]
if (-not $prop) { continue }
$value = $prop.Value
# Check emptiness by type: casting to [string] would make a one-item list of objects ''
if ($null -eq $value) { continue }
if ($value -is [string] -and $value -eq '') { continue }
if ($value -is [Collections.ICollection] -and $value.Count -eq 0) { continue }
return $value
} catch { }
}
return $null
}
function Import-Wsb { function Import-Wsb {
if (Get-Command Get-WBJob -ErrorAction SilentlyContinue) { return } # already loaded if (Get-Command Get-WBJob -ErrorAction SilentlyContinue) { return } # already loaded
# Present only when the Windows Server Backup feature (with its command-line tools) is installed # Present only when the Windows Server Backup feature (with its command-line tools) is installed
@@ -288,49 +247,16 @@ function Get-NextRun {
} }
} }
function Send-Report($Body) {
$json = $Body | ConvertTo-Json -Depth 6 -Compress
if ($DryRun) {
Write-Host $json
return
}
# Send bytes: Windows PowerShell would otherwise encode a string body as Latin-1
$params = @{
Uri = "$($Config.ApiUrl.TrimEnd('/'))/api/v1/ingest"
Method = 'Post'
Headers = @{ Authorization = "Bearer $($Config.ApiKey)" }
ContentType = 'application/json; charset=utf-8'
Body = [Text.Encoding]::UTF8.GetBytes($json)
TimeoutSec = 30
}
Invoke-RestMethod @params | Out-Null
}
# ---------- Main ---------- # ---------- Main ----------
try { try {
$Config = Get-Content -Path $ConfigPath -Raw | ConvertFrom-Json $Config = Initialize-Collector -Name 'wsb' -ConfigPath $ConfigPath -LookbackHours $LookbackHours -DryRun:$DryRun
foreach ($key in 'ApiUrl', 'ApiKey', 'Tenant') {
if (-not $Config.$key) { throw "Config $ConfigPath is missing '$key'" }
}
$SourceSystem = if ($Config.SourceSystem) { [string]$Config.SourceSystem } else { 'Windows Server Backup' } $SourceSystem = if ($Config.SourceSystem) { [string]$Config.SourceSystem } else { 'Windows Server Backup' }
$JobName = if ($Config.JobName) { [string]$Config.JobName } else { "$env:COMPUTERNAME - Windows Server Backup" } $JobName = if ($Config.JobName) { [string]$Config.JobName } else { "$env:COMPUTERNAME - Windows Server Backup" }
if ($LookbackHours -le 0) { $LookbackHours = $Config.LookbackHours
$LookbackHours = if ($Config.LookbackHours) { [int]$Config.LookbackHours } else { 48 }
}
# Windows PowerShell 5.1 may default to TLS 1.0
[Net.ServicePointManager]::SecurityProtocol = [Net.ServicePointManager]::SecurityProtocol -bor [Net.SecurityProtocolType]::Tls12
Import-Wsb Import-Wsb
# Run start time (ISO, as sent) -> Unix seconds, for runs already delivered in their final state
$sent = @{}
if (Test-Path $StatePath) {
$saved = Get-Content -Path $StatePath -Raw | ConvertFrom-Json
if ($saved) { foreach ($p in $saved.PSObject.Properties) { $sent[$p.Name] = [long]$p.Value } }
}
$since = (Get-Date).AddHours(-$LookbackHours) $since = (Get-Date).AddHours(-$LookbackHours)
# Get-WBJob with no arguments returns the job in progress (if any); -Previous lists finished ones # Get-WBJob with no arguments returns the job in progress (if any); -Previous lists finished ones
$jobs = @(@(Get-WBJob) + @(Get-WBJob -Previous $MaxPreviousJobs) | Where-Object { $jobs = @(@(Get-WBJob) + @(Get-WBJob -Previous $MaxPreviousJobs) | Where-Object {
@@ -342,7 +268,7 @@ try {
if ($start -lt $since) { continue } if ($start -lt $since) { continue }
$byStart[(ConvertTo-IsoTime $start)] = $job # a job in both lists: the finished entry (later) wins $byStart[(ConvertTo-IsoTime $start)] = $job # a job in both lists: the finished entry (later) wins
} }
$pending = @($byStart.Keys | Sort-Object | Where-Object { -not $sent.ContainsKey($_) } | $pending = @($byStart.Keys | Sort-Object | Where-Object { -not (Test-RunSent ($_)) } |
ForEach-Object { $byStart[$_] }) ForEach-Object { $byStart[$_] })
$jobInfo = [ordered]@{ $jobInfo = [ordered]@{
@@ -358,38 +284,19 @@ try {
} }
$failures = 0 $failures = 0
$runsSent = 0
try { try {
if ($pending.Count -eq 0) { if ($pending.Count -eq 0) {
# Keep next_run_at fresh even when nothing new ran # Keep next_run_at fresh even when nothing new ran
Send-Report ([ordered]@{ tenant = $Config.Tenant; source_system = $SourceSystem; job = $jobInfo }) Send-JobReport -Tenant $Config.Tenant -SourceSystem $SourceSystem -Job $jobInfo
} }
foreach ($job in $pending) { foreach ($job in $pending) {
$run = ConvertTo-Run $job $run = ConvertTo-Run $job
Send-Report ([ordered]@{ tenant = $Config.Tenant; source_system = $SourceSystem; job = $jobInfo; run = $run }) Send-JobReport -Tenant $Config.Tenant -SourceSystem $SourceSystem -Job $jobInfo -Run $run -SentKey $run.started_at -StartedAt ((Get-JobTime $job 'StartTime'))
$runsSent++
if ($run.status -ne 'running') {
$sent[$run.started_at] = ([DateTimeOffset](Get-JobTime $job 'StartTime')).ToUnixTimeSeconds()
}
} }
} catch { } catch {
$failures++ $failures++
Write-Log "Failed to report: $(Get-ErrorText $_)" 'ERROR' Write-Log "Failed to report: $(Get-ErrorText $_)" 'ERROR'
} }
if (-not $DryRun) { $exitCode = Complete-Collector -Failures $failures
# Forget runs that have aged out of the lookback window (plus a day of slack) if ($exitCode) { exit $exitCode }
$cutoff = [DateTimeOffset]::Now.AddHours(-$LookbackHours - 24).ToUnixTimeSeconds()
$keep = [ordered]@{}
foreach ($key in $sent.Keys) {
if ($sent[$key] -ge $cutoff) { $keep[$key] = $sent[$key] }
}
try {
$keep | ConvertTo-Json -Compress | Set-Content -Path $StatePath -Encoding UTF8
} catch {
Write-Log "Couldn't save state file: $(Get-ErrorText $_)" 'WARN'
}
}
Write-Log ("Reported {0} runs, {1} failures" -f $runsSent, $failures)
if ($failures) { exit 1 }
+54 -8
View File
@@ -1,7 +1,7 @@
server { server {
listen 80; listen 80;
server_name backups.example.com; server_name backups.example.com;
# Basic auth sends the password with every request — never serve this over plain HTTP # Session cookies and API keys must never cross plain HTTP
return 301 https://$host$request_uri; return 301 https://$host$request_uri;
} }
@@ -23,26 +23,72 @@ server {
add_header Referrer-Policy "no-referrer" always; add_header Referrer-Policy "no-referrer" always;
add_header Strict-Transport-Security "max-age=31536000" always; add_header Strict-Transport-Security "max-age=31536000" always;
# Dashboard users. Create with: htpasswd -B -c /etc/nginx/backup-dashboard.htpasswd <user> # ---------- Microsoft 365 sign-in ----------
# To move to Microsoft 365 sign-in later, replace these two lines with an auth_request to # oauth2-proxy (deploy/oauth2-proxy.cfg) runs on 127.0.0.1:4180 and does the sign-in with
# oauth2-proxy (provider=entra-id); the app itself doesn't need to change. # Entra ID. Every request below asks it "is this browser signed in?" via auth_request.
auth_basic "Backup Monitor";
auth_basic_user_file /etc/nginx/backup-dashboard.htpasswd; # Its own pages: /oauth2/start (go to Microsoft), /oauth2/callback, /oauth2/sign_out
location /oauth2/ {
proxy_pass http://127.0.0.1:4180;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
# Where to return after signing in: the page that was asked for
proxy_set_header X-Auth-Request-Redirect $request_uri;
# Entra ID session cookies are large
proxy_buffer_size 16k;
proxy_buffers 8 16k;
}
# The sign-in check itself, only reachable as an auth_request subrequest
location = /oauth2/auth {
internal;
proxy_pass http://127.0.0.1:4180;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Uri $request_uri;
proxy_set_header Content-Length "";
proxy_pass_request_body off;
proxy_buffer_size 16k;
proxy_buffers 8 16k;
}
# ---------- Dashboard ----------
location / { location / {
auth_request /oauth2/auth;
# Not signed in: straight to Microsoft's sign-in, then back to this page
error_page 401 = /oauth2/start;
# Make browsers check for a newer page/script on every load (a cheap 304 when unchanged),
# so an update shows up at once. (expires, not add_header: an add_header here would drop
# the security headers above.)
expires -1;
try_files $uri $uri/ /index.html; try_files $uri $uri/ /index.html;
} }
location /api/ { location /api/ {
auth_request /oauth2/auth;
# No redirect here: API calls from the page get a plain 401, and the page asks the user to
# reload, which goes through sign-in above.
# Who's signed in, for the dashboard's "signed in as" (proxy_set_header replaces any value
# the browser sent, so these can't be spoofed)
auth_request_set $auth_email $upstream_http_x_auth_request_email;
auth_request_set $auth_user $upstream_http_x_auth_request_user;
auth_request_set $auth_name $upstream_http_x_auth_request_preferred_username;
proxy_set_header X-Forwarded-Email $auth_email;
proxy_set_header X-Forwarded-User $auth_user;
proxy_set_header X-Forwarded-Preferred-Username $auth_name;
proxy_pass http://127.0.0.1:8000; proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host; proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme; proxy_set_header X-Forwarded-Proto $scheme;
} }
# Collectors authenticate with their own API key (Authorization: Bearer …), checked by the app # Collectors authenticate with their own API key (Authorization: Bearer …), checked by the app,
# so no Microsoft sign-in here
location = /api/v1/ingest { location = /api/v1/ingest {
auth_basic off;
client_max_body_size 256k; client_max_body_size 256k;
proxy_pass http://127.0.0.1:8000; proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host; proxy_set_header Host $host;
+46
View File
@@ -0,0 +1,46 @@
# oauth2-proxy config for Backup Dashboard's Microsoft 365 (Entra ID) sign-in.
# Install as /etc/oauth2-proxy/backup-dashboard.cfg, readable only by root and the oauth2-proxy
# user (it holds the cookie secret). Needs oauth2-proxy 7.8 or newer for the entra-id provider.
# Replace the four <...> values; see the README's "Microsoft 365 sign-in" section.
## Where it listens: only nginx talks to it
http_address = "127.0.0.1:4180"
reverse_proxy = true
# Only nginx (on this machine) may supply X-Forwarded-* headers
trusted_proxy_ips = ["127.0.0.1/32", "::1/128"]
## Entra ID app registration (single tenant)
provider = "entra-id"
oidc_issuer_url = "https://login.microsoftonline.com/<TENANT_ID>/v2.0"
client_id = "<APPLICATION_CLIENT_ID>"
# The client secret's *value*, in a file of its own, root:oauth2-proxy, mode 640
client_secret_file = "/etc/oauth2-proxy/client-secret"
redirect_url = "https://<DASHBOARD_HOST>/oauth2/callback"
## Who may sign in
# Anyone in the tenant that Entra lets through. Limit it in Entra itself: Enterprise
# applications -> this app -> Properties -> "Assignment required" = Yes, then assign users or a
# group. Entra then refuses everyone else before they ever reach the dashboard.
email_domains = ["*"]
# The user's sign-in name (UPN), which every work account has. The "email" claim is only present
# when the account has a mailbox address.
oidc_email_claim = "preferred_username"
## Session cookie
# Generate with: openssl rand -base64 32 | tr -- '+/' '-_'
cookie_secret = "<COOKIE_SECRET>"
cookie_name = "_backup_dashboard"
cookie_secure = true
cookie_samesite = "lax"
# Sign in again (usually silently, via the Microsoft session) after this long
cookie_expire = "12h"
## Behaviour with nginx's auth_request
# Auth-only: nginx serves the dashboard; oauth2-proxy just answers "signed in or not"
upstreams = ["static://202"]
# Put the user in X-Auth-Request-* headers on /oauth2/auth responses, for nginx to pass on
set_xauthrequest = true
# Go straight to Microsoft instead of showing a "Sign in with..." page first
skip_provider_button = true
# Allow "Sign out" to continue to Microsoft's sign-out page
whitelist_domains = ["login.microsoftonline.com"]
+18
View File
@@ -0,0 +1,18 @@
[Unit]
Description=oauth2-proxy (Microsoft 365 sign-in for Backup Dashboard)
After=network-online.target
Wants=network-online.target
[Service]
User=oauth2-proxy
Group=oauth2-proxy
ExecStart=/usr/local/bin/oauth2-proxy --config=/etc/oauth2-proxy/backup-dashboard.cfg
Restart=on-failure
NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
PrivateTmp=true
[Install]
WantedBy=multi-user.target
+6
View File
@@ -71,6 +71,12 @@ body {
} }
.topbar__title { margin: 0; font-size: 1.25rem; } .topbar__title { margin: 0; font-size: 1.25rem; }
.topbar__meta { display: flex; align-items: center; gap: 0.75rem; font-size: 0.875rem; } .topbar__meta { display: flex; align-items: center; gap: 0.75rem; font-size: 0.875rem; }
/* Only when shown: display:flex on the bare class would override the hidden attribute */
.topbar__user:not([hidden]) {
display: flex; align-items: center; gap: 0.75rem;
padding-left: 0.75rem; border-left: 1px solid var(--border);
}
a.btn { text-decoration: none; }
.container { .container {
max-width: 1400px; max-width: 1400px;
+5
View File
@@ -14,6 +14,11 @@
<div class="topbar__meta"> <div class="topbar__meta">
<span id="last-updated" class="muted" aria-live="polite">Loading…</span> <span id="last-updated" class="muted" aria-live="polite">Loading…</span>
<button id="refresh-btn" class="btn" type="button">Refresh</button> <button id="refresh-btn" class="btn" type="button">Refresh</button>
<!-- Filled in from /api/v1/me when signed in with Microsoft 365 -->
<span id="signed-in" class="topbar__user" hidden>
<span id="signed-in-email" class="muted"></span>
<a id="sign-out" class="btn" href="#">Sign out</a>
</span>
</div> </div>
</header> </header>
+4
View File
@@ -49,6 +49,10 @@ const httpApi = {
getRun(id) { getRun(id) {
return request(`/runs/${encodeURIComponent(id)}`); return request(`/runs/${encodeURIComponent(id)}`);
}, },
/** GET /me → { email, name } of the Microsoft 365 user, both null without sign-in */
getMe() {
return request('/me');
},
}; };
// Mock is loaded lazily so it never ships in the production request path. // Mock is loaded lazily so it never ships in the production request path.
+23 -4
View File
@@ -56,6 +56,9 @@ const dom = {
empty: $('#empty-state'), empty: $('#empty-state'),
search: $('#search'), search: $('#search'),
refreshBtn: $('#refresh-btn'), refreshBtn: $('#refresh-btn'),
signedIn: $('#signed-in'),
signedInEmail: $('#signed-in-email'),
signOut: $('#sign-out'),
lastUpdated: $('#last-updated'), lastUpdated: $('#last-updated'),
errorBanner: $('#error-banner'), errorBanner: $('#error-banner'),
dialog: $('#detail-dialog'), dialog: $('#detail-dialog'),
@@ -95,8 +98,8 @@ function statusBadge(status, { asButton = false, opensDialog = false, dataset =
const runningTitle = (since) => (since ? `Running since ${formatDateTime(since)}` : null); const runningTitle = (since) => (since ? `Running since ${formatDateTime(since)}` : null);
function timeCell(date, extra = null) { function timeCell(date, { extra = null, emptyLabel = 'Never' } = {}) {
if (!date) return el('td', {}, el('span', { class: 'muted' }, 'Never')); if (!date) return el('td', {}, el('span', { class: 'muted' }, emptyLabel));
return el('td', {}, return el('td', {},
el('time', { class: 'time', datetime: date.toISOString(), title: date.toString() }, el('time', { class: 'time', datetime: date.toISOString(), title: date.toString() },
el('span', {}, formatDateTime(date), extra), el('span', {}, formatDateTime(date), extra),
@@ -112,7 +115,8 @@ function renderRow(job) {
asButton: true, opensDialog: true, dataset: { jobId: job.id }, title: runningTitle(job.runningSince), asButton: true, opensDialog: true, dataset: { jobId: job.id }, title: runningTitle(job.runningSince),
})), })),
timeCell(job.lastRun), timeCell(job.lastRun),
timeCell(job.nextRun, overdueTag)); // No next run: unscheduled, or e.g. a Veeam copy job in immediate mode
timeCell(job.nextRun, { extra: overdueTag, emptyLabel: 'Not scheduled' }));
} }
const collator = new Intl.Collator(undefined, { sensitivity: 'base', numeric: true }); const collator = new Intl.Collator(undefined, { sensitivity: 'base', numeric: true });
@@ -304,7 +308,7 @@ function renderSummarySection() {
// While a job is running, its "last run" is the previous one — say how that went // While a job is running, its "last run" is the previous one — say how that went
status === 'running' && lastStatus ? detailRow('Previous result', STATUS[lastStatus].label) : [], status === 'running' && lastStatus ? detailRow('Previous result', STATUS[lastStatus].label) : [],
detailRow('Last finished', formatDateTime(detail ? parseDate(detail.last_run_at) : job.lastRun)), detailRow('Last finished', formatDateTime(detail ? parseDate(detail.last_run_at) : job.lastRun)),
detailRow('Next scheduled', formatDateTime(detail ? parseDate(detail.next_run_at) : job.nextRun)), detailRow('Next scheduled', formatDateTime(detail ? parseDate(detail.next_run_at) : job.nextRun, 'Not scheduled')),
detailRow('Duration', formatDuration(detail?.duration_seconds)), detailRow('Duration', formatDuration(detail?.duration_seconds)),
detailRow('Size', formatBytes(detail?.size_bytes)), detailRow('Size', formatBytes(detail?.size_bytes)),
detailRow('Source', detail?.source_system))); detailRow('Source', detail?.source_system)));
@@ -523,5 +527,20 @@ dom.search.addEventListener('input', () => {
dom.refreshBtn.addEventListener('click', () => refresh().then(schedulePoll)); dom.refreshBtn.addEventListener('click', () => refresh().then(schedulePoll));
// ---------- Signed-in user ----------
async function showSignedInUser() {
try {
const me = await api.getMe();
const who = me?.email || me?.name;
if (!who) return; // no Microsoft 365 sign-in in front of the dashboard (dev, mock)
dom.signedInEmail.textContent = who;
dom.signOut.href = CONFIG.signOutUrl;
dom.signedIn.hidden = false;
} catch (err) {
console.error(err); // not worth a banner: the jobs request will surface real problems
}
}
// ---------- Boot ---------- // ---------- Boot ----------
showSignedInUser();
refresh().then(schedulePoll); refresh().then(schedulePoll);
+4
View File
@@ -12,4 +12,8 @@ export const CONFIG = Object.freeze({
historyDays: 30, historyDays: 30,
// ?mock=1 → use in-browser sample data, no backend needed // ?mock=1 → use in-browser sample data, no backend needed
useMock: params.has('mock'), useMock: params.has('mock'),
// oauth2-proxy clears the dashboard session, then hands over to Microsoft's sign-out page.
// Signing out of the dashboard alone would be undone at once: the browser's Microsoft session
// would sign it straight back in.
signOutUrl: '/oauth2/sign_out?rd=' + encodeURIComponent('https://login.microsoftonline.com/common/oauth2/v2.0/logout'),
}); });
+1 -1
View File
@@ -13,7 +13,7 @@ export function parseDate(value) {
return Number.isNaN(d.getTime()) ? null : d; return Number.isNaN(d.getTime()) ? null : d;
} }
export const formatDateTime = (d) => (d ? dateTimeFmt.format(d) : '—'); export const formatDateTime = (d, emptyLabel = '—') => (d ? dateTimeFmt.format(d) : emptyLabel);
export const formatTime = (d) => timeFmt.format(d); export const formatTime = (d) => timeFmt.format(d);
export function formatRelative(d, now = Date.now()) { export function formatRelative(d, now = Date.now()) {
+3
View File
@@ -126,4 +126,7 @@ export const mockApi = {
} }
throw new Error('Run not found'); throw new Error('Run not found');
}, },
async getMe() {
return { email: null, name: null }; // no sign-in in mock mode
},
}; };