From abb684ff0d82c57fecbd2961f1c8881c9746a578 Mon Sep 17 00:00:00 2001 From: kyle Date: Thu, 8 Oct 2026 14:24:51 -0600 Subject: [PATCH] Document updating the dashboard server The README only covered the first install, so the admin page update stalled at a step that wasn't written down: nginx serves its own copy of public/ in /var/www, and without copying it a new page quietly falls back to index.html. "Updating the server" covers: backing up the database, pulling and listing what changed, Python packages, copying the web files, restarting, health checks, which deploy/ templates map to which installed files, and redeploying collectors when they change. A rollback section restores the database before starting the older API (which refuses a newer database), using sqlite3 .restore rather than cp because of the WAL file. Co-Authored-By: Claude Opus 5.5 --- README.md | 85 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 85 insertions(+) diff --git a/README.md b/README.md index 8f40471..72ac30d 100644 --- a/README.md +++ b/README.md @@ -173,3 +173,88 @@ Collector tests (PowerShell 7 + Pester; see [collectors/tests](collectors/tests/ Back up the SQLite file with `sqlite3 backup-dashboard.db ".backup copy.db"`. Don't use a plain `cp` while the service is running. + +### Updating the server + +After changes are merged into `main`, on the dashboard server: + +1. **Back up the database.** It takes seconds, and an update may upgrade the database's structure + when the API restarts, which only this backup can reverse. Needs the `sqlite3` package: + ```bash + sudo -u backupdash sqlite3 /var/lib/backup-dashboard/backup-dashboard.db ".backup /var/lib/backup-dashboard/before-update.db" + ``` +2. **Pull, and see what changed** (the list tells you whether steps 7 and 8 apply): + ```bash + sudo git -C /opt/backup-dashboard pull + ``` + ```bash + git -C /opt/backup-dashboard diff --stat ORIG_HEAD HEAD + ``` +3. **Python packages.** Does nothing unless `backend/requirements.txt` changed: + ```bash + sudo /opt/backup-dashboard/venv/bin/pip install -q -r /opt/backup-dashboard/backend/requirements.txt + ``` +4. **Web files.** nginx serves its own copy in `/var/www`, so a pull alone doesn't change what + browsers get. If this step is skipped, new pages quietly show the dashboard instead (nginx + falls back to `index.html` for files it doesn't have): + ```bash + sudo rsync -a /opt/backup-dashboard/public/ /var/www/backup-dashboard/public/ + ``` + Browsers pick up the new files on their next page load. +5. **Restart the API**, which loads the new code and settings and upgrades the database if needed: + ```bash + sudo systemctl restart backup-dashboard-api + ``` +6. **Check it's healthy.** The first should print `{"status":"ok"}`; the log shouldn't show errors: + ```bash + curl -s http://127.0.0.1:8000/api/v1/health + ``` + ```bash + journalctl -u backup-dashboard-api -n 20 --no-pager + ``` +7. **If files in `deploy/` changed:** those are templates whose installed copies live elsewhere, + so a pull doesn't update them. Compare each changed one with its installed copy (e.g. + `diff /opt/backup-dashboard/deploy/oauth2-proxy.service /etc/systemd/system/oauth2-proxy.service`) + and bring the changes across: + + | Changed in the repo | Installed copy | After updating it | + |---|---|---| + | `deploy/backup-dashboard.env` | `/etc/backup-dashboard/backup-dashboard.env` | Add any new settings, keeping your values; restart the API | + | `deploy/backup-dashboard-api.service` | `/etc/systemd/system/backup-dashboard-api.service` | `sudo systemctl daemon-reload`, restart the API | + | `deploy/nginx.conf` | Your nginx site config | Keep your server name and certificate paths; `sudo nginx -t && sudo systemctl reload nginx` | + | `deploy/oauth2-proxy.cfg` | `/etc/oauth2-proxy/backup-dashboard.cfg` | Keep your IDs and secrets; `sudo systemctl restart oauth2-proxy` | + | `deploy/oauth2-proxy.service` | `/etc/systemd/system/oauth2-proxy.service` | `sudo systemctl daemon-reload && sudo systemctl restart oauth2-proxy` | + +8. **If files in `collectors/` changed:** the backup servers keep running their installed copy. + On each affected server, copy over the whole `collectors` folder, re-run that collector's + installer, and do a dry run (see its README). The changes are already covered by the + collector tests (`./collectors/tests/Invoke-CollectorTests.ps1`) if you want to run them first. + +**If something goes wrong**, go back to the version from before step 2 (`ORIG_HEAD`). The order +matters: an older API won't start on a database a newer one has upgraded, so restore the database +before starting it. + +1. Stop the API: + ```bash + sudo systemctl stop backup-dashboard-api + ``` +2. Check whether the update upgraded the database: if these two numbers differ, it did. + ```bash + for db in backup-dashboard before-update; do sudo -u backupdash sqlite3 /var/lib/backup-dashboard/$db.db "PRAGMA user_version"; done + ``` + If so, restore the backup from step 1 (`.restore`, not `cp`: it also deals with SQLite's `-wal` + file). Runs reported since the backup drop out of the history, since collectors have already + marked them as sent; backups themselves are unaffected. + ```bash + sudo -u backupdash sqlite3 /var/lib/backup-dashboard/backup-dashboard.db ".restore /var/lib/backup-dashboard/before-update.db" + ``` +3. Go back to the previous code and web files, then start the API: + ```bash + sudo git -C /opt/backup-dashboard checkout ORIG_HEAD + ``` + ```bash + sudo rsync -a /opt/backup-dashboard/public/ /var/www/backup-dashboard/public/ && sudo systemctl start backup-dashboard-api + ``` + +Once the problem is fixed, return to `main` with `sudo git -C /opt/backup-dashboard checkout main` +and update as above.