Add a literal, runnable VPS release runbook

VPS_STAGING_PLAN.md describes the release process at a policy level; the
changelog table narrates what happened after each release. Neither is a
step-by-step "how" a person (or Guntis) could actually follow without
already knowing the process from having run it. This is that document --
copy-pasteable commands, in order, parameterized by commit SHA.

Writing it surfaced a real regression: earlier releases (through ef64895)
retained the release tarball under /srv/weathertool/releases/<sha>/ and
backed up .env.staging before every edit; tonight's releases (0be325f,
9bab93d, 001b014) did neither -- I'd been deleting the transfer copy from
/tmp after each deploy and sed-ing .env.staging directly with no backup.
Practical cost: rolling back to any of tonight's releases would have meant
rebuilding from git history and re-transferring, instead of just re-pointing
.env.staging at an already-present image. Closed the gap for the current
live release (001b014) retroactively -- exported it directly on the VPS
(checksum matches the exact bytes shipped tonight, confirming no drift) and
backed up the current .env.staging -- and the runbook restores both habits
going forward.

docs/README.md

@@ -40,6 +40,7 @@ This directory contains the working documentation for the WeatherTool modernizat
 - [Continuous integration](CONTINUOUS_INTEGRATION.md) — Gitea Actions topology, isolation boundary, and the current CI workflow.
 - [Third-party notices](THIRD_PARTY_NOTICES.md) — licenses and attribution for adapted interface components.
 - [Temporary VPS staging plan](VPS_STAGING_PLAN.md) — isolation, authentication, prepared deployment bundle, release, backup, verification, and rollback model for external user testing.
+- [VPS release runbook](VPS_RELEASE_RUNBOOK.md) — the literal, copy-pasteable command sequence for shipping and rolling back a release, step by step.
 
 ## Documentation rules
 

docs/VPS_RELEASE_RUNBOOK.md

@@ -0,0 +1,183 @@
+# VPS release runbook
+
+This is the literal, runnable command sequence for shipping a new WeatherTool
+release to the VPS. `VPS_STAGING_PLAN.md` explains *why* the process looks
+this way (topology, isolation boundary, acceptance checklist) and gives the
+same steps at a policy level; this document is the *how* — copy-pasteable
+commands, run manually, in order, every time. There is no automation here
+deliberately: continuous deployment is out of scope while the VPS carries
+real testers, and every release is a deliberate human checkpoint (see
+`CONTINUOUS_INTEGRATION.md` for why CI does not perform this step either).
+
+Run every command from the Rocky checkout at
+`/home/sandbox/Documents/projects/WeatherTool` unless noted otherwise.
+
+## 0. Preconditions
+
+```bash
+git status --short   # must be empty — never build from an uncommitted tree
+git log --oneline -1
+```
+
+If CI exists for this commit, check it's green before proceeding — nothing
+else currently stops a red-CI commit from being deployed.
+
+## 1. Build the release image
+
+Always build from the committed tree via `git archive`, never from the
+working directory — a dirty or stale build has bitten this project before.
+
+```bash
+SHA=$(git rev-parse HEAD)
+echo "Building weathertool:$SHA"
+git archive HEAD | docker build -f deploy/vps/Dockerfile -t weathertool:$SHA -
+```
+
+## 2. Smoke-test the exact image
+
+Test the literal artifact that will ship, not just "the code" — against a
+throwaway Postgres and placeholder credentials, never real ones.
+
+```bash
+docker network create weathertool-smoketest-net
+docker run -d --name weathertool-smoketest-pg --network weathertool-smoketest-net \
+  -e POSTGRES_DB=smoketest -e POSTGRES_USER=smoketest -e POSTGRES_PASSWORD=smoketestpass \
+  postgres:16.1
+until docker exec weathertool-smoketest-pg pg_isready -U smoketest -d smoketest >/dev/null 2>&1; do sleep 1; done
+
+docker run -d --name weathertool-smoketest --network weathertool-smoketest-net \
+  -p 18080:8080 \
+  -e POSTGRES_DB=smoketest -e POSTGRES_USER=smoketest -e POSTGRES_PASSWORD=smoketestpass -e POSTGRES_HOST=weathertool-smoketest-pg \
+  -e LVGMC_URL=placeholder -e LVGMC_USER=placeholder -e LVGMC_PASSWORD=placeholder \
+  -e HARMONIE_EDR_URL=placeholder -e HARMONIE_EDR_API_KEY=placeholder -e HARMONIE_STAC_URL=placeholder -e HARMONIE_STAC_API_KEY=placeholder \
+  -e ENABLE_SCHEDULED_JOBS=false -e ENABLE_LVGMC_FTP_JOBS=false -e ENABLE_HARMONIE_JOBS=false \
+  weathertool:$SHA
+
+sleep 6
+docker logs weathertool-smoketest --tail 20   # expect clean startup, no "Fatal error"
+curl -s -o /dev/null -w "%{http_code}\n" http://localhost:18080/   # expect 200
+```
+
+Add release-specific checks here (route behavior, a known bug that's
+supposedly fixed, a real data path). Then tear down:
+
+```bash
+docker stop weathertool-smoketest weathertool-smoketest-pg
+docker rm weathertool-smoketest weathertool-smoketest-pg
+docker network rm weathertool-smoketest-net
+```
+
+If anything looks wrong, stop here. Fix it, commit, and restart from step 1
+with the new commit.
+
+## 3. Save, checksum, and retain the artifact
+
+```bash
+mkdir -p /tmp/weathertool-release
+OUT=/tmp/weathertool-release/weathertool-$SHA.tar.gz
+docker save weathertool:$SHA | gzip > "$OUT"
+sha256sum "$OUT" | tee "$OUT.sha256"
+```
+
+## 4. Transfer to the VPS and verify
+
+```bash
+scp "$OUT" "$OUT.sha256" vps:/tmp/
+
+ssh vps "cd /tmp && sha256sum -c weathertool-$SHA.tar.gz.sha256"
+# must print: weathertool-<SHA>.tar.gz: OK
+```
+
+Do not proceed past a checksum mismatch. Re-transfer or re-build.
+
+## 5. Retain the release on the VPS
+
+This is the step tonight's releases skipped — restore it. Keeping the
+tarball means a future rollback never needs Rocky at all.
+
+```bash
+ssh vps "sudo mkdir -p /srv/weathertool/releases/$SHA && sudo mv /tmp/weathertool-$SHA.tar.gz /tmp/weathertool-$SHA.tar.gz.sha256 /srv/weathertool/releases/$SHA/"
+```
+
+## 6. Load the image
+
+```bash
+ssh vps "sudo docker load -i /srv/weathertool/releases/$SHA/weathertool-$SHA.tar.gz"
+```
+
+## 7. Back up `.env.staging` before editing it
+
+Also restore this — it's how earlier releases (through `ef64895`) protected
+against a bad edit to a file that lives outside Git and holds real secrets.
+
+```bash
+ssh vps "sudo mkdir -p /srv/weathertool/backups && sudo cp /srv/weathertool/.env.staging /srv/weathertool/backups/env.staging.before-$SHA && sudo chmod 600 /srv/weathertool/backups/env.staging.before-$SHA"
+```
+
+## 8. Point at the new image and roll out
+
+```bash
+ssh vps "cd /srv/weathertool && sudo sed -i 's|^WEATHERTOOL_IMAGE=.*|WEATHERTOOL_IMAGE=weathertool:$SHA|' .env.staging && grep WEATHERTOOL_IMAGE .env.staging"
+ssh vps "cd /srv/weathertool && sudo docker compose --env-file .env.staging up -d --no-deps app"
+```
+
+`--no-deps app` is deliberate — this recreates only the WeatherTool
+container. PostgreSQL and Authelia are never touched by an application
+release.
+
+If this release also needs a database change (schema, backfill, wipe),
+back up PostgreSQL first — see `VPS_STAGING_PLAN.md`'s acceptance
+checklist. That is a separate, explicit decision from the steps here.
+
+## 9. Verify
+
+```bash
+ssh vps "sudo docker ps --filter name=weathertool-uat-app-1 --format '{{.Status}}'"   # expect Up ... (healthy)
+ssh vps "sudo docker logs weathertool-uat-app-1 --tail 20"                            # expect clean startup, no errors
+ssh vps "curl -s -o /dev/null -w '%{http_code}\n' http://127.0.0.1:8002/"             # expect 200
+
+# public boundary — confirm Authelia/Cloudflare gate is unchanged
+curl -s -o /dev/null -w "%{http_code}\n" https://laikapstak.li/          # expect 302 (unauthenticated redirect)
+curl -s -o /dev/null -w "%{http_code}\n" https://laikapstak.li/api/warnings   # expect 401 (unauthenticated API)
+```
+
+Add release-specific loopback checks for whatever this release actually
+changed — a bugfix should be checked against the live VPS data, not assumed
+from the Rocky smoke test alone.
+
+## 10. Clean up and record
+
+```bash
+rm -f /tmp/weathertool-release/weathertool-$SHA.tar.gz*
+```
+
+The transfer copy in `/tmp` on both ends is temporary and safe to delete —
+the real, retained copy is `/srv/weathertool/releases/$SHA/` from step 5.
+
+Then update:
+
+- `docs/README.md` — the "Release `X` is deployed..." status line, with the
+  new image tag and rollback target (the *previous* release's SHA).
+- `docs/UPDATE_ROADMAP.md`'s change log table — one row: date, SHA, what
+  shipped, and exactly how it was verified (not just "yes," the specific
+  checks run).
+
+## Rollback
+
+Only needed if verification in step 9 fails, or a problem surfaces after
+release.
+
+```bash
+# PREVIOUS_SHA = the rollback target recorded in the last README release line
+ssh vps "sudo docker load -i /srv/weathertool/releases/$PREVIOUS_SHA/weathertool-$PREVIOUS_SHA.tar.gz"   # only if that image was pruned locally; usually still loaded
+ssh vps "cd /srv/weathertool && sudo sed -i 's|^WEATHERTOOL_IMAGE=.*|WEATHERTOOL_IMAGE=weathertool:$PREVIOUS_SHA|' .env.staging"
+ssh vps "cd /srv/weathertool && sudo docker compose --env-file .env.staging up -d --no-deps app"
+```
+
+Then repeat step 9's verification against the rolled-back release, and
+record the rollback in the roadmap change log the same way as a forward
+release.
+
+A database change is not undone by an application rollback — if the release
+being rolled back touched the database, that needs its own explicit
+decision and the relevant PostgreSQL backup from step 8.