diff --git a/docs/DEVELOPMENT_AND_STAGING.md b/docs/DEVELOPMENT_AND_STAGING.md index 4b8c3cb..d1dd17f 100644 --- a/docs/DEVELOPMENT_AND_STAGING.md +++ b/docs/DEVELOPMENT_AND_STAGING.md @@ -156,7 +156,7 @@ The temporary VPS must not become another development checkout. The proposed rel Windows source and Git -> Rocky build and verification -> immutable image + checksum -> VPS deployment ``` -Releases are manual during user acceptance and are identified by the source commit SHA. The first installed release was `7dbc55b`; the current UAT release is `b5150ab`, deployed with its full immutable image tag. Release `df911f1` is retained as the immediate application rollback. This release adds the full Brīdinājumi (LVĢMC warning-map) workspace on top of the earlier JSON API response types, Latvian workflow labels, and independently calibrated water-temperature templates. The VPS keeps WeatherTool in its own `/srv/weathertool` tree, publishes the application only to loopback ports, and relies on Cloudflare, Nginx, and Authelia at the public boundary. See `VPS_STAGING_PLAN.md` before changing the VPS deployment. +Releases are manual during user acceptance and are identified by the source commit SHA. The first installed release was `7dbc55b`; the current UAT release is `6185dbf`, deployed with its full immutable image tag. Release `b5150ab` is retained as the immediate application rollback. This release hardens LVĢMC warning fetching (bounded outage retry, per-warning fetch isolation, a live-API-verified CKAN filter, and visible logging for a previously-silent vertex-order default) on top of the full Brīdinājumi (LVĢMC warning-map) workspace, JSON API response types, Latvian workflow labels, and independently calibrated water-temperature templates. The VPS keeps WeatherTool in its own `/srv/weathertool` tree, publishes the application only to loopback ports, and relies on Cloudflare, Nginx, and Authelia at the public boundary. See `VPS_STAGING_PLAN.md` before changing the VPS deployment. The committed templates live under `deploy/vps/`. They are not a second local development environment. Validate them on Rocky, build the exact Git revision, diff --git a/docs/README.md b/docs/README.md index 549afde..18bf46c 100644 --- a/docs/README.md +++ b/docs/README.md @@ -14,13 +14,13 @@ This directory contains the working documentation for the WeatherTool modernizat - **Ūdens** uses separate authoritative 1920×1080 and 3840×1440 production templates. Its six value rectangles are measured per template and the text is optically centered from rendered glyph bounds; both exports have been visually validated. - **Brīdinājumi** renders current LVĢMC warning polygons over a production border overlay with feathered severity fills. Its lon/lat-to-pixel projection is an affine fit calibrated against the same validated city pixel positions Kartes/Faktiskā already use, replacing an earlier bounding-box calibration that drifted up to ~200px on the 3840 canvas. - Confirmed local Monda Regular/Bold files provide interface and generated-graphic typography; weather symbols use normalized transparent image assets. -- Release `b5150ab` is deployed as immutable image `weathertool:b5150abd469036ca84b94e9b749d5d0232690f38`; release `df911f1` remains the immediate application rollback. This release adds the full Brīdinājumi (LVĢMC warning-map) workspace on top of the earlier JSON API, Latvian workflow labels, and calibrated water-temperature work. +- Release `6185dbf` is deployed as immutable image `weathertool:6185dbfdb0752db7d715f1061d74baab3c43122e`; release `b5150ab` remains the immediate application rollback. This release hardens LVĢMC warning fetching (bounded retry during upstream outages, per-warning fetch isolation, a CKAN filter verified against the live API, and visible logging for a previously-silent vertex-order fallback) on top of the full Brīdinājumi workspace, JSON API, Latvian workflow labels, and calibrated water-temperature work. - The isolated VPS UAT stack is running and healthy: WeatherTool is bound to `127.0.0.1:8002`, Authelia to `127.0.0.1:9091`, and PostgreSQL has no host port. Public access is routed through Cloudflare, Nginx, and Authelia. - Cloudflare delegation is active, strict origin TLS covers only `laikapstak.li` and `auth.laikapstak.li`, and the public Nginx/Authelia login flow is operational without changing the existing HOP site. - The VPS database contains a deliberately limited 14-day, 34-city synthetic dataset for newsroom visual testing. It must be cleared before authorized real-provider ingestion begins. - Approved 1920×1080 and 3840×1440 PNG production bases are now the rendering source for Faktiskā and Ūdens temperatūra; code draws only the changing values, selected weather symbols, and wind data over those fixed newsroom graphics. - Browser branding assets and Latvian Open Graph/Twitter metadata are included for favicon, Apple home-screen icon, and link-preview support. Public crawler access still depends on the Nginx/Authelia policy used for the metadata and preview image. -- Browser verification is complete for the deployed `b5150ab` release, including Brīdinājumi against real live LVĢMC data. Newsroom testers will be given the domain first, without walkthrough, to see how intuitive the workflow is before any guidance is provided. +- Browser verification is complete for the deployed `6185dbf` release, including Brīdinājumi against real live LVĢMC data (a genuinely severe live wind warning tested the hardened fetch path end to end). Newsroom testers will be given the domain first, without walkthrough, to see how intuitive the workflow is before any guidance is provided. - Frontend dependency maintenance is complete: Solid runtime and Vite tooling were updated, obsolete packages were removed, TypeScript checking was added, and a clean Rocky `npm ci`, typecheck, production build, full audit, and production-only audit all pass with zero known vulnerabilities. - This is not yet approved or hardened for workplace production. diff --git a/docs/UPDATE_ROADMAP.md b/docs/UPDATE_ROADMAP.md index 8c8df29..3342fcf 100644 --- a/docs/UPDATE_ROADMAP.md +++ b/docs/UPDATE_ROADMAP.md @@ -7,7 +7,7 @@ This document tracks proposed WeatherTool improvements. Work should be delivered - **Windows source workspace:** source editing, review, and Git operations only; do not install dependencies, compile, build, run, or test here. - **Rocky development and staging:** the sole compile, build, development-runtime, and test environment, with production-like Docker staging at `http://192.168.1.101:9190`. - **Git over SSH:** Windows pushes reviewed commits to a private bare repository on Rocky; the Rocky staging checkout pulls those commits and rebuilds. -- **Ubuntu VPS deployment:** release `b5150ab` is publicly operational behind Cloudflare strict TLS, Nginx, and Authelia with a limited 14-day synthetic dataset; the VPS does not compile or build the project. +- **Ubuntu VPS deployment:** release `6185dbf` is publicly operational behind Cloudflare strict TLS, Nginx, and Authelia with a limited 14-day synthetic dataset; the VPS does not compile or build the project. - **Workplace production:** remains separate until changes are reviewed, tested, and explicitly approved for workplace use. Do not synchronize `.env`, database directories, generated dependencies, build output, or provider credentials between machines. @@ -87,7 +87,7 @@ Dependency changes must not be combined with a visual redesign unless a package Status: pending -- [ ] Rotate and remove the API key exposed in a source comment. +- [ ] Rotate and remove the API key exposed in a source comment (`src/main/scala/fetch/dmi/FetchService.scala:56`, a DMI HARMONIE key introduced 2025-02-01; confirmed the repo's GitHub `origin` is public, so this has been externally exposed for roughly 18 months — decision on rotation vs. a git-history rewrite is still open). - [ ] Remove credentials from connection-error messages. - [ ] Protect or remove debug and administrative endpoints. - [ ] Convert state-changing `GET` routes to appropriate methods. @@ -110,6 +110,7 @@ Status: pending - [ ] Define PostgreSQL and GRIB-data backup/restore procedures. - [ ] Add container resource limits and deployment health checks. - [ ] Document monitoring, update, rollback, and incident procedures. +- [x] Harden `WarningService`: bound LVĢMC retry attempts to once per cache window during an upstream outage, isolate one warning's polygon-fetch failure from the rest instead of failing the whole response, verify the CKAN polygon filter against real column-type ambiguity rather than guessing, and log (rather than silently swallow) a defaulted vertex-order value. ## Phase 6 — Information architecture and UI redesign @@ -165,14 +166,14 @@ Status: in progress - [x] Provision a dedicated `/srv/weathertool` tree, Compose project, private network, database storage, and loopback-only application and Authelia ports. - [x] Configure Authelia with one temporary shared account, Argon2id password storage, rate limiting, and temporary IP bans. - [x] Add an exact-host Cloudflare origin certificate and Nginx authorization routing without disrupting the existing HOP site. -- [x] Build and verify commit-addressed images on Rocky, transfer and checksum-verify them on the VPS, and update only the application service. Current full-SHA image: `weathertool:b5150abd469036ca84b94e9b749d5d0232690f38`; PostgreSQL and Authelia were not restarted. +- [x] Build and verify commit-addressed images on Rocky, transfer and checksum-verify them on the VPS, and update only the application service. Current full-SHA image: `weathertool:6185dbfdb0752db7d715f1061d74baab3c43122e`; PostgreSQL and Authelia were not restarted. - [ ] Complete UAT verification: public authentication and application health pass; direct-origin blocking, logout, throttling, PNG downloads, backups, logs, and rollback remain. - [ ] Share the domain with newsroom testers without a walkthrough first, to observe unassisted intuitiveness, before providing any guidance. - [ ] Run the month-long user test with manual releases and record feedback before any workplace-production decision. ## Known current limitations -- The fixed production PNG bases, browser/social metadata, reviewed frontend dependency updates, stabilized header navigation, authenticated API routing, and the full Brīdinājumi warning-map workspace are deployed in UAT release `b5150ab`; newsroom workflow validation is in progress. +- The fixed production PNG bases, browser/social metadata, reviewed frontend dependency updates, stabilized header navigation, authenticated API routing, the full Brīdinājumi warning-map workspace, and hardened LVĢMC warning fetching are deployed in UAT release `6185dbf`; newsroom workflow validation is in progress. - Link-preview crawlers cannot authenticate through Authelia; the final Nginx policy must deliberately expose only the minimum preview metadata/assets if WhatsApp previews are required. - Staging uses synthetic PostgreSQL station data. @@ -217,3 +218,4 @@ Record completed work here by date and commit after the Git workflow is establis | 2026-08-22 | `70fd24c` | Give Brīdinājumi chips a hover-expanding severity capsule, adapted from a reviewed Uiverse.io reference component | Yes — Rocky typecheck, build, and headless-browser screenshot verification of rest/hover/active chip states; deployed to VPS in release `b5150ab` on 2026-08-22 | | 2026-08-22 | `6fbf672` | Split Brīdinājumi warning selection (checkbox) from viewing detail (chip click) and widen the detail popup | Yes — Rocky typecheck, build, and headless-browser verification that checkbox clicks change the map selection without opening the popup and chip clicks open the correct warning's detail without changing the selection; deployed to VPS in release `b5150ab` on 2026-08-22 | | 2026-08-22 | `56b8716` | Replace single-warning Brīdinājumi selection with multi-select checkboxes and severity-ordered polygon compositing, and restyle chips as narrower, fully severity-colored, white-icon/white-text buttons with a ported Uiverse.io checkbox component | Yes — Rocky typecheck, build, and headless-browser screenshot verification of multiple simultaneous checked warnings compositing correctly on the map; deployed to VPS in release `b5150ab` on 2026-08-22 | +| 2026-08-22 | `6185dbf` | Harden `WarningService` against partial upstream failures: bounded outage retry, per-warning fetch isolation, a live-API-verified CKAN polygon filter, and visible logging for a previously-silent vertex-order default | Yes — Scala tests, Rocky full-stack rebuild verified against real live LVĢMC data (including a genuinely severe 52,345-point wind-warning polygon rendering correctly); deployed to VPS in release `6185dbf` on 2026-08-22, verified against live data on the VPS itself | diff --git a/docs/VPS_STAGING_PLAN.md b/docs/VPS_STAGING_PLAN.md index f8fba69..861d58a 100644 --- a/docs/VPS_STAGING_PLAN.md +++ b/docs/VPS_STAGING_PLAN.md @@ -1,6 +1,6 @@ # Temporary VPS staging plan -Status: **Release `b5150ab` (Brīdinājumi warning maps) publicly operational behind Cloudflare, Nginx, and Authelia with temporary synthetic data** +Status: **Release `6185dbf` (hardened LVĢMC warning fetching) publicly operational behind Cloudflare, Nginx, and Authelia with temporary synthetic data** This document defines the next deployment stage for approximately one month of newsroom-user testing. It records the intended boundary before any VPS changes @@ -59,10 +59,10 @@ Cloudflare -> Nginx -> Authelia authorization -> WeatherTool The host root plus `releases/` and `backups/` exist with owner `ubuntu:ubuntu` and mode `0750`. Baseline release `7dbc55b` installed the reviewed deployment -bundle. The current release `b5150ab` was built from the exact Git archive on +bundle. The current release `6185dbf` was built from the exact Git archive on Rocky, smoke-tested via a full local Rocky staging rebuild, verified against its SHA-256 manifest on both Rocky and the VPS, and was loaded as the full-SHA -image `weathertool:b5150abd469036ca84b94e9b749d5d0232690f38`. It was deployed +image `weathertool:6185dbfdb0752db7d715f1061d74baab3c43122e`. It was deployed by recreating only the application service. The dedicated Compose stack is running; no existing HOP container, network, volume, port, or Nginx site was changed. @@ -286,6 +286,38 @@ detail popup, and a phenomenon-templated title. Newsroom testers have not yet been walked through the tool; the plan is to share the domain first and observe how intuitive the workflow is unassisted before providing guidance. +LVĢMC warning-fetch hardening completed on 2026-08-22: + +- a backend code review of `WarningService.scala` found the cache never + re-stamped its timestamp on a failed refresh (causing a failing upstream to + be retried on every request instead of once per cache window), one failing + per-warning polygon fetch aborting the entire response instead of only that + warning, a type-guessed CKAN filter that could silently return zero polygon + rows, and a silently-defaulted vertex-order field that could garble a + polygon's shape; +- the CKAN list-filter fix was verified directly against the live + `data.gov.lv` API (not assumed) before being applied; +- exact source revision `6185dbfdb0752db7d715f1061d74baab3c43122e` was built + from a clean Git archive using `deploy/vps/Dockerfile`; all three existing + Scala tests passed; +- a full local Rocky staging rebuild was verified against real live LVĢMC + data (7 active warnings, including a coincidentally severe live wind + warning with a 52,345-point polygon that rendered correctly and confirmed + the fix introduced no regression); +- the compressed image artifact passed SHA-256 verification on Rocky and the + VPS, and the loaded VPS image ID matched the Rocky image ID + `sha256:2ccd95f549cad52c38f50422d4426779595f10c8f9a4c8146d07522358c8892d`; +- `.env.staging` was backed up as `backups/env.staging.before-6185dbf` with + mode `0600` before selecting the new full-SHA image tag; +- only the WeatherTool application container was recreated; PostgreSQL and + Authelia were left running and healthy throughout; +- the application became healthy; loopback WeatherTool and Authelia health + checks returned HTTP 200; the unauthenticated public route returned the + expected Authelia redirect; the loopback `/api/warnings` route returned 7 + real live warnings on the VPS itself; and the unrelated HOP containers + remained healthy and untouched; and +- release `b5150ab` remains the immediate compatible application rollback. + ## Manual release procedure Continuous deployment is deliberately out of scope for the first testing