Record the 138f57c8 full go-live VPS release
Deployed everything since 6185dbf (Brīdinājumi draggable symbols, real
open-data station/water-temperature ingestion, synthetic-data removal,
scheduler split, METEO_* deletion, real per-zone water-temperature
ranges), flipped ENABLE_SCHEDULED_JOBS=true on the VPS, and wiped the
VPS weather table's synthetic rows after a PostgreSQL backup.
This commit is contained in:
b0txec
+25
-17
@@ -7,7 +7,7 @@ This document tracks proposed WeatherTool improvements. Work should be delivered
|
||||
- **Windows source workspace:** source editing, review, and Git operations only; do not install dependencies, compile, build, run, or test here.
|
||||
- **Rocky development and staging:** the sole compile, build, development-runtime, and test environment, with production-like Docker staging at `http://192.168.1.101:9190`.
|
||||
- **Git over SSH:** Windows pushes reviewed commits to a private bare repository on Rocky; the Rocky staging checkout pulls those commits and rebuilds.
|
||||
- **Ubuntu VPS deployment:** release `6185dbf` is publicly operational behind Cloudflare strict TLS, Nginx, and Authelia with a limited 14-day synthetic dataset; the VPS does not compile or build the project.
|
||||
- **Ubuntu VPS deployment:** release `138f57c8` is publicly operational behind Cloudflare strict TLS, Nginx, and Authelia, ingesting real LVĢMC open-data station/water-temperature observations on a schedule (synthetic data wiped 2026-08-23); the VPS does not compile or build the project.
|
||||
- **Workplace production:** remains separate until changes are reviewed, tested, and explicitly approved for workplace use.
|
||||
|
||||
Do not synchronize `.env`, database directories, generated dependencies, build output, or provider credentials between machines.
|
||||
@@ -178,10 +178,16 @@ running in parallel until each real source is proven, not cut over in one step.
|
||||
real/fake data — both shown identically as "older observation" — is a
|
||||
real hazard, not just messiness: removed `dev/seed_weather.sql`, the
|
||||
`seed` Compose service, and the README section describing it.
|
||||
- [ ] Wipe the synthetic rows already sitting in the `weather` table (a
|
||||
separate step from stopping generation — the seed already ran once
|
||||
before this decision) so history going forward is real data only, not
|
||||
a mix. Cities with no matching open-data station (Valmiera, Cēsis)
|
||||
- [x] Wipe the synthetic rows already sitting in the VPS `weather` table
|
||||
(backed up to `/srv/weathertool/backups/pre-real-data-release/` first)
|
||||
as part of the 2026-08-23 VPS release, so VPS history going forward is
|
||||
real data only, not a mix — confirmed Valmiera (no matching open-data
|
||||
station) now shows no data rather than a stale synthetic fallback.
|
||||
**Rocky's own `weather` table still holds its original synthetic rows
|
||||
mixed with real ones (449k rows as of 2026-08-23) — not wiped yet,
|
||||
since Rocky is a development/staging environment rather than the
|
||||
newsroom-facing UAT domain; revisit if it starts interfering with
|
||||
testing.** Cities with no matching open-data station (Valmiera, Cēsis)
|
||||
should show no data rather than falling back to old synthetic rows —
|
||||
deferred by design, revisit once a real source is found or accepted as
|
||||
permanently unavailable.
|
||||
@@ -231,20 +237,20 @@ Status: in progress
|
||||
- [x] Provision a dedicated `/srv/weathertool` tree, Compose project, private network, database storage, and loopback-only application and Authelia ports.
|
||||
- [x] Configure Authelia with one temporary shared account, Argon2id password storage, rate limiting, and temporary IP bans.
|
||||
- [x] Add an exact-host Cloudflare origin certificate and Nginx authorization routing without disrupting the existing HOP site.
|
||||
- [x] Build and verify commit-addressed images on Rocky, transfer and checksum-verify them on the VPS, and update only the application service. Current full-SHA image: `weathertool:6185dbfdb0752db7d715f1061d74baab3c43122e`; PostgreSQL and Authelia were not restarted.
|
||||
- [x] Build and verify commit-addressed images on Rocky, transfer and checksum-verify them on the VPS, and update only the application service. Current full-SHA image: `weathertool:138f57c808631dc17396b22ac0305b670fbcf776`; PostgreSQL and Authelia were not restarted.
|
||||
- [ ] Complete UAT verification: public authentication and application health pass; direct-origin blocking, logout, throttling, PNG downloads, backups, logs, and rollback remain.
|
||||
- [ ] Share the domain with newsroom testers without a walkthrough first, to observe unassisted intuitiveness, before providing any guidance.
|
||||
- [ ] Run the month-long user test with manual releases and record feedback before any workplace-production decision.
|
||||
|
||||
## Known current limitations
|
||||
|
||||
- The fixed production PNG bases, browser/social metadata, reviewed frontend dependency updates, stabilized header navigation, authenticated API routing, the full Brīdinājumi warning-map workspace, and hardened LVĢMC warning fetching are deployed in UAT release `6185dbf`; newsroom workflow validation is in progress.
|
||||
- The fixed production PNG bases, browser/social metadata, reviewed frontend dependency updates, stabilized header navigation, authenticated API routing, the full Brīdinājumi warning-map workspace (including draggable per-warning symbol placement), hardened LVĢMC warning fetching, real open-data station/water-temperature ingestion, and the scheduler safety split are deployed in UAT release `138f57c8`; newsroom workflow validation is in progress.
|
||||
- Link-preview crawlers cannot authenticate through Authelia; the final Nginx policy must deliberately expose only the minimum preview metadata/assets if WhatsApp previews are required.
|
||||
|
||||
- Staging uses synthetic PostgreSQL station data.
|
||||
- The VPS `weather` table now holds only real open-data station observations (synthetic rows wiped 2026-08-23); Rocky's own `weather` table still has its original synthetic rows mixed in with real ones, not yet wiped.
|
||||
- LVGMC forecast CSV fixtures are not yet available.
|
||||
- HARMONIE GRIB fixtures are not yet available.
|
||||
- Scheduled provider downloads are disabled in development and staging.
|
||||
- The legacy private LVGMC FTP feed and DMI HARMONIE forecast feed remain gated behind `ENABLE_LEGACY_PROVIDER_JOBS` (default off) pending real credentials, in both development and staging (Rocky and VPS).
|
||||
- Existing automated test coverage is minimal.
|
||||
- Direct refreshes on newer frontend routes can return 404 until the backend gains a general SPA fallback.
|
||||
- Full Docker build context scanning on Rocky can fail on the container-owned `postgres/` bind directory; do not loosen its permissions.
|
||||
@@ -284,11 +290,13 @@ Record completed work here by date and commit after the Git workflow is establis
|
||||
| 2026-08-22 | `6fbf672` | Split Brīdinājumi warning selection (checkbox) from viewing detail (chip click) and widen the detail popup | Yes — Rocky typecheck, build, and headless-browser verification that checkbox clicks change the map selection without opening the popup and chip clicks open the correct warning's detail without changing the selection; deployed to VPS in release `b5150ab` on 2026-08-22 |
|
||||
| 2026-08-22 | `56b8716` | Replace single-warning Brīdinājumi selection with multi-select checkboxes and severity-ordered polygon compositing, and restyle chips as narrower, fully severity-colored, white-icon/white-text buttons with a ported Uiverse.io checkbox component | Yes — Rocky typecheck, build, and headless-browser screenshot verification of multiple simultaneous checked warnings compositing correctly on the map; deployed to VPS in release `b5150ab` on 2026-08-22 |
|
||||
| 2026-08-22 | `6185dbf` | Harden `WarningService` against partial upstream failures: bounded outage retry, per-warning fetch isolation, a live-API-verified CKAN polygon filter, and visible logging for a previously-silent vertex-order default | Yes — Scala tests, Rocky full-stack rebuild verified against real live LVĢMC data (including a genuinely severe 52,345-point wind-warning polygon rendering correctly); deployed to VPS in release `6185dbf` on 2026-08-22, verified against live data on the VPS itself |
|
||||
| 2026-08-23 | `c97875d` | Add draggable, resizable per-warning weather-symbol placement to Brīdinājumi, reusing Faktiskā's symbol set and image cache | Yes — Rocky typecheck, build, and headless-browser mouse-drag/resize verification against fixture warning data (the live LVĢMC feed had zero active warnings at test time); confirmed the exported PNG excludes the preview-only drag/resize handles; VPS deployment pending |
|
||||
| 2026-08-23 | `893a09a` | Add `fetch.lvgmc.OpenDataStationService`: real, free, keyless current station observations from LVĢMC's open-data portal, scheduled alongside (not replacing) the private FTP feed | Yes — Scala tests, manual `sbt runMain` fetch+write against the real feed on Rocky staging (1,577 rows, 33 cities), confirmed in `weather` via direct SQL, Faktiskā's 13 fixed stations rendering real temperatures with correct stale-fallback for the one uncovered city (Valmiera), and Kartes' aggregate query API returning sane blended values; VPS deployment pending |
|
||||
| 2026-08-23 | `cb25316` | Stop generating synthetic weather data (remove `dev/seed_weather.sql`, the `seed` Compose service, and the README section describing it) now that real data is verified and this tool may plausibly be used against live broadcasts | Yes — `docker compose config` valid, running Rocky stack unaffected (profile-gated service, no running containers touched); VPS deployment pending |
|
||||
| 2026-08-23 | `6ddfe73` | Split scheduled jobs (`ENABLE_SCHEDULED_JOBS` vs. new `ENABLE_LEGACY_PROVIDER_JOBS`) so the working open-data station fetch runs without the FTP/Harmonie jobs' placeholder credentials crash-looping the whole app via `parMapN` | Yes — Scala tests, Rocky rebuild deployed with the flag on: logs confirm only the safe jobs scheduled, FTP/Harmonie never attempted, app stayed up; VPS deployment pending |
|
||||
| 2026-08-23 | `39fcb3e` | Remove the dead `METEO_*` fetch path (`fetch/csv/FetchService.scala`, its empty test, and its env vars everywhere) — confirmed via git history it's the same underlying LVĢMC data as the FTP/open-data paths, just an earlier, superseded delivery mechanism that was never wired into anything running | Yes — Scala tests (2 suites now, down from 3, the deleted one was empty); VPS deployment pending |
|
||||
| 2026-08-23 | `4036d24`–`4b92272` | Add `fetch.lvgmc.WaterTemperatureService`: real, free, keyless water temperatures for all 6 Ūdens zones (2 coastal `SEDUT` stations for Jūra/Līcis, 4 inland `WTEMD` stations for Kurzeme/Zemgale/Vidzeme/Latgale), fetch-on-demand with a 15-minute in-memory cache mirroring `WarningService` rather than the persisted station-observation path, since Ūdens has never stored history | Yes — Scala tests, Rocky rebuild, direct API curl confirming real values for all 6 zones; caught and fixed one station-selection error (Daugavpils only reports water level, not temperature — swapped for Ludza) via direct verification before committing; VPS deployment pending |
|
||||
| 2026-08-23 | `5a92465` | Auto-populate Ūdens's min/max fields from `/api/water-temperatures` on load, mirroring Faktiskā's fetch/override/reset pattern, with per-zone manual-override tracking and an "Atiestatīt" reset button | Yes — Rocky typecheck, build, and headless-browser verification (Playwright in Docker, since `chromium-cli` wasn't available in this environment) confirming auto-populated values match the API, manual edit correctly flips a zone to "Manuāli" and enables its reset button, and reset correctly restores the fetched value; noted along the way that `/udens-temperatura` 404s on a direct hit (no SPA fallback yet, consistent with the known limitation already listed) but works via client-side nav; VPS deployment pending |
|
||||
| 2026-08-23 | `9eca9eb` | Fix the one-station-per-zone design catching a real problem after deploy: every zone showed the same value duplicated as both min and max ("19...19"). Classified all 65 LVĢMC stations that report water temperature (56 inland WTEMD, 9 coastal SEDUT) into the 6 zones by geography and report the real min/max across each zone's currently-reporting stations, dropping readings over 12h stale so one stuck sensor can't skew a range | Yes — Scala tests, Rocky rebuild, direct API curl confirming a genuine spread per zone (e.g. Vidzeme 11.4–18.7°C, traced the low point to a real fresh reading from a colder headwater station near Cēsis, not a stale-sensor artifact), and headless-browser screenshot confirming the UI renders the real spread (e.g. "11...19") with no console errors; VPS deployment pending |
|
||||
| 2026-08-23 | `c97875d` | Add draggable, resizable per-warning weather-symbol placement to Brīdinājumi, reusing Faktiskā's symbol set and image cache | Yes — Rocky typecheck, build, and headless-browser mouse-drag/resize verification against fixture warning data (the live LVĢMC feed had zero active warnings at test time); confirmed the exported PNG excludes the preview-only drag/resize handles; deployed to VPS in release `138f57c8` on 2026-08-23 |
|
||||
| 2026-08-23 | `893a09a` | Add `fetch.lvgmc.OpenDataStationService`: real, free, keyless current station observations from LVĢMC's open-data portal, scheduled alongside (not replacing) the private FTP feed | Yes — Scala tests, manual `sbt runMain` fetch+write against the real feed on Rocky staging (1,577 rows, 33 cities), confirmed in `weather` via direct SQL, Faktiskā's 13 fixed stations rendering real temperatures with correct stale-fallback for the one uncovered city (Valmiera), and Kartes' aggregate query API returning sane blended values; deployed to VPS in release `138f57c8` on 2026-08-23 |
|
||||
| 2026-08-23 | `cb25316` | Stop generating synthetic weather data (remove `dev/seed_weather.sql`, the `seed` Compose service, and the README section describing it) now that real data is verified and this tool may plausibly be used against live broadcasts | Yes — `docker compose config` valid, running Rocky stack unaffected (profile-gated service, no running containers touched); deployed to VPS in release `138f57c8` on 2026-08-23 |
|
||||
| 2026-08-23 | `6ddfe73` | Split scheduled jobs (`ENABLE_SCHEDULED_JOBS` vs. new `ENABLE_LEGACY_PROVIDER_JOBS`) so the working open-data station fetch runs without the FTP/Harmonie jobs' placeholder credentials crash-looping the whole app via `parMapN` | Yes — Scala tests, Rocky rebuild deployed with the flag on: logs confirm only the safe jobs scheduled, FTP/Harmonie never attempted, app stayed up; deployed to VPS in release `138f57c8` on 2026-08-23 |
|
||||
| 2026-08-23 | `39fcb3e` | Remove the dead `METEO_*` fetch path (`fetch/csv/FetchService.scala`, its empty test, and its env vars everywhere) — confirmed via git history it's the same underlying LVĢMC data as the FTP/open-data paths, just an earlier, superseded delivery mechanism that was never wired into anything running | Yes — Scala tests (2 suites now, down from 3, the deleted one was empty); deployed to VPS in release `138f57c8` on 2026-08-23 |
|
||||
| 2026-08-23 | `4036d24`–`4b92272` | Add `fetch.lvgmc.WaterTemperatureService`: real, free, keyless water temperatures for all 6 Ūdens zones (2 coastal `SEDUT` stations for Jūra/Līcis, 4 inland `WTEMD` stations for Kurzeme/Zemgale/Vidzeme/Latgale), fetch-on-demand with a 15-minute in-memory cache mirroring `WarningService` rather than the persisted station-observation path, since Ūdens has never stored history | Yes — Scala tests, Rocky rebuild, direct API curl confirming real values for all 6 zones; caught and fixed one station-selection error (Daugavpils only reports water level, not temperature — swapped for Ludza) via direct verification before committing; deployed to VPS in release `138f57c8` on 2026-08-23 |
|
||||
| 2026-08-23 | `5a92465` | Auto-populate Ūdens's min/max fields from `/api/water-temperatures` on load, mirroring Faktiskā's fetch/override/reset pattern, with per-zone manual-override tracking and an "Atiestatīt" reset button | Yes — Rocky typecheck, build, and headless-browser verification (Playwright in Docker, since `chromium-cli` wasn't available in this environment) confirming auto-populated values match the API, manual edit correctly flips a zone to "Manuāli" and enables its reset button, and reset correctly restores the fetched value; noted along the way that `/udens-temperatura` 404s on a direct hit (no SPA fallback yet, consistent with the known limitation already listed) but works via client-side nav; deployed to VPS in release `138f57c8` on 2026-08-23 |
|
||||
| 2026-08-23 | `9eca9eb` | Fix the one-station-per-zone design catching a real problem after deploy: every zone showed the same value duplicated as both min and max ("19...19"). Classified all 65 LVĢMC stations that report water temperature (56 inland WTEMD, 9 coastal SEDUT) into the 6 zones by geography and report the real min/max across each zone's currently-reporting stations, dropping readings over 12h stale so one stuck sensor can't skew a range | Yes — Scala tests, Rocky rebuild, direct API curl confirming a genuine spread per zone (e.g. Vidzeme 11.4–18.7°C, traced the low point to a real fresh reading from a colder headwater station near Cēsis, not a stale-sensor artifact), and headless-browser screenshot confirming the UI renders the real spread (e.g. "11...19") with no console errors; deployed to VPS in release `138f57c8` on 2026-08-23 |
|
||||
| 2026-08-23 | `69a5260` | Bring architecture/workflow docs current with real-data ingestion (removed synthetic-seed staging descriptions, documented Ūdens auto-populate, the scheduler split) | Yes — doc-only change, no rebuild needed |
|
||||
| 2026-08-23 | `138f57c8` | **Full go-live VPS release**: deployed image `weathertool:138f57c808631dc17396b22ac0305b670fbcf776` (everything from `c97875d` through `69a5260` — Brīdinājumi draggable symbol placement, real open-data station/water-temperature ingestion, synthetic-data removal, scheduler split, `METEO_*` deletion, real per-zone water-temperature ranges), flipped `ENABLE_SCHEDULED_JOBS=true` on the VPS now that the open-data path is proven, and wiped the VPS `weather` table's original 12,784 synthetic rows down to 0 (Rocky's own table was left untouched — separate decision, not requested for this release) | Yes — exact release image smoke-tested on Rocky (healthy, real warnings/water-temperature data) before transfer; VPS PostgreSQL backed up to `/srv/weathertool/backups/pre-real-data-release/` before the wipe; checksum verified on both ends; `app` service recreated without touching PostgreSQL/Authelia; confirmed healthy container, working loopback and public HTTPS (302 unauthenticated page, 401 unauthenticated API — matching established behavior), real warnings/water-temperature API responses, the scheduled open-data job firing on schedule and landing 1,577 real rows within ~2 minutes, and a real (non-Valmiera) city query returning genuine current temperatures with Valmiera correctly absent rather than falling back to wiped synthetic data |
|
||||
|
||||
Reference in New Issue
Block a user