Document scheduler split and METEO_* removal

This commit is contained in:
b0txec
2026-08-23 10:50:08 +03:00
parent 39fcb3ed6b
commit ea1c563ff3
+23 -8
View File
@@ -185,14 +185,27 @@ running in parallel until each real source is proven, not cut over in one step.
should show no data rather than falling back to old synthetic rows —
deferred by design, revisit once a real source is found or accepted as
permanently unavailable.
- [ ] Once the open-data fetch has run for a trial period, remove the
private FTP path (`fetch/lvgmc/FetchService.scala`'s
`fetchWeatherStations`, its `LVGMC_*` env vars) if it remains unused.
- [ ] Register free DMI Open Data credentials and validate DMI STAC discovery
and EDR GRIB downloads (code already exists and is already scheduled;
this is credential registration and verification, not new code —
confirm first whether DMI's endpoint migration away from
`dmigw.govcloud.dk` changes what's needed).
- [x] Deleted the dead `METEO_*` fetch path (`fetch/csv/FetchService.scala`,
its empty test, and its env vars everywhere). Confirmed via git history
it wasn't a separate vendor: the first commit (2023-04-13) included
real sample CSVs from it with a Latvian header identical to the
LVGMC/open-data fields — the same underlying LVĢMC data, just an
earlier delivery mechanism superseded by the FTP feed and never
cleaned up. It was never wired into anything that runs.
- [x] Split scheduled jobs so the working open-data fetch (and cleanup) can
run without the FTP/Harmonie jobs' placeholder credentials
crash-looping the whole app: `ENABLE_SCHEDULED_JOBS` now only gates
the safe jobs; a new `ENABLE_LEGACY_PROVIDER_JOBS` (default false)
independently gates the FTP station fetch and Harmonie fetch. Turned
the former on for Rocky staging; real station data now accumulates
automatically every 30 minutes with no manual trigger needed.
- [ ] FTP (`fetch/lvgmc/FetchService.scala`) and Harmonie/DMI credentials are
the user's own follow-up (asking the original dev about the FTP
partner feed; registering for free DMI Open Data). Both stay gated
behind `ENABLE_LEGACY_PROVIDER_JOBS`/missing keys — not removed — so
no code changes are needed once real credentials arrive, just setting
them and flipping the flag. Worth confirming DMI's endpoint migration
away from `dmigw.govcloud.dk` before wiring anything.
- [ ] Investigate open real sources for water temperature (Ūdens) and the
LVĢMC forecast CSV workspace, following the same open-data-portal
approach used for station observations.
@@ -266,3 +279,5 @@ Record completed work here by date and commit after the Git workflow is establis
| 2026-08-23 | `c97875d` | Add draggable, resizable per-warning weather-symbol placement to Brīdinājumi, reusing Faktiskā's symbol set and image cache | Yes — Rocky typecheck, build, and headless-browser mouse-drag/resize verification against fixture warning data (the live LVĢMC feed had zero active warnings at test time); confirmed the exported PNG excludes the preview-only drag/resize handles; VPS deployment pending |
| 2026-08-23 | `893a09a` | Add `fetch.lvgmc.OpenDataStationService`: real, free, keyless current station observations from LVĢMC's open-data portal, scheduled alongside (not replacing) the private FTP feed | Yes — Scala tests, manual `sbt runMain` fetch+write against the real feed on Rocky staging (1,577 rows, 33 cities), confirmed in `weather` via direct SQL, Faktiskā's 13 fixed stations rendering real temperatures with correct stale-fallback for the one uncovered city (Valmiera), and Kartes' aggregate query API returning sane blended values; VPS deployment pending |
| 2026-08-23 | `cb25316` | Stop generating synthetic weather data (remove `dev/seed_weather.sql`, the `seed` Compose service, and the README section describing it) now that real data is verified and this tool may plausibly be used against live broadcasts | Yes — `docker compose config` valid, running Rocky stack unaffected (profile-gated service, no running containers touched); VPS deployment pending |
| 2026-08-23 | `6ddfe73` | Split scheduled jobs (`ENABLE_SCHEDULED_JOBS` vs. new `ENABLE_LEGACY_PROVIDER_JOBS`) so the working open-data station fetch runs without the FTP/Harmonie jobs' placeholder credentials crash-looping the whole app via `parMapN` | Yes — Scala tests, Rocky rebuild deployed with the flag on: logs confirm only the safe jobs scheduled, FTP/Harmonie never attempted, app stayed up; VPS deployment pending |
| 2026-08-23 | `39fcb3e` | Remove the dead `METEO_*` fetch path (`fetch/csv/FetchService.scala`, its empty test, and its env vars everywhere) — confirmed via git history it's the same underlying LVĢMC data as the FTP/open-data paths, just an earlier, superseded delivery mechanism that was never wired into anything running | Yes — Scala tests (2 suites now, down from 3, the deleted one was empty); VPS deployment pending |