Document scheduler split and METEO_* removal
This commit is contained in:
+23
-8
@@ -185,14 +185,27 @@ running in parallel until each real source is proven, not cut over in one step.
|
|||||||
should show no data rather than falling back to old synthetic rows —
|
should show no data rather than falling back to old synthetic rows —
|
||||||
deferred by design, revisit once a real source is found or accepted as
|
deferred by design, revisit once a real source is found or accepted as
|
||||||
permanently unavailable.
|
permanently unavailable.
|
||||||
- [ ] Once the open-data fetch has run for a trial period, remove the
|
- [x] Deleted the dead `METEO_*` fetch path (`fetch/csv/FetchService.scala`,
|
||||||
private FTP path (`fetch/lvgmc/FetchService.scala`'s
|
its empty test, and its env vars everywhere). Confirmed via git history
|
||||||
`fetchWeatherStations`, its `LVGMC_*` env vars) if it remains unused.
|
it wasn't a separate vendor: the first commit (2023-04-13) included
|
||||||
- [ ] Register free DMI Open Data credentials and validate DMI STAC discovery
|
real sample CSVs from it with a Latvian header identical to the
|
||||||
and EDR GRIB downloads (code already exists and is already scheduled;
|
LVGMC/open-data fields — the same underlying LVĢMC data, just an
|
||||||
this is credential registration and verification, not new code —
|
earlier delivery mechanism superseded by the FTP feed and never
|
||||||
confirm first whether DMI's endpoint migration away from
|
cleaned up. It was never wired into anything that runs.
|
||||||
`dmigw.govcloud.dk` changes what's needed).
|
- [x] Split scheduled jobs so the working open-data fetch (and cleanup) can
|
||||||
|
run without the FTP/Harmonie jobs' placeholder credentials
|
||||||
|
crash-looping the whole app: `ENABLE_SCHEDULED_JOBS` now only gates
|
||||||
|
the safe jobs; a new `ENABLE_LEGACY_PROVIDER_JOBS` (default false)
|
||||||
|
independently gates the FTP station fetch and Harmonie fetch. Turned
|
||||||
|
the former on for Rocky staging; real station data now accumulates
|
||||||
|
automatically every 30 minutes with no manual trigger needed.
|
||||||
|
- [ ] FTP (`fetch/lvgmc/FetchService.scala`) and Harmonie/DMI credentials are
|
||||||
|
the user's own follow-up (asking the original dev about the FTP
|
||||||
|
partner feed; registering for free DMI Open Data). Both stay gated
|
||||||
|
behind `ENABLE_LEGACY_PROVIDER_JOBS`/missing keys — not removed — so
|
||||||
|
no code changes are needed once real credentials arrive, just setting
|
||||||
|
them and flipping the flag. Worth confirming DMI's endpoint migration
|
||||||
|
away from `dmigw.govcloud.dk` before wiring anything.
|
||||||
- [ ] Investigate open real sources for water temperature (Ūdens) and the
|
- [ ] Investigate open real sources for water temperature (Ūdens) and the
|
||||||
LVĢMC forecast CSV workspace, following the same open-data-portal
|
LVĢMC forecast CSV workspace, following the same open-data-portal
|
||||||
approach used for station observations.
|
approach used for station observations.
|
||||||
@@ -266,3 +279,5 @@ Record completed work here by date and commit after the Git workflow is establis
|
|||||||
| 2026-08-23 | `c97875d` | Add draggable, resizable per-warning weather-symbol placement to Brīdinājumi, reusing Faktiskā's symbol set and image cache | Yes — Rocky typecheck, build, and headless-browser mouse-drag/resize verification against fixture warning data (the live LVĢMC feed had zero active warnings at test time); confirmed the exported PNG excludes the preview-only drag/resize handles; VPS deployment pending |
|
| 2026-08-23 | `c97875d` | Add draggable, resizable per-warning weather-symbol placement to Brīdinājumi, reusing Faktiskā's symbol set and image cache | Yes — Rocky typecheck, build, and headless-browser mouse-drag/resize verification against fixture warning data (the live LVĢMC feed had zero active warnings at test time); confirmed the exported PNG excludes the preview-only drag/resize handles; VPS deployment pending |
|
||||||
| 2026-08-23 | `893a09a` | Add `fetch.lvgmc.OpenDataStationService`: real, free, keyless current station observations from LVĢMC's open-data portal, scheduled alongside (not replacing) the private FTP feed | Yes — Scala tests, manual `sbt runMain` fetch+write against the real feed on Rocky staging (1,577 rows, 33 cities), confirmed in `weather` via direct SQL, Faktiskā's 13 fixed stations rendering real temperatures with correct stale-fallback for the one uncovered city (Valmiera), and Kartes' aggregate query API returning sane blended values; VPS deployment pending |
|
| 2026-08-23 | `893a09a` | Add `fetch.lvgmc.OpenDataStationService`: real, free, keyless current station observations from LVĢMC's open-data portal, scheduled alongside (not replacing) the private FTP feed | Yes — Scala tests, manual `sbt runMain` fetch+write against the real feed on Rocky staging (1,577 rows, 33 cities), confirmed in `weather` via direct SQL, Faktiskā's 13 fixed stations rendering real temperatures with correct stale-fallback for the one uncovered city (Valmiera), and Kartes' aggregate query API returning sane blended values; VPS deployment pending |
|
||||||
| 2026-08-23 | `cb25316` | Stop generating synthetic weather data (remove `dev/seed_weather.sql`, the `seed` Compose service, and the README section describing it) now that real data is verified and this tool may plausibly be used against live broadcasts | Yes — `docker compose config` valid, running Rocky stack unaffected (profile-gated service, no running containers touched); VPS deployment pending |
|
| 2026-08-23 | `cb25316` | Stop generating synthetic weather data (remove `dev/seed_weather.sql`, the `seed` Compose service, and the README section describing it) now that real data is verified and this tool may plausibly be used against live broadcasts | Yes — `docker compose config` valid, running Rocky stack unaffected (profile-gated service, no running containers touched); VPS deployment pending |
|
||||||
|
| 2026-08-23 | `6ddfe73` | Split scheduled jobs (`ENABLE_SCHEDULED_JOBS` vs. new `ENABLE_LEGACY_PROVIDER_JOBS`) so the working open-data station fetch runs without the FTP/Harmonie jobs' placeholder credentials crash-looping the whole app via `parMapN` | Yes — Scala tests, Rocky rebuild deployed with the flag on: logs confirm only the safe jobs scheduled, FTP/Harmonie never attempted, app stayed up; VPS deployment pending |
|
||||||
|
| 2026-08-23 | `39fcb3e` | Remove the dead `METEO_*` fetch path (`fetch/csv/FetchService.scala`, its empty test, and its env vars everywhere) — confirmed via git history it's the same underlying LVĢMC data as the FTP/open-data paths, just an earlier, superseded delivery mechanism that was never wired into anything running | Yes — Scala tests (2 suites now, down from 3, the deleted one was empty); VPS deployment pending |
|
||||||
|
|||||||
Reference in New Issue
Block a user