Split ENABLE_LEGACY_PROVIDER_JOBS into independent FTP/HARMONIE flags
Real LVGMC FTP credentials arrived today; real DMI HARMONIE credentials haven't. The combined flag would have enabled both the moment FTP's were ready, crash-looping the app on HARMONIE's still- placeholder values via parMapN — caught this before it happened (the Grib job was ~15 min from its first scheduled run). Split into ENABLE_LVGMC_FTP_JOBS and ENABLE_HARMONIE_JOBS so each provider can be enabled independently as its own credentials become real.
This commit is contained in:
@@ -26,7 +26,7 @@ DMI HARMONIE APIs (legacy, gated off) --------------+
|
||||
Canvas preview and PNG export
|
||||
```
|
||||
|
||||
Staging exercises the PostgreSQL → API → frontend path with real LVĢMC open-data observations (station temperatures, water temperatures, warnings) as of 2026-08-23. The private FTP station feed and the DMI HARMONIE forecast feed still exist in code but are gated behind `ENABLE_LEGACY_PROVIDER_JOBS` (default off) pending real credentials — see `docs/UPDATE_ROADMAP.md` Phase 7.
|
||||
Staging exercises the PostgreSQL → API → frontend path with real LVĢMC open-data observations (station temperatures, water temperatures, warnings) as of 2026-08-23. The private FTP station feed is real and enabled on Rocky as of 2026-08-23 (`ENABLE_LVGMC_FTP_JOBS=true`); the DMI HARMONIE forecast feed still exists in code but stays gated (`ENABLE_HARMONIE_JOBS`, default off) pending real credentials — see `docs/UPDATE_ROADMAP.md` Phase 7.
|
||||
|
||||
## Components
|
||||
|
||||
@@ -37,7 +37,7 @@ Staging exercises the PostgreSQL → API → frontend path with real LVĢMC open
|
||||
- Contains ingestion/parsing code for LVGMC station data: a free/keyless open-data path (`fetch.lvgmc.OpenDataStationService`) that pivots LVĢMC's public `data.gov.lv` station-observation feed into the `weather` table, and the original private FTP feed (`fetch.lvgmc.FetchService`), kept but gated off pending real credentials.
|
||||
- Fetches LVĢMC open hydrometeorological-warning metadata and polygon coordinates on request (`fetch.warnings.WarningService`), and water temperatures per named zone (`fetch.lvgmc.WaterTemperatureService`) — both free/keyless `data.gov.lv` reads with a short in-memory cache and stale-data fallback if the upstream call fails; neither persists to PostgreSQL.
|
||||
- Contains HARMONIE discovery, download, GRIB parsing, and rendering support, gated off pending real DMI credentials.
|
||||
- Starts the safe scheduled ingestion/cleanup tasks (open-data stations, GRIB cleanup) unless `ENABLE_SCHEDULED_JOBS=false`. The legacy FTP station fetch and HARMONIE fetch are scheduled independently and only run if `ENABLE_LEGACY_PROVIDER_JOBS=true` — kept separate so placeholder credentials on the legacy path can't crash-loop the whole app (both paths use `cats.effect.IO`'s `parMapN`, which cancels and fails every sibling task the instant one throws).
|
||||
- Starts the safe scheduled ingestion/cleanup tasks (open-data stations, GRIB cleanup) unless `ENABLE_SCHEDULED_JOBS=false`. The legacy FTP station fetch and HARMONIE fetch are each gated by their own flag (`ENABLE_LVGMC_FTP_JOBS`, `ENABLE_HARMONIE_JOBS`) rather than one combined flag — real credentials for the two providers arrive on different timelines, and a single flag would enable both together the moment either one's were ready, crash-looping the app on whichever provider still had placeholders (both paths use `cats.effect.IO`'s `parMapN`, which cancels and fails every sibling task the instant one throws).
|
||||
|
||||
### SolidJS frontend
|
||||
|
||||
@@ -54,7 +54,7 @@ Staging exercises the PostgreSQL → API → frontend path with real LVĢMC open
|
||||
|
||||
## Main observation-data flow
|
||||
|
||||
1. A scheduled ingestion job parses provider station data (the open-data path by default; the legacy FTP path only if `ENABLE_LEGACY_PROVIDER_JOBS=true`).
|
||||
1. A scheduled ingestion job parses provider station data (the open-data path always; the FTP path too if `ENABLE_LVGMC_FTP_JOBS=true`).
|
||||
2. Observations are stored in the `weather` table.
|
||||
3. The frontend requests an API route containing cities, time range, granularity, field, and aggregate key.
|
||||
4. The backend performs the database query and returns JSON.
|
||||
|
||||
Reference in New Issue
Block a user