Record the missed FTP route and its fix in the roadmap changelog

Caught by an independent follow-up review: the FTP auth-bypass fix in
6b9c7cf only gated /api/fetch/lvgmc/stations, the one route the original
security review named. This sibling route calls the same fetch.fetchFile,
which opens a real, unauthenticated FTP login to LVGMC regardless of any
caller-supplied filename, and was only getting the traversal fix
(ValidateFileName) applied to it, not the auth-bypass fix. Traced every
caller of fetch.fetchFile/fetchWeatherStations in Server.scala/Main.scala
this time to confirm these are the only two HTTP-reachable call sites and
both are now gated.

docs/DEVELOPMENT_AND_STAGING.md

@@ -116,7 +116,9 @@ docker compose run --rm --no-deps node npm audit --omit=dev
 
 At commit `de6f279`, all five checks pass and both audit scopes report zero known vulnerabilities. The npm major-version availability notice is informational and does not require changing the npm version independently of the pinned Node build image.
 
-Known limitation: direct browser refreshes on newer client-side routes such as `/faktiska` can return 404 because the backend static-route list does not yet provide a general SPA fallback. Navigate from the home page until that backend behavior is fixed.
+Fixed 2026-08-24 (`2c44888`): direct browser refreshes on any client-side route now correctly serve the app via a general SPA fallback, replacing the old hardcoded per-route static-file list.
+
+Fixed 2026-08-24 (`6b9c7cf`, `8c45d8d`): a fresh-eyes security review found a live-verified CRITICAL path-traversal vulnerability and several HIGH/MEDIUM findings (unauthenticated FTP-trigger route, SQL injection via an unvalidated `field` path segment, unbounded-allocation DoS, `/api/*` falling through to the SPA shell) — all fixed and live-verified on Rocky; see the roadmap changelog for details. Rocky's `POSTGRES_PASSWORD` was rotated afterward via `ALTER ROLE` against the running container, since the old value had been exposed into the review's output.
 
 ## Authenticated API requests
 

docs/README.md

@@ -6,23 +6,25 @@ This directory contains the working documentation for the WeatherTool modernizat
 
 - Windows is restricted to source editing, review, and Git operations. Rocky is the sole compile, build, development-runtime, and test environment. The Ubuntu VPS is a deployment target only.
 - A production-like staging copy runs through Docker Compose on Rocky Linux at `http://192.168.1.101:9190`.
-- Rocky now ingests real data from two independent LVĢMC sources at once: the free `data.gov.lv` open-data feed and the private FTP feed (`ftp.meteo.lv`, real credentials obtained and verified 2026-08-23). The VPS currently runs open-data only — FTP was briefly enabled there too but reverted the same night after the `ltv` account failed to authenticate from the VPS specifically (see below). The two providers are gated independently (`ENABLE_LVGMC_FTP_JOBS`, `ENABLE_HARMONIE_JOBS`) since real credentials for FTP and DMI HARMONIE arrive on different timelines. HARMONIE's credentials turned out not to be needed at all (DMI dropped its API key requirement — confirmed live, see the roadmap changelog) but wiring it up is deferred to a dedicated verification session, since the GRIB-parsing and map-rendering code has real, untested risk (a hardcoded crop/rotation calibration that may not match the current model grid).
+- Both Rocky and the VPS currently run on the free `data.gov.lv` open-data feed only. Real private FTP credentials (`ftp.meteo.lv`) were obtained 2026-08-23 and the code path is proven working (manual tests succeeded repeatedly, including once from Rocky as recently as 2026-08-24 morning), but enabling the *scheduled* fetch has failed unpredictably on both machines at different times, for reasons not yet understood — see the roadmap Phase 7 for the full timeline. `ENABLE_LVGMC_FTP_JOBS` stays `false` everywhere until the LVGMC contact can explain the account's actual connection/rate policy. DMI HARMONIE's credentials turned out not to be needed at all (DMI dropped its API key requirement — confirmed live, see the roadmap changelog) but wiring it up is deferred to a dedicated verification session, since the GRIB-parsing and map-rendering code has real, untested risk (a hardcoded crop/rotation calibration that may not match the current model grid).
 - A real UTC-vs-local timezone mismatch between the two sources (open-data's timestamps were UTC, FTP's already local, both stored in the same column with no conversion) was found and fixed before enabling both together; both `weather` tables were backed up and wiped for a clean, consistently-timestamped restart.
-- The safe scheduled jobs (open-data station ingestion, GRIB cleanup) always run; the FTP job now runs too since real credentials exist; HARMONIE's job stays off pending its own implementation work.
+- The safe scheduled jobs (open-data station ingestion, GRIB cleanup) always run; the FTP and HARMONIE jobs both stay off — FTP pending the LVGMC answer above, HARMONIE pending its own implementation/verification work.
 - PostgreSQL is private to the project Compose network; only the Scala application publishes a host port.
 - The operator-facing workspaces now use the Latvian workflow names **Stacijas**, **Kartes**, **Faktiskā**, **Ūdens**, **Brīdinājumi**, **Apskats**, **Arhīvs**, **Harmonie**, and **LVĢMC**. Kartes retains custom analytical map outputs, while Faktiskā is a fixed 13-position, latest-temperature newsroom workflow with a locked 3840×1440 export.
 - Faktiskā symbol placement is automatic after manual image selection and is anchored to each rendered temperature badge.
 - **Ūdens** auto-populates its six ranges on load with real per-zone water-temperature min/max (65 LVĢMC stations classified into the 6 named zones), with manual override and reset still available. Uses separate authoritative 1920×1080 and 3840×1440 production templates; both exports have been visually validated.
 - **Brīdinājumi** renders current LVĢMC warning polygons over a production border overlay with feathered severity fills, plus draggable/resizable per-warning weather-symbol placement. Its lon/lat-to-pixel projection is an affine fit calibrated against the same validated city pixel positions Kartes/Faktiskā already use, replacing an earlier bounding-box calibration that drifted up to ~200px on the 3840 canvas.
 - Confirmed local Monda Regular/Bold files provide interface and generated-graphic typography; weather symbols use normalized transparent image assets.
-- Release `3eddf95` is deployed as immutable image `weathertool:3eddf95008b103f34d50dbc86c5634d3c0fa3523`; release `e446f1ff` remains the immediate application rollback. On top of the earlier frontend design pass (visual palette rework, Faktiskā decluttering, header nav, weather-icon CSS consolidation), this release fixes the UTC-vs-local timezone mismatch described above, hardens the `weather` table upsert to be non-destructive across sources (`COALESCE` instead of a blind overwrite), and splits the scheduler flag so FTP and HARMONIE can be enabled independently. `deploy/vps/compose.yml` was also fixed to actually read `LVGMC_*` from `.env.staging` (it was hardcoding inert placeholders directly in the file before, so real credentials alone wouldn't have taken effect) — real credentials and `ENABLE_LVGMC_FTP_JOBS=true` are live on the VPS as of 2026-08-23.
+- Release `b0b58d2` is deployed as immutable image `weathertool:b0b58d2f1e0ed47ca13795b64595386ec2f0e0c7`; release `3eddf95` remains the immediate application rollback. On top of the earlier frontend design pass and the timezone/upsert/scheduler-split fixes, this release replaces the hardcoded per-route static-file list with a general SPA fallback — a real, user-reported bug where refreshing `/faktiska` or `/udens-temperatura` 404ed instead of loading the app (those two routes were never added to the old list). A missing `/assets` file still 404s properly rather than silently serving HTML.
+- An independent fresh-eyes security review (requested against the local Rocky version only) found a live-verified CRITICAL path-traversal vulnerability plus several HIGH/MEDIUM findings — see the roadmap changelog for `6b9c7cf`/`8c45d8d` for the full list. All are fixed, compiled, live-verified on Rocky, and deployed to the VPS as release `0be325f` (image `weathertool:0be325fbbbf92b03bc8d574dc6f7b9449ef310ea`); release `b0b58d2` remains the immediate application rollback. Rocky's `POSTGRES_PASSWORD` was rotated afterward since the old value had been exposed into the review's own output via `/proc/self/environ`. `LVGMC_PASSWORD` rotation is not yet done — that needs the user's own action via the LVGMC contact.
+- FTP (`ENABLE_LVGMC_FTP_JOBS`) is currently `false` on both Rocky and the VPS — re-testing the morning after enabling it turned up a second, unexplained failure (Rocky alone, VPS confirmed off, ~4 minutes after a successful manual test) that doesn't fit the original two-machine-collision theory. Decided to stop self-testing via trial and error and wait for the user to ask the LVGMC contact directly about the `ltv` account's connection/rate policy, rather than risk repeatedly tripping an unknown limit. See the roadmap Phase 7 for the full diagnosis timeline.
 - The isolated VPS UAT stack is running and healthy: WeatherTool is bound to `127.0.0.1:8002`, Authelia to `127.0.0.1:9091`, and PostgreSQL has no host port. Public access is routed through Cloudflare, Nginx, and Authelia.
 - Cloudflare delegation is active, strict origin TLS covers only `laikapstak.li` and `auth.laikapstak.li`, and the public Nginx/Authelia login flow is operational without changing the existing HOP site.
 - The VPS `weather` table was wiped twice on 2026-08-23: first to remove the original 14-day synthetic dataset (backed up to `/srv/weathertool/backups/pre-real-data-release/`), then again after the UTC/local timezone fix (backed up to `/srv/weathertool/backups/pre-timezone-fix-wipe/`) so the two real sources it now holds — open-data (minutes 15/45) and FTP (minutes 11/13/23/30) — are consistently timestamped from a clean start.
 - Approved 1920×1080 and 3840×1440 PNG production bases are now the rendering source for Faktiskā and Ūdens temperatūra; code draws only the changing values, selected weather symbols, and wind data over those fixed newsroom graphics.
 - Browser branding assets and Latvian Open Graph/Twitter metadata are included for favicon, Apple home-screen icon, and link-preview support. Public crawler access still depends on the Nginx/Authelia policy used for the metadata and preview image.
 - Browser and API verification is complete for the deployed `3eddf95` release: exact release image smoke-tested on Rocky before transfer (bundle hash and correct-local-time API responses matched the known-good local build), checksum verified on both ends, container health/loopback/public HTTPS confirmed.
-- **VPS FTP is currently disabled again** (`ENABLE_LVGMC_FTP_JOBS=false`), despite being enabled earlier tonight. Watching for its first scheduled fetch (rather than assuming it was safe just because Rocky worked) caught a real problem: the `ltv` account fails to authenticate from the VPS specifically, and because a scheduled-task failure cascades through `parMapN` up to the app's top-level error handler, this was actively crash-looping the entire app in-process (not a Docker-level restart, but the whole app — DB, HTTP server, schedulers — tearing down and rebuilding every ~5s, repeating every ~2 minutes as the schedule re-fired). Reverted immediately; confirmed stable afterward. VPS network connectivity to `ftp.meteo.lv:21` itself is fine (verified with a raw connection test); this is specifically the `ltv` account failing from the VPS's IP, not a firewall issue — see the roadmap changelog for the full diagnosis and next steps (check the password transcription, or ask LVGMC whether the account is IP-restricted). FTP stays enabled on Rocky, where it's confirmed working.
+- Every FTP failure so far has cascaded through `parMapN` into the app's top-level error handler, crash-looping the *entire* app in-process (DB, HTTP server, schedulers all torn down and rebuilt every ~5s) rather than just failing the one scheduled task — caught twice by watching logs after enabling the flag rather than assuming it was safe, reverted both times within minutes.
 - Frontend dependency maintenance is complete: Solid runtime and Vite tooling were updated, obsolete packages were removed, TypeScript checking was added, and a clean Rocky `npm ci`, typecheck, production build, full audit, and production-only audit all pass with zero known vulnerabilities.
 - This is not yet approved or hardened for workplace production.
 

docs/UPDATE_ROADMAP.md

@@ -7,7 +7,7 @@ This document tracks proposed WeatherTool improvements. Work should be delivered
 - **Windows source workspace:** source editing, review, and Git operations only; do not install dependencies, compile, build, run, or test here.
 - **Rocky development and staging:** the sole compile, build, development-runtime, and test environment, with production-like Docker staging at `http://192.168.1.101:9190`.
 - **Git over SSH:** Windows pushes reviewed commits to a private bare repository on Rocky; the Rocky staging checkout pulls those commits and rebuilds.
-- **Ubuntu VPS deployment:** release `3eddf95` is publicly operational behind Cloudflare strict TLS, Nginx, and Authelia, ingesting real LVĢMC open-data station/water-temperature observations on a schedule; the `weather` table was wiped and re-populated fresh once the UTC/local timezone mismatch between sources was fixed. FTP (`ENABLE_LVGMC_FTP_JOBS`) was briefly enabled 2026-08-23 but reverted to `false` the same night after the `ltv` account failed to authenticate from the VPS specifically (crash-looped the app in-process until caught and reverted) — real credentials work fine from Rocky, not yet from the VPS; see the changelog for full diagnosis. The VPS does not compile or build the project.
+- **Ubuntu VPS deployment:** release `b0b58d2` is publicly operational behind Cloudflare strict TLS, Nginx, and Authelia, ingesting real LVĢMC open-data station/water-temperature observations on a schedule; the `weather` table was wiped and re-populated fresh once the UTC/local timezone mismatch between sources was fixed. FTP (`ENABLE_LVGMC_FTP_JOBS`) stays `false` on both Rocky and the VPS pending a real answer from the LVGMC contact about the `ltv` account's connection/rate policy — see Phase 7 for the full, still-unresolved diagnosis. The VPS does not compile or build the project.
 - **Workplace production:** remains separate until changes are reviewed, tested, and explicitly approved for workplace use.
 
 Do not synchronize `.env`, database directories, generated dependencies, build output, or provider credentials between machines.
@@ -88,12 +88,12 @@ Dependency changes must not be combined with a visual redesign unless a package
 Status: pending
 
 - [ ] Rotate and remove the API key exposed in a source comment (`src/main/scala/fetch/dmi/FetchService.scala:56`, a DMI HARMONIE key introduced 2025-02-01; confirmed the repo's GitHub `origin` is public, so this has been externally exposed for roughly 18 months — decision on rotation vs. a git-history rewrite is still open).
-- [ ] Remove credentials from connection-error messages.
+- [x] Remove credentials/internal error details from connection-error messages: `/api/fetch/lvgmc/stations` was returning `error.getMessage` straight to the caller; now logs it server-side and returns a generic message.
-- [ ] Protect or remove debug and administrative endpoints.
+- [ ] Protect or remove debug and administrative endpoints. Partial: `/api/fetch/lvgmc/stations` (the one that could trigger a real FTP login) is now gated behind `ENABLE_LVGMC_FTP_JOBS`, but `/api/debug/time`, `/api/debug/folder-structure`, and `/api/debug/delete-tmp` remain unauthenticated.
 - [ ] Convert state-changing `GET` routes to appropriate methods.
-- [ ] Introduce closed, validated weather-field and aggregation types.
+- [ ] Introduce closed, validated weather-field and aggregation types. Partial: the one route that let a raw field string reach SQL now validates against `WeatherData.getKeys` (see below), but this is a route-level allowlist, not a closed type threaded through the codebase.
-- [ ] Eliminate raw user-controlled SQL identifiers.
+- [x] Eliminate raw user-controlled SQL identifiers: `/api/query/city/.../{field}/...` let an arbitrary path segment reach `Fragment.const` unescaped; added `ValidateField`, the same allowlist-against-`WeatherData.getKeys` pattern `/query/country` already used.
-- [ ] Validate and constrain filenames, resolved paths, offsets, and byte lengths.
+- [x] Validate and constrain filenames, resolved paths, offsets, and byte lengths: added `ValidateFileName` (allowlist regex, rejects `..` and separators including decoded `%2f`) and applied it to every route that concatenated a raw path segment into a filesystem or FTP path (`/debug/file`, `/show/grib`, `/grib/binary-chunk`, `/show/lvgmc-forecast`); hardened `ValidateInt` to reject negative integers; capped `binary-chunk` read length at 64MB to remove an unbounded allocation.
 - [ ] Add query-range, response-size, request-rate, and timeout limits.
 - [ ] Restrict CORS to intended origins.
 - [ ] Define authentication and authorization requirements for workplace deployment.
@@ -272,13 +272,24 @@ running in parallel until each real source is proven, not cut over in one step.
          IP isn't on that list even though Rocky's apparently is. The VPS's
          outbound IP is `57.128.250.38`, in case it needs to be given to the
          LVGMC contact for allowlisting.
-      **Next step (morning, user): double-check the password in
+      **2026-08-24 morning update: the two-machine-collision theory above
-      `/srv/weathertool/.env.staging` on the VPS character-for-character
+      does not fully hold.** Independently confirmed the `ltv` account
-      against the decoded value; if it's correct, ask the LVGMC contact
+      itself isn't hard-locked (a FileZilla login from a separate PC on the
-      whether the `ltv` account is IP-restricted and whether the VPS's
+      same network succeeded, browsed `/ltv/tabulas`, saw
-      outbound IP needs adding.** FTP stays on for Rocky (working there);
+      `Latvija_faktiskais_laiks.csv` freshly modified — LVGMC's feed is
-      VPS runs open-data only until this is resolved and re-verified the
+      alive). A manual isolated fetch from Rocky then succeeded too. But
-      same careful way — watch-and-confirm, not assume-and-move-on.
+      re-enabling Rocky's *scheduler* (`ENABLE_LVGMC_FTP_JOBS=true`, VPS
+      confirmed still `false` the whole time) failed on its very next
+      scheduled attempt, ~4 minutes after the successful manual test, on
+      Rocky alone with nothing else touching the account. Three attempts in
+      ~10 minutes, two succeeded and one didn't, with no pattern found yet
+      (not simultaneity, not which machine, not obviously attempt spacing).
+      Reverted `ENABLE_LVGMC_FTP_JOBS` to `false` on Rocky again immediately.
+      **Decision: stop self-testing this via trial and error — every
+      attempt might be feeding an unknown rate/session limit — and wait for
+      the user to ask the LVGMC contact directly what the account's
+      connection policy actually is.** Both Rocky and the VPS run open-data
+      only until there's a real answer.
 - [x] Researched DMI HARMONIE credentials while waiting on the above and
       confirmed live (not just from search results, which claimed a specific
       date that wasn't independently verified): DMI's Forecast Data EDR and
@@ -369,9 +380,8 @@ Status: in progress
 - Both the VPS and Rocky `weather` tables now hold only real open-data station observations; their original synthetic rows were backed up and wiped 2026-08-23.
 - LVGMC forecast CSV fixtures are not yet available.
 - HARMONIE GRIB fixtures are not yet available.
-- The private LVGMC FTP feed has real credentials as of 2026-08-23 and is enabled on Rocky (`ENABLE_LVGMC_FTP_JOBS=true`); the DMI HARMONIE forecast feed remains gated (`ENABLE_HARMONIE_JOBS`, default off) pending real credentials.
+- The private LVGMC FTP feed has real credentials as of 2026-08-23, but `ENABLE_LVGMC_FTP_JOBS` stays `false` on both Rocky and the VPS pending a real answer from LVGMC about the `ltv` account's connection/rate policy (see Phase 7). The DMI HARMONIE forecast feed remains gated (`ENABLE_HARMONIE_JOBS`, default off) pending real credentials — turned out not to need any (DMI dropped its key requirement), but wiring it up is deferred to a dedicated verification session (see Phase 7).
 - Existing automated test coverage is minimal.
-- Direct refreshes on newer frontend routes can return 404 until the backend gains a general SPA fallback.
 - Full Docker build context scanning on Rocky can fail on the container-owned `postgres/` bind directory; do not loosen its permissions.
 
 ## Change log
@@ -431,3 +441,9 @@ Record completed work here by date and commit after the Git workflow is establis
 | 2026-08-23 | `3315f00` | Fix the same UTC-vs-local mismatch in Ūdens's `observedAt` display (`WaterTemperatureService`) — same open-data portal, same root cause, display-only (internal recency filtering was already self-consistent either way) | Yes — Scala tests; verified live via `/api/water-temperatures`: `observedAt` now matches real local time |
 | 2026-08-23 | `3eddf95` | Split `ENABLE_LEGACY_PROVIDER_JOBS` into independent `ENABLE_LVGMC_FTP_JOBS`/`ENABLE_HARMONIE_JOBS` — real LVGMC FTP credentials arrived today, real DMI HARMONIE credentials haven't, and the combined flag would have enabled both together, crash-looping the app on HARMONIE's still-placeholder values via `parMapN` | Yes — Scala tests; caught before it could happen (the Grib job was ~15 min from its first scheduled run when noticed) and reverted within under a minute; re-verified after the fix that only "Fetch Weather Stations" scheduled, not "Fetch Grib" |
 | 2026-08-23 | `dc04f66` | Fix `deploy/vps/compose.yml` hardcoding `LVGMC_USER`/`PASSWORD`/`URL` to inert placeholder strings directly in the file (unlike `POSTGRES_*`, which already read from `.env.staging`) — real credentials added to `.env.staging` alone would have had no effect until this switched to the same `${VAR}` substitution pattern. Enabled `ENABLE_LVGMC_FTP_JOBS` on the VPS | Yes — `docker compose config` syntax valid; VPS `app` container came up healthy with real credentials loaded (would have crashed immediately on missing-config if not) |
+| 2026-08-24 | `6b9c7cf` | Fix path traversal, auth-bypass, and DoS findings from an independent fresh-eyes security review (requested by the user specifically to get a second, skeptical pass against the local Rocky version): a live-verified CRITICAL — `/api/debug/file/{fileName}` allowed unauthenticated arbitrary file read, confirmed by reading `/proc/self/environ` and leaking `LVGMC_PASSWORD`/`POSTGRES_PASSWORD` into the review's own output — plus the same raw-filename pattern in `/show/grib`, `/grib/binary-chunk`, and `/show/lvgmc-forecast`. Added `ValidateFileName` (allowlist regex; rejects `..` and separators including decoded `%2f`) and applied it everywhere a path segment reached a filesystem or FTP path; hardened `ValidateInt` to reject negatives; gated `/api/fetch/lvgmc/stations` behind `ENABLE_LVGMC_FTP_JOBS` (it could otherwise trigger a real, unauthenticated FTP login regardless of the flag) and stopped it leaking `error.getMessage`; added an explicit `/api/*` catch-all so an unmatched API route can never fall through to the SPA shell; capped `binary-chunk` read length at 64MB | Yes — Scala tests, Rocky rebuild, live curl verification of `%2f`-encoded and literal traversal payloads (404), the gated FTP route (503 while the flag is off), an unmatched `/api/*` route (404, not `index.html`), and legitimate filenames/queries still returning correct 200s |
+| 2026-08-24 | `8c45d8d` | Fix SQL injection surfaced by the same security review: `/api/query/city/.../{field}/...` passed its `field` path segment unvalidated into `PostgresService.query`, which splices it into SQL via `Fragment.const` (unescaped) whenever the aggregate key is min/max/avg/sum/distinct, or whenever granularity is `hour` in the list branch. Added `ValidateField`, reusing the `WeatherData.getKeys` allowlist `AggFieldList` already applied to `/query/country`. `AggregateKey` values reaching `Fragment.const` elsewhere were already safe — they come from a closed ADT (`AggregateKey.fromString`), not raw input | Yes — Scala tests, Rocky rebuild, live curl verification that SQL-injection payloads in the `field` segment 404 and a legitimate query still returns correct data |
+| 2026-08-24 | *(operational, no commit)* | Rotated Rocky's `POSTGRES_PASSWORD` following the security review, since the old value had been exposed into this session's context multiple times (two of my own sloppy shell commands, plus the `/proc/self/environ` read the review used as its traversal proof). Changed the role's actual password via `ALTER ROLE` against the running container (editing `.env` alone has no effect on an already-initialized PostgreSQL data directory), then updated `.env` and recreated the `scala` service. The Postgres container itself was also recreated as a side effect (its own `POSTGRES_PASSWORD` env interpolation changed too), which is harmless — that variable only takes effect on a fresh, empty data directory, not an existing one — but is worth knowing about | Yes — clean scala container startup with no auth errors, and a real query (`/api/query/latest-temperatures/Rīga`) returning live data over the new password |
+| 2026-08-24 | `0be325f` | Deploy the security-review fixes (`6b9c7cf`, `8c45d8d`) to the VPS as image `weathertool:0be325fbbbf92b03bc8d574dc6f7b9449ef310ea`; release `b0b58d2` remains the immediate application rollback. Application-only release — PostgreSQL and Authelia were not restarted | Yes — exact release image smoke-tested on Rocky against a throwaway local Postgres before transfer (traversal/injection payloads 404, gated FTP route 503, legitimate queries 200); checksum verified on both ends; `app` service recreated cleanly (healthy, no auth/DB errors in logs); loopback re-verification of the same traversal/injection/FTP-gate checks plus real `/api/warnings` and `/api/water-temperatures` responses; public HTTPS confirmed unchanged (302 unauthenticated page, 401 unauthenticated API) |
+| 2026-08-24 | `9bab93d` | A follow-up independent fresh-eyes review (requested specifically as a second pass, not a rerun of the same pentest) caught that the `6b9c7cf` FTP auth-bypass fix was incomplete: only `/api/fetch/lvgmc/stations` (the one route the original review named) was gated behind `ENABLE_LVGMC_FTP_JOBS`. A sibling route, `/api/show/lvgmc-forecast/{fileName}`, calls the same `fetch.fetchFile` — a real, unauthenticated LVGMC FTP login — and had only gotten the traversal fix (`ValidateFileName`), not the auth-bypass fix, because the two routes were fixed along different mental categories (traversal batch vs. the one named auth-bypass fix) instead of by tracing every caller of the dangerous method. Gated this route the same way; traced every HTTP-reachable caller of `fetch.fetchFile`/`fetchWeatherStations` this time to confirm no others remain. Deployed to both Rocky and the VPS (image `weathertool:9bab93daa6b3a0fb99165a8fac9accfa3a3ccbf1`) immediately given this was live and exploitable in production | Yes — Scala tests, Rocky rebuild and curl verification (both FTP routes 503 while the flag is off), VPS image smoke-tested against a throwaway local Postgres before transfer, checksum verified, `app` service recreated cleanly, loopback re-verification of both gated routes, public HTTPS unchanged (302/401) |
+| 2026-08-24 | `2c44888`–`b0b58d2` | Replace the hardcoded per-route static-file list (`/station`, `/cities`, `/latvia`, `/database`, `/harmonie`, `/lvgmc-forecast`, `/bridinajumi`) with a general SPA fallback: real files serve as-is, anything else falls back to `index.html` so the SolidJS router handles it client-side — fixes a real, user-reported bug where a direct hit (e.g. a browser refresh) on `/faktiska` or `/udens-temperatura` 404ed instead of loading the app, since those two routes were never added to the old list. A missing file under `/assets` specifically still 404s properly rather than silently serving HTML, so a stale tab after a future deploy gets a clean error instead of a confusing JS parse failure | Yes — first attempt silently no-op'd because the fix was built into a `git archive HEAD` image before being committed (classic mistake, caught immediately by re-testing and finding identical old behavior); after committing, verified via curl on all previously-working routes, both previously-broken routes, a missing asset (404), a real asset (200), and `/api` (200), then a full headless-browser render check (zero console errors, real data, correct nav state) on a genuine direct hit — not just HTTP status codes; deployed to VPS in release `b0b58d2`, verified the same way through both the loopback port and the public domain (Authelia gate still correctly redirects unauthenticated requests) |

src/main/scala/data/DataService.scala

@@ -81,7 +81,16 @@ class DataService(log: Logger[IO]) {
     Executors.newFixedThreadPool(8) // limit concurrency
   )
 
+  private val MAX_BINARY_CHUNK_BYTES = 64 * 1024 * 1024
+
   def getBinaryChunk(offset: Int, length: Int, fileName: String): IO[Array[Byte]] = {
+    if (length > MAX_BINARY_CHUNK_BYTES || length < 0)
+      IO.raiseError(new IllegalArgumentException(s"Requested length $length is invalid (max $MAX_BINARY_CHUNK_BYTES bytes)"))
+    else
+      getBinaryChunkUnchecked(offset, length, fileName)
+  }
+
+  private def getBinaryChunkUnchecked(offset: Int, length: Int, fileName: String): IO[Array[Byte]] = {
     val fileResource = Resource.make(
       IO.blocking(new RandomAccessFile(s"$GRIB_FOLDER/$fileName", "r"))
     )(file => IO.blocking(file.close()))

src/main/scala/server/Server.scala

@@ -2,6 +2,7 @@ package server
 
 import cats.effect._
 import cats.implicits.toTraverseOps
+import cats.syntax.semigroupk._
 import com.comcast.ip4s.IpLiteralSyntax
 import data.DataService
 import db.PostgresService
@@ -9,7 +10,7 @@ import fetch.csv.FileNameService
 import fetch.lvgmc.{FetchService, WaterTemperatureService}
 import fetch.warnings.WarningService
 import fs2.io.file.{Files, Path}
-import server.ValidateRoutes.{AggFieldList, AggKey, CityList, DateTimeRange, Granularity, ValidateDate, ValidateDateTime, ValidateInt, ValidateMonths, ValidateZonedDateTime}
+import server.ValidateRoutes.{AggFieldList, AggKey, CityList, DateTimeRange, Granularity, ValidateDate, ValidateDateTime, ValidateField, ValidateFileName, ValidateInt, ValidateMonths, ValidateZonedDateTime}
 import org.http4s._
 import org.http4s.dsl.io._
 import org.http4s.implicits._
@@ -63,24 +64,39 @@ class Server(postgresService: PostgresService, dataService: DataService, fetch:
         }
 
     // http://0.0.0.0:8080/api/show/lvgmc-forecast/Latvija_LTV_pilsetas_tekosa_dn.csv
-    case GET -> Root / "show" / "lvgmc-forecast" / fileName =>
+    // Gated behind the same flag as the scheduled FTP task and
-      fetch.fetchFile(fileName).flatMap(bytes =>
+    // /fetch/lvgmc/stations below: fetch.fetchFile opens a real FTP login to
-        Ok(bytes).map(_.withContentType(`Content-Type`(MediaType.text.csv)))
+    // LVGMC regardless of any caller-supplied filename, so an unauthenticated
-      )
+    // caller could otherwise trigger unlimited outbound FTP sessions no
+    // matter what ENABLE_LVGMC_FTP_JOBS says.
+    case GET -> Root / "show" / "lvgmc-forecast" / ValidateFileName(fileName) =>
+      if (sys.env.get("ENABLE_LVGMC_FTP_JOBS").exists(_.equalsIgnoreCase("true")))
+        fetch.fetchFile(fileName).flatMap(bytes =>
+          Ok(bytes).map(_.withContentType(`Content-Type`(MediaType.text.csv)))
+        )
+      else
+        ServiceUnavailable("LVGMC FTP fetching is currently disabled")
 
     // http://0.0.0.0:8080/api/fetch/lvgmc/stations
+    // Gated behind the same flag as the scheduled FTP task: this route would
+    // otherwise let anyone unauthenticated trigger a real LVGMC FTP login on
+    // demand, bypassing ENABLE_LVGMC_FTP_JOBS entirely.
     case GET -> Root / "fetch" / "lvgmc" / "stations" =>
-      (
+      if (sys.env.get("ENABLE_LVGMC_FTP_JOBS").exists(_.equalsIgnoreCase("true")))
-        for {
+        (
-          fileName <- new FileNameService().generateCurrentHour
+          for {
-          stationDataStr <- fetch.fetchWeatherStations()
+            fileName <- new FileNameService().generateCurrentHour
-          _ <- postgresService.save(fileName, stationDataStr)
+            stationDataStr <- fetch.fetchWeatherStations()
-        } yield stationDataStr
+            _ <- postgresService.save(fileName, stationDataStr)
-      )
+          } yield stationDataStr
-        .flatMap(content => Ok(content))
-        .handleErrorWith(error =>
-          InternalServerError(s"Failed to fetch stations: ${error.getMessage}")
         )
+          .flatMap(content => Ok(content))
+          .handleErrorWith(error =>
+            log.error(error)("Failed to fetch LVGMC stations") *>
+              InternalServerError("Failed to fetch stations")
+          )
+      else
+        ServiceUnavailable("LVGMC FTP fetching is currently disabled")
 
     // http://0.0.0.0:8080/api/show/grib-all-structure
     case GET -> Root / "show" / "grib-all-structure" =>
@@ -91,10 +107,10 @@ class Server(postgresService: PostgresService, dataService: DataService, fetch:
       dataService.getFileList().flatMap(fileList => Ok(fileList.asJson))
 
     // http://0.0.0.0:8080/api/show/grib-name/harmonie_2025-02-01T1500Z_2025-02-01T180000Z.grib
-    case GET -> Root / "show" / "grib" / fileName =>
+    case GET -> Root / "show" / "grib" / ValidateFileName(fileName) =>
       dataService.getGribStucture(fileName).flatMap(response => Ok(response.asJson))
 
-    case GET -> Root / "grib" / "binary-chunk" / ValidateInt(binaryOffset) / ValidateInt(binaryLength) / fileName =>
+    case GET -> Root / "grib" / "binary-chunk" / ValidateInt(binaryOffset) / ValidateInt(binaryLength) / ValidateFileName(fileName) =>
       dataService.getBinaryChunk(binaryOffset, binaryLength, fileName).flatMap(buffer => Ok(buffer))
 
     // http://0.0.0.0:8080/api/grib/delete-old-forecasts
@@ -116,7 +132,7 @@ class Server(postgresService: PostgresService, dataService: DataService, fetch:
       DebugUtils.getFolderStructure.flatMap(response => Ok(response.asJson))
 
     // http://0.0.0.0:8080/api/debug/file/error_2025-03-16_152201.txt
-    case GET -> Root / "debug" / "file" / fileName =>
+    case GET -> Root / "debug" / "file" / ValidateFileName(fileName) =>
       val filePath = Path(s"data/tmp/$fileName")
 
       Files[IO].exists(filePath).flatMap {
@@ -131,7 +147,7 @@ class Server(postgresService: PostgresService, dataService: DataService, fetch:
       }
 
     // http://0.0.0.0:8080/api/query/city/Liepāja,Rēzekne/20230414_2200-20230501_1230/hour/tempMax/max
-    case GET -> Root / "query" / "city" / CityList(cities) / DateTimeRange(from, to) / Granularity(granularity) / field / AggKey(key) =>
+    case GET -> Root / "query" / "city" / CityList(cities) / DateTimeRange(from, to) / Granularity(granularity) / ValidateField(field) / AggKey(key) =>
       val userQuery = UserQuery(cities, field, key, granularity, from, to)
 
       postgresService.query(userQuery)
@@ -192,6 +208,12 @@ class Server(postgresService: PostgresService, dataService: DataService, fetch:
     // http://0.0.0.0:8080/api/show/datetime/20230423_1300
     case GET -> Root / "show" / "datetime" / ValidateDateTime(datetime) =>
       postgresService.getDateTimeEntries(datetime).flatMap(content => Ok(content.asJson))
+
+    // Explicit catch-all: guarantees every /api/* request resolves inside
+    // this route set (never falls through to the SPA fallback below and
+    // gets served index.html as a false 200).
+    case _ =>
+      NotFound()
   }
 
   private val corsConfig = CORSConfig.default
@@ -203,18 +225,25 @@ class Server(postgresService: PostgresService, dataService: DataService, fetch:
 
   private val apiRoutesCors = CORS(apiRoutes, corsConfig)
 
-  private val httpApp = Router(
+  // Real files (JS/CSS/images/favicon) are served as-is; anything else falls
-    "/" -> staticcontent.fileService[IO](FileService.Config("./web/dist")),
+  // back to index.html so the SolidJS router can handle it client-side. This
-    "/api" -> apiRoutesCors,
+  // replaces an explicit per-route list that silently 404ed on a direct hit
+  // (e.g. a browser refresh) for any client-side route not on the list —
+  // real, observed on /faktiska and /udens-temperatura in production.
+  private val assets = staticcontent.fileService[IO](FileService.Config("./web/dist"))
+  private val spaFallback = HttpRoutes.of[IO] {
+    // A missing file under /assets (the only place Vite emits hashed build
+    // output) should stay a real 404, not silently serve HTML — otherwise a
+    // stale tab referencing a since-removed bundle after a future deploy
+    // would get a confusing "unexpected token" JS parse error instead.
+    case GET -> Root / "assets" / _ => NotFound()
+    case req @ GET -> _ =>
+      StaticFile.fromPath(Path("./web/dist/index.html"), Some(req)).getOrElseF(NotFound())
+  }
 
-    // TODO rewrite in more generic way
+  private val httpApp = Router(
-    "/station" -> staticcontent.fileService[IO](FileService.Config("./web/dist")),
+    "/api" -> apiRoutesCors,
-    "/cities" -> staticcontent.fileService[IO](FileService.Config("./web/dist")),
+    "/" -> (assets <+> spaFallback),
-    "/latvia" -> staticcontent.fileService[IO](FileService.Config("./web/dist")),
-    "/database" -> staticcontent.fileService[IO](FileService.Config("./web/dist")),
-    "/harmonie" -> staticcontent.fileService[IO](FileService.Config("./web/dist")),
-    "/lvgmc-forecast" -> staticcontent.fileService[IO](FileService.Config("./web/dist")),
-    "/bridinajumi" -> staticcontent.fileService[IO](FileService.Config("./web/dist")),
   ).orNotFound
 
   def run: IO[ExitCode] =

src/main/scala/server/ValidateRoutes.scala

@@ -64,6 +64,14 @@ object ValidateRoutes {
     }
   }
 
+  // Restricts a raw path segment to a known WeatherData field name before it
+  // can reach Fragment.const (unescaped SQL splicing) in PostgresService.
+  object ValidateField {
+    def unapply(str: String): Option[String] = {
+      if (WeatherData.getKeys.contains(str)) Some(str) else None
+    }
+  }
+
   object AggFieldList {
     def unapply(str: String): Option[NonEmptyList[String]] = {
       val weatherFields = WeatherData.getKeys
@@ -92,7 +100,19 @@ object ValidateRoutes {
 
   object ValidateInt {
     def unapply(str: String): Option[Int] = {
-      Option(str.toInt)
+      Try(str.toInt).toOption.filter(_ >= 0)
+    }
+  }
+
+  // Rejects path traversal and directory separators (including URL-decoded
+  // %2f, which arrives here as a literal '/' after http4s decodes the path
+  // segment) by allowlisting a safe filename character set, rather than
+  // trying to blocklist every encoding of "..". Used anywhere a path
+  // segment gets concatenated directly into a filesystem Path.
+  object ValidateFileName {
+    private val safeFileName = "^[A-Za-z0-9._-]+$".r
+    def unapply(str: String): Option[String] = {
+      if (str.nonEmpty && !str.contains("..") && safeFileName.matches(str)) Some(str) else None
     }
   }
 }