Files
WeatherTool/docs/UPDATE_ROADMAP.md
T
b0txec cb7e9aa06b
CI / backend (push) Successful in 1m3s
CI / frontend (push) Successful in 40s
Record the Faktiskā blank-temperature fix and correct a runbook bug
Documents release a6b2b84 (the queryLatestTemperatures fix and the user's
CSS polish pass) in the changelog and README status line. Also fixes a real
bug in VPS_RELEASE_RUNBOOK.md's own checksum step, found by actually running
it for this release: it generated the .sha256 file against a full local
path rather than a bare filename, which fails verification on the VPS since
that path doesn't exist there.
2026-08-25 17:19:17 +03:00

72 KiB
Raw Blame History

WeatherTool update roadmap

This document tracks proposed WeatherTool improvements. Work should be delivered in small, reviewable phases rather than as one large rewrite. Each phase should leave the application runnable and independently testable.

Environments and workflow

  • Windows source workspace: source editing, review, and Git operations only; do not install dependencies, compile, build, run, or test here.
  • Rocky development and staging: the sole compile, build, development-runtime, and test environment, with production-like Docker staging at http://192.168.1.101:9190.
  • Git over SSH: Windows pushes reviewed commits to a private bare repository on Rocky; the Rocky staging checkout pulls those commits and rebuilds.
  • Ubuntu VPS deployment: release b0b58d2 is publicly operational behind Cloudflare strict TLS, Nginx, and Authelia, ingesting real LVĢMC open-data station/water-temperature observations on a schedule; the weather table was wiped and re-populated fresh once the UTC/local timezone mismatch between sources was fixed. FTP (ENABLE_LVGMC_FTP_JOBS) stays false on both Rocky and the VPS pending a real answer from the LVGMC contact about the ltv account's connection/rate policy — see Phase 7 for the full, still-unresolved diagnosis. The VPS does not compile or build the project.
  • Workplace production: remains separate until changes are reviewed, tested, and explicitly approved for workplace use.

Do not synchronize .env, database directories, generated dependencies, build output, or provider credentials between machines.

Working principles

  1. Make one coherent change at a time.
  2. Record the existing behavior before intentionally changing it.
  3. Keep dependency updates separate from UI redesign and functional changes.
  4. Review and commit on Windows, then deploy and test the same commit on Rocky staging.
  5. Use synthetic or sanitized data outside the workplace environment.
  6. Never enable external provider schedules with placeholder credentials.
  7. Do not connect staging to workplace services without explicit authorization.

Phase 0 — Reproducible development baseline

Status: in progress

  • Review backend, frontend, deployment, and security structure.
  • Run the project locally through Docker Desktop.
  • Add configurable host port and scheduled-job switch.
  • Add deterministic synthetic station data for the repository station set and the Faktiskā-required Valmiera position.
  • Create an isolated Rocky Linux staging deployment.
  • Keep staging PostgreSQL private to its Compose network.
  • Commit the baseline changes and establish the Git-over-SSH workflow.
  • Document normal build, seed, frontend deploy, source rollback, and current operational limitations. (Database backup/restore remains pending.)
  • Capture representative screenshots and expected API responses.

Phase 1 — Behavior discovery and bug inventory

Status: in progress

  • Walk through every page with synthetic data.
  • Document the current understood purpose of each workflow; validate it with workplace users over time.
  • Separate analytical dashboard features from broadcast-graphic authoring tools on the city-results map.
  • Record unclear controls, missing units, broken states, and layout problems.
  • Fix the atmPressire frontend field typo.
  • Fix shifted Database export columns caused by duplicated tempMax.
  • Correct malformed integer route handling.
  • Add consistent loading, empty, and error states. (City selection empty state completed.)

Phase 2 — Test safety net

Status: in progress

  • Add backend route tests for representative station and country queries.
  • Add database integration tests for aggregation and export behavior.
  • Restore and expand CSV parser tests.
  • Add GRIB parser boundary and malformed-file tests.
  • Add security tests for invalid fields, filenames, offsets, lengths, and date ranges. Highest-priority target per an independent review: ValidateFileName/ValidateField/ValidateInt are pure functions with zero I/O and are currently the entire path-traversal/SQL-injection security boundary, verified only by manual curl.
  • PostgresService.query's "list" branch (byField match, ~13 hardcoded literal cases, no case _ =>) is only safe today because ValidateField's allowlist and WeatherData's case class fields happen to stay in sync with it — nothing enforces that. Add a field to WeatherData without updating this match and any request for it with granularity=hour throws an uncaught MatchError (a real risk given the DMI/open-data provider work in progress). A test iterating every WeatherData.getKeys value through this path would catch it before it ships; needs either a test-container Postgres or refactoring SQL-fragment-building apart from execution so it's testable without a DB.
  • Add frontend type checking (npm run typecheck, part of the dependency-maintenance work in Phase 3) — now also runs automatically in CI (see below). Critical workflow smoke tests remain manual/headless-browser only, not automated.
  • Run tests automatically before staging deployment. Partial: Gitea Actions CI (see CONTINUOUS_INTEGRATION.md) now runs sbt test and the frontend typecheck/build/audit routine automatically on every push/PR to codex/staging-baseline — this is CI, not CD; it isn't yet wired as a required gate before Rocky/VPS deployment, which remains a manual decision independent of CI status.

Phase 3 — Dependency modernization

Status: in progress — frontend maintenance complete; backend maintenance pending

The first observed frontend install reported 15 vulnerabilities: 1 critical, 10 high, 3 moderate, and 1 low. The frontend dependency tree was reviewed in controlled groups, obsolete packages were removed, and both the complete and production-only npm audits now report zero known vulnerabilities at commit de6f279.

  • Capture and review the full npm audit report.
  • Establish that the production dependency audit is primarily blocked by solid-js@1.9.4 resolving vulnerable seroval@1.2.0; confirm that the full audit also contains development-tool advisories.
  • Update direct frontend dependencies in controlled groups.
  • Replace or remove obsolete frontend packages where appropriate.
  • Add a committed TypeScript typecheck command and validate the updated frontend with a clean npm ci, typecheck, and production build.
  • Resolve the remaining transitive build-tool advisories after reviewing the proposed npm audit fix changes; verify full and production-only audits at zero.
  • Build, smoke-test, checksum, transfer, and deploy a commit-addressed VPS release containing de6f279 or later without restarting PostgreSQL or Authelia. (ef64895 deployed 2026-08-22.)
  • Visually compare every page and representative exported PNG in the deployed dependency-maintenance release.
  • Update Scala within the supported 2.13 line before considering larger migration.
  • Update http4s, Doobie, Circe, Cats Effect, Logback, and test libraries in compatible groups.
  • Replace release-candidate dependencies with stable releases where possible.
  • Update Docker base images deliberately and pin reproducible versions.
  • Verify database compatibility and generated artifacts after every group.

Dependency changes must not be combined with a visual redesign unless a package migration strictly requires it.

Phase 4 — Security hardening

Status: pending

  • Rotate and remove the API key exposed in a source comment (src/main/scala/fetch/dmi/FetchService.scala:56, a DMI HARMONIE key introduced 2025-02-01; confirmed the repo's GitHub origin is public, so this has been externally exposed for roughly 18 months — decision on rotation vs. a git-history rewrite is still open).
  • Remove credentials/internal error details from connection-error messages: /api/fetch/lvgmc/stations was returning error.getMessage straight to the caller; now logs it server-side and returns a generic message.
  • Protect or remove debug and administrative endpoints. Partial: /api/fetch/lvgmc/stations (the one that could trigger a real FTP login) is now gated behind ENABLE_LVGMC_FTP_JOBS, but /api/debug/time, /api/debug/folder-structure, and /api/debug/delete-tmp remain unauthenticated.
  • Convert state-changing GET routes to appropriate methods.
  • Introduce closed, validated weather-field and aggregation types. Partial: the one route that let a raw field string reach SQL now validates against WeatherData.getKeys (see below), but this is a route-level allowlist, not a closed type threaded through the codebase.
  • Eliminate raw user-controlled SQL identifiers: /api/query/city/.../{field}/... let an arbitrary path segment reach Fragment.const unescaped; added ValidateField, the same allowlist-against-WeatherData.getKeys pattern /query/country already used.
  • Validate and constrain filenames, resolved paths, offsets, and byte lengths: added ValidateFileName (allowlist regex, rejects .. and separators including decoded %2f) and applied it to every route that concatenated a raw path segment into a filesystem or FTP path (/debug/file, /show/grib, /grib/binary-chunk, /show/lvgmc-forecast); hardened ValidateInt to reject negative integers; capped binary-chunk read length at 64MB to remove an unbounded allocation.
  • Add query-range, response-size, request-rate, and timeout limits.
  • Restrict CORS to intended origins.
  • Define authentication and authorization requirements for workplace deployment.

Phase 5 — Runtime reliability and operations

Status: pending

  • Manage custom executors as resources and close them cleanly.
  • Remove explicit System.gc() calls. Confirmed still present in DataService.getBinaryChunk (runs on every GRIB binary-chunk request — likely a latency source on what's probably a hot path; a commented-out logMemory block next to it suggests leftover debugging code, not an intentional design choice), per a 2026-08-24 independent review.
  • Supervise scheduled jobs independently instead of recursively restarting the application. Confirmed still fully live by a 2026-08-24 independent review: Scheduler.scheduleTask has zero .attempt/error handling on any task, none of the task bodies wired into it (FTP fetch, open-data fetch, cleanup, GRIB fetch) handle their own errors either, and (serverTask, scheduledTasks).parMapN(...) means any one failure cancels the HTTP server itself — this is the same defect that caused the actual VPS/Rocky crash-loop incidents earlier this session, patched around with feature flags rather than closed. OpenDataStationService also has no client timeout configured, unlike the DMI fetch service. WarningService/WaterTemperatureService already show the correct pattern (.attempt + stale-cache fallback) the scheduler tasks should follow.
  • Add a pooled JDBC transactor. DBConnection.scala uses Transactor.fromDriverManager, which opens a brand-new physical connection for every query — doobie's documented behavior for scripts/tests, not production services. Not urgent at current traffic, but an easy, well-known swap (HikariTransactor) worth doing before real load.
  • Add application health and readiness endpoints.
  • Add structured logging without leaking secrets.
  • Define PostgreSQL and GRIB-data backup/restore procedures.
  • Add container resource limits and deployment health checks.
  • Document monitoring, update, rollback, and incident procedures.
  • Harden WarningService: bound LVĢMC retry attempts to once per cache window during an upstream outage, isolate one warning's polygon-fetch failure from the rest instead of failing the whole response, verify the CKAN polygon filter against real column-type ambiguity rather than guessing, and log (rather than silently swallow) a defaulted vertex-order value.

Phase 6 — Information architecture and UI redesign

Status: pending

  • Identify primary user roles and their most frequent tasks.
  • Separate historical analysis, live station monitoring, database inspection, HARMONIE visualization, and broadcast graphics.
  • Replace technical/internal labels with task-oriented language.
  • Replace the copy/paste weather-character workflow with a direct visual font-glyph picker, bulk assignment, and city exceptions.
  • Explain and visually separate manual wind and weather-icon inputs from queried data.
  • Add units, legends, contextual help, and clear date semantics.
  • Establish the first responsive layout, typography, spacing, and component-system foundation.
  • Add an explicit PNG download workflow for broadcast map assets.
  • Implement the fixed Faktiskā station set, latest-temperature loading, manual overrides, and locked 3840×1440 export.
  • Make Faktiskā weather-symbol selection manual while keeping placement automatic and badge-relative.
  • Replace Daira font glyphs with normalized 256×256 transparent image assets across the editor and canvas renderers.
  • Add a separate Latvian-named Ūdens workspace with fixed manual fields.
  • Calibrate and visually validate independent Ūdens exports at 1920×1080 and 3840×1440 against the supplied production templates.
  • Bundle the confirmed Monda Regular/Bold files and replace temporary Rubik rendering.
  • Perform pixel-level Monda comparison against the authoritative production masters.
  • Add a separate Latvian-named Brīdinājumi workspace that renders current LVĢMC warning polygons without manual tracing.
  • Add production border overlay artwork and feathered warning-polygon fills to Brīdinājumi.
  • Replace Brīdinājumi's 4-corner bounding-box projection with an affine fit calibrated against the same validated Kartes/Faktiskā city pixel positions.
  • Limit the Brīdinājumi canvas to one operator-selected warning (superseded below by multi-warning selection), remove the on-canvas legend, collapse card detail to phenomenon/severity by default with full detail on click, and pre-fill the title from the selected warning's phenomenon.
  • Redesign Brīdinājumi warning cards as compact phenomenon-icon chips and move full warning text into a dialog popup over a dimmed, blurred backdrop.
  • Give Brīdinājumi chips a hover state where the severity-colored icon capsule expands to fill the chip, adapted from a reviewed external Uiverse.io reference component.
  • Split Brīdinājumi's warning selection (checkbox, drives the map) from viewing a warning's detail (clicking the chip, opens the popup) so choosing a map warning no longer forces the popup open; widen the detail popup for readability.
  • Replace single-warning selection with multi-select checkboxes so combined-severity newsroom maps (e.g. yellow + orange wind together) can be produced; re-add severity-ordered compositing of every checked warning's polygons. Restyle chips as narrower, fully severity-colored, white-icon/white-text buttons, and port a reviewed Uiverse.io checkbox component for the selection control.
  • Add per-warning editorial weather-symbol placement to Brīdinājumi, reusing Faktiskā's existing symbol set: drag to move, drag a corner handle to resize, stored as canvas-fraction coordinates so placement holds across both export resolutions; the drag/resize affordance is preview-only and never reaches the exported PNG. This is the app's first direct-manipulation canvas control (everywhere else is plain form inputs).
  • Decide whether Brīdinājumi should actively prevent checking warnings of different phenomena together, or continue leaving that as an operator responsibility.
  • Tune Brīdinājumi feather-blur bleed at sharp coastline curves (e.g. the Gulf of Rīga indentation).
  • Test target resolutions and real workplace display conditions.
  • Check keyboard navigation, contrast, focus states, and screen-reader labeling.

Phase 7 — Real data and production readiness

Status: in progress

Phased plan: real station/temperature data first, then Harmonie once DMI credentials are registered, then remaining sources. Synthetic data is kept running in parallel until each real source is proven, not cut over in one step.

  • Confirm the actual workplace deployment topology and current deployed commit.

  • Find a credential-free real alternative for current station observations: LVĢMC also publishes hidrometeorologiskie-noverojumi on data.gov.lv (the same free, keyless CKAN API already used for Brīdinājumi warnings), including a 48h rolling "operational" resource with the same fields the private FTP feed provides (minus visibilityMin/dewPoint/sunDuration, which aren't published there).

  • Add fetch.lvgmc.OpenDataStationService, pivoting that feed's tall {STATION_ID, ABBREVIATION, DATETIME, VALUE} rows into the existing WeatherData shape via a verified STATION_ID → city map (32 of the app's 37 tracked cities have a direct station), and schedule it in Main.scala (minutes 15/45) alongside — not replacing — the existing FTP fetch, writing into the same weather table via the existing PostgresService.insertInWeatherTable upsert.

  • Verify end to end on Rocky: real rows confirmed in weather via direct SQL, Faktiskā's 13 fixed stations rendering real current temperatures (Valmiera, the one city with no matching open-data station, correctly falls back to older synthetic data via the existing stale-observation UI rather than breaking), and Kartes' aggregate query API returning sane blended real+synthetic values.

  • Stop synthetic data generation now that real data is verified and this tool may plausibly be used against live broadcasts, where mixed real/fake data — both shown identically as "older observation" — is a real hazard, not just messiness: removed dev/seed_weather.sql, the seed Compose service, and the README section describing it.

  • Wipe the synthetic rows already sitting in the weather table on both the VPS and Rocky (both backed up first — VPS to /srv/weathertool/backups/pre-real-data-release/, Rocky to a local pg_dump — before truncating) so history going forward is real data only, not a mix — confirmed Valmiera (no matching open-data station) now shows no data rather than a stale synthetic fallback. Cities with no matching open-data station (Valmiera, Cēsis) should show no data rather than falling back to old synthetic rows — deferred by design, revisit once a real source is found or accepted as permanently unavailable.

  • Deleted the dead METEO_* fetch path (fetch/csv/FetchService.scala, its empty test, and its env vars everywhere). Confirmed via git history it wasn't a separate vendor: the first commit (2023-04-13) included real sample CSVs from it with a Latvian header identical to the LVGMC/open-data fields — the same underlying LVĢMC data, just an earlier delivery mechanism superseded by the FTP feed and never cleaned up. It was never wired into anything that runs.

  • Split scheduled jobs so the working open-data fetch (and cleanup) can run without the FTP/Harmonie jobs' placeholder credentials crash-looping the whole app: ENABLE_SCHEDULED_JOBS now only gates the safe jobs; a new ENABLE_LEGACY_PROVIDER_JOBS (default false) independently gates the FTP station fetch and Harmonie fetch. Turned the former on for Rocky staging; real station data now accumulates automatically every 30 minutes with no manual trigger needed.

  • FTP (fetch/lvgmc/FetchService.scala) credentials obtained 2026-08-23 (ftp.meteo.lv, user ltv) and verified working against both a sample file and the real production file (Latvija_faktiskais_laiks.csv) via a manual sbt runMain check before any scheduling was touched. No code changes were needed to use them — confirming the original design intent. Split the combined ENABLE_LEGACY_PROVIDER_JOBS flag into independent ENABLE_LVGMC_FTP_JOBS/ENABLE_HARMONIE_JOBS flags first, since real credentials for the two providers arrive on different timelines and a single flag would have enabled Harmonie (still placeholder) the moment FTP's were ready — caught this before it could crash-loop the app (the Grib job was ~15 min from its first scheduled run when this was noticed). FTP enabled on Rocky. Also found and fixed a real UTC-vs-local timezone mismatch while verifying FTP alongside open-data: both write into weather.dateTime with no conversion, but the open-data portal's DATETIME is UTC while FTP's "Laiks" column is already local — silently present since open-data went live, only becoming an active problem now that a second, correctly-labeled source exists alongside it. Fixed by converting open-data's timestamps to Europe/Riga at ingestion (OpenDataStationService) and in the Ūdens water-temperature display (WaterTemperatureService, same root cause, display-only). Also hardened insert_weather_table.sql's upsert from a blind overwrite to COALESCE(excluded.field, weather.field), since open-data's rows always carry null visibilityMin/dewPoint/sunDuration and an empty phenomena array — a blind overwrite would silently erase FTP's real values for those fields whenever open-data's write landed later.

  • Went live on both Rocky and the VPS with all of the above: backed up and wiped both weather tables (Rocky: /tmp scratch backup this session; VPS: /srv/weathertool/backups/pre-timezone-fix-wipe/) now that the timezone bug is fixed, so history going forward is correctly and consistently timestamped rather than mixing the old UTC-mislabeled open-data rows with correctly-local FTP rows. Fixed a second real bug found only when wiring up the VPS specifically: deploy/vps/compose.yml hardcoded LVGMC_USER/PASSWORD/URL to inert placeholder strings directly in the file (unlike POSTGRES_*, which already read from .env.staging) — real credentials in .env.staging alone would have had no effect until this was switched to the same ${VAR} substitution pattern. VPS now runs release 3eddf95/dc04f66 with real FTP credentials (added directly on the VPS by the user, never typed into this session) and ENABLE_LVGMC_FTP_JOBS=true. Good thing this was watched instead of assumed safe: the VPS's first scheduled FTP fetch (19:13 UTC) failed with Failed to login to FTP server, and because parMapN cancels every sibling task the instant one throws, this cascaded all the way up through Main.scala's top-level handleErrorWith — which tears down and rebuilds the entire app (DB connections, HTTP server, schedulers) and retries every ~5s. Not a Docker-level restart (RestartCount stayed 0, since the JVM process itself never exited) but a real in-process crash-loop on the public site, repeating every ~2 minutes as the FTP schedule re-fired. Reverted ENABLE_LVGMC_FTP_JOBS to false on the VPS immediately (the user was already asleep; this is undoing a flag flipped minutes earlier as part of the same authorized change, not new destructive action, so didn't wait) — confirmed stable afterward, no fatal errors, public site responding normally again.

    Diagnosed before reverting further: from the VPS, `ftp.meteo.lv:21` is
    reachable and returns a normal vsFTPd banner, and an anonymous login
    attempt correctly gets rejected (`530 Login incorrect`) — so this is
    **not** a network/firewall problem, outbound FTP works fine from the
    VPS. The real `ltv` account specifically fails to authenticate from
    the VPS despite working from Rocky a couple of hours earlier with the
    same credentials. Two candidate causes, neither confirmed (didn't
    inspect the actual password value to check):
    1. A transcription error when the password was typed into the VPS's
       `.env.staging` (manual entry, real risk for a long decoded string).
    2. LVGMC IP-allowlists the `ltv` account to specific source addresses
       — common for partner/business FTP accounts — and the VPS's outbound
       IP isn't on that list even though Rocky's apparently is. The VPS's
       outbound IP is `57.128.250.38`, in case it needs to be given to the
       LVGMC contact for allowlisting.
    **2026-08-24 morning update: the two-machine-collision theory above
    does not fully hold.** Independently confirmed the `ltv` account
    itself isn't hard-locked (a FileZilla login from a separate PC on the
    same network succeeded, browsed `/ltv/tabulas`, saw
    `Latvija_faktiskais_laiks.csv` freshly modified — LVGMC's feed is
    alive). A manual isolated fetch from Rocky then succeeded too. But
    re-enabling Rocky's *scheduler* (`ENABLE_LVGMC_FTP_JOBS=true`, VPS
    confirmed still `false` the whole time) failed on its very next
    scheduled attempt, ~4 minutes after the successful manual test, on
    Rocky alone with nothing else touching the account. Three attempts in
    ~10 minutes, two succeeded and one didn't, with no pattern found yet
    (not simultaneity, not which machine, not obviously attempt spacing).
    Reverted `ENABLE_LVGMC_FTP_JOBS` to `false` on Rocky again immediately.
    **Decision: stop self-testing this via trial and error — every
    attempt might be feeding an unknown rate/session limit — and wait for
    the user to ask the LVGMC contact directly what the account's
    connection policy actually is.** Both Rocky and the VPS run open-data
    only until there's a real answer.
    
  • Researched DMI HARMONIE credentials while waiting on the above and confirmed live (not just from search results, which claimed a specific date that wasn't independently verified): DMI's Forecast Data EDR and STAC APIs no longer validate an API key at all — a direct request to both the old dmigw.govcloud.dk and the new opendataapi.dmi.dk with no api-key parameter returned real GRIB/STAC data from both. New collection-items STAC path confirmed as https://opendataapi.dmi.dk/v1/forecastdata/collections/harmonie_dini_sf/items, with the same properties.modelRun/properties.datetime schema fetch.dmi.FetchService already expects — no response-shape changes needed, just the domain and dropping real key handling (the app's own config loader still needs some non-empty string for HARMONIE_EDR_API_KEY/HARMONIE_STAC_API_KEY, but DMI ignores it). Not wired up yet — see "Harmonie verification (planned)" below.

  • Harmonie verification (planned, next session) — the user confirmed this data is used daily today (exported as a numbered PNG frame sequence per forecast hour, zipped, imported into After Effects for the on-air animated forecast loop — confirmed directly in SlideShow.tsx's downloadImagesAsZip), so getting it working correctly matters, but the code has real, untested risk after a long dormant period (no HARMONIE fixtures, no configured provider access in staging, ever, per the existing product-workflow notes). Real data access itself is solved (see above); what's unverified is everything downstream. Plan, in order: 1. Fetch one real current GRIB file and confirm parse.grib.GribParser decodes it cleanly with sane values — the low-level GRIB2 parsing reads discipline/category/product generically per the spec, so likely still fine, but not yet checked against a real current file. 2. Verify each frontend field renderer (draw/temperature.ts, windDirection.ts, precipitation.ts, snowDepth.ts) against real data via screenshot — check in particular whether "Crop Latvia" (interfaces.ts's CROP_BOUNDS = { x: 1906-1-660, y: 840, width: 660, height: 620, angle: 26 }) still aligns correctly. This is a hardcoded pixel/grid offset into HARMONIE's specific grid domain/rotation — there's already one commented-out earlier version of this constant, meaning it's been hand-recalibrated at least once before, so it's a real, not hypothetical, risk if DMI's grid definition has shifted at all since. 3. Verify the frame-export ZIP workflow end-to-end with real multi-hour data — frame count, naming, timing — against the user's own older HARMONIE exports and existing After Effects projects, which they have on hand for direct comparison. 4. Only then: update HARMONIE_EDR_URL/HARMONIE_STAC_URL to opendataapi.dmi.dk (see above), set HARMONIE_EDR_API_KEY/HARMONIE_STAC_API_KEY to any non-empty placeholder (DMI no longer validates them), flip ENABLE_HARMONIE_JOBS=true. 5. Lower priority, later: the Harmonie workspace is still raw/English dev-tool UI ("Crop Latvia", "Contour", "Interpolate" checkboxes, unstyled file list) unlike the polished, Latvian, newly-redesigned Faktiskā/Ūdens/Brīdinājumi workspaces — worth the same treatment once the data/rendering itself is trusted.

  • Add real water temperature data for Ūdens: fetch.lvgmc.WaterTemperatureService fetches LVĢMC's open hydrological data (data.gov.lv, same free CKAN API), mapping one representative real station per named zone (coastal SEDUT stations for Jūra/Līcis, inland WTEMD stations for the four historical regions). Fetch-on-demand with in-memory caching only — water temperature has never been persisted or needed history, unlike station observations. The frontend auto-fills both min/max per zone from the real reading, mirroring Faktiskā's override/reset pattern.

  • Investigate a real source for the LVĢMC forecast CSV workspace, following the same open-data-portal approach used for station observations and water temperature.

  • Test scheduled ingestion failure and recovery behavior.

  • Rehearse deployment and rollback using sanitized data.

  • Obtain technical and operational review before workplace rollout.

Phase 8 — Temporary VPS user acceptance

Status: in progress

  • Document the proposed isolated VPS topology and trusted Rocky-to-VPS release flow.
  • Choose laikapstak.li and auth.laikapstak.li, create their proxied Cloudflare DNS records, and activate Cloudflare delegation.
  • Record a read-only VPS inventory before provisioning and confirm ports, networks, storage, and capacity do not collide with existing services.
  • Provision a dedicated /srv/weathertool tree, Compose project, private network, database storage, and loopback-only application and Authelia ports.
  • Configure Authelia with one temporary shared account, Argon2id password storage, rate limiting, and temporary IP bans.
  • Add an exact-host Cloudflare origin certificate and Nginx authorization routing without disrupting the existing HOP site.
  • Build and verify commit-addressed images on Rocky, transfer and checksum-verify them on the VPS, and update only the application service. Current full-SHA image: weathertool:138f57c808631dc17396b22ac0305b670fbcf776; PostgreSQL and Authelia were not restarted.
  • Complete UAT verification: public authentication and application health pass; direct-origin blocking, logout, throttling, PNG downloads, backups, logs, and rollback remain.
  • Share the domain with newsroom testers without a walkthrough first, to observe unassisted intuitiveness, before providing any guidance.
  • Run the month-long user test with manual releases and record feedback before any workplace-production decision.

Known current limitations

  • The fixed production PNG bases, browser/social metadata, reviewed frontend dependency updates, stabilized header navigation, authenticated API routing, the full Brīdinājumi warning-map workspace (including draggable per-warning symbol placement), hardened LVĢMC warning fetching, real open-data station/water-temperature ingestion, and the scheduler safety split are deployed in UAT release 138f57c8; newsroom workflow validation is in progress.

  • Link-preview crawlers cannot authenticate through Authelia; the final Nginx policy must deliberately expose only the minimum preview metadata/assets if WhatsApp previews are required.

  • Both the VPS and Rocky weather tables now hold only real open-data station observations; their original synthetic rows were backed up and wiped 2026-08-23.

  • LVGMC forecast CSV fixtures are not yet available.

  • HARMONIE GRIB fixtures are not yet available.

  • The private LVGMC FTP feed has real credentials as of 2026-08-23, but ENABLE_LVGMC_FTP_JOBS stays false on both Rocky and the VPS pending a real answer from LVGMC about the ltv account's connection/rate policy (see Phase 7). The DMI HARMONIE forecast feed remains gated (ENABLE_HARMONIE_JOBS, default off) pending real credentials — turned out not to need any (DMI dropped its key requirement), but wiring it up is deferred to a dedicated verification session (see Phase 7).

  • Existing automated test coverage is minimal.

  • Full Docker build context scanning on Rocky can fail on the container-owned postgres/ bind directory; do not loosen its permissions.

Change log

Record completed work here by date and commit after the Git workflow is established.

Date Commit Summary Verified on Rocky
2026-08-18 b1fff67 Docker development baseline, isolated staging, scheduler switch, and synthetic station data Yes
2026-08-18 bb1d9fc Home/UI foundation, map preview layout, friendly empty state, and PNG export Yes
2026-08-18 67decb3 Adaptive temperature badges and title/source overlays Yes
2026-08-18 82199b8 Separate City Analysis and Faktiskā workflows with direct symbol assignment Yes
2026-08-18 0c78d3c Optical centering for bundled weather-font glyphs Pending Rocky verification
2026-08-19 e726291 Fixed Faktiskā workflow with latest observations, manual overrides, and production-size export Yes
2026-08-19 5913217 Add Valmiera to the deterministic Faktiskā development data Yes
2026-08-19 632f377 Arrange manual Faktiskā wind controls horizontally Yes
2026-08-19 340c3703aba9b4 Anchor and visually calibrate manually selected Faktiskā symbols Yes
2026-08-19 18680416c9290b Add Ūdens temperatūra, bundle and load Monda, fix overlay scaling, and center its locked nameplate Yes
2026-08-19 0d641bd Replace runtime Daira glyph rendering with normalized transparent image assets Yes
2026-08-20 754415a Match editable title/source overlays to production-safe right margins Yes
2026-08-20 16a8c68 Deploy approved production map bases, normalized branding assets, favicon, Apple icon, and social metadata to public UAT Yes — Rocky build and VPS/browser smoke checks
2026-08-21 5007517de6f279 Update Solid runtime and frontend build tooling, add type checking, remove obsolete packages, prune the lockfile, and resolve all npm advisories Yes — clean npm ci, typecheck, production build, full audit, and production-only audit
2026-08-22 ef64895 Deploy the reviewed zero-advisory frontend dependency state as an immutable full-SHA VPS release while leaving PostgreSQL, Authelia, and HOP uninterrupted Yes — isolated Rocky smoke test, dual-host checksum, matching image ID, container health, loopback, authentication-gate, and HOP checks
2026-08-22 36d094e Stabilize the header, keep the primary workspaces visible, and move secondary workspaces into an icon-labelled menu Yes — Rocky browser and Jam navigation checks
2026-08-22 df911f1 Preserve API 401 responses through Authelia/Nginx so frontend queries do not follow cross-origin login redirects Yes — Rocky and VPS health checks; authenticated browser query verified after refreshing synthetic data
2026-08-22 f3197bf Return query API payloads with an explicit JSON content type Yes — Rocky API and browser verification; deployed to VPS in release b5150ab on 2026-08-22
2026-08-22 0608865 Localize the operator-facing workspace names and concise workflow copy in Latvian Yes — Rocky browser verification; deployed to VPS in release b5150ab on 2026-08-22
2026-08-22 7f09181 Independently calibrate both Ūdens templates and optically center Monda values using visible glyph bounds Yes — both native-resolution PNG exports visually validated on Rocky; deployed to VPS in release b5150ab on 2026-08-22
2026-08-22 bc5bee1 Add the Brīdinājumi LVĢMC warning-map workflow with metadata/polygon ingestion, day/phenomenon filtering, and manual title entry Yes — Rocky browser verification; deployed to VPS in release b5150ab on 2026-08-22
2026-08-22 13115f0 Add production border overlay artwork and feathered warning-polygon fills to Brīdinājumi Yes — Rocky browser verification; deployed to VPS in release b5150ab on 2026-08-22
2026-08-22 4395935 Replace Brīdinājumi's 4-corner bounding-box projection, which drifted up to ~200px on the 3840 canvas, with an affine fit calibrated against the validated Kartes/Faktiskā city pixel positions Yes — Rocky typecheck, build, and headless-browser screenshot comparison at both native export resolutions; deployed to VPS in release b5150ab on 2026-08-22
2026-08-22 924e410 Limit Brīdinājumi to one selected warning per export, remove the on-canvas legend, collapse card detail until clicked, and pre-fill the title from the selected warning's phenomenon Yes — Rocky typecheck, build, and headless-browser screenshot verification of selection switching and card detail; deployed to VPS in release b5150ab on 2026-08-22
2026-08-22 cafc75a Redesign Brīdinājumi warning cards as compact phenomenon-icon chips and move full warning text into a dialog popup over a dimmed, blurred backdrop Yes — Rocky typecheck, build, and headless-browser screenshot verification of the popup open/close flow; deployed to VPS in release b5150ab on 2026-08-22
2026-08-22 70fd24c Give Brīdinājumi chips a hover-expanding severity capsule, adapted from a reviewed Uiverse.io reference component Yes — Rocky typecheck, build, and headless-browser screenshot verification of rest/hover/active chip states; deployed to VPS in release b5150ab on 2026-08-22
2026-08-22 6fbf672 Split Brīdinājumi warning selection (checkbox) from viewing detail (chip click) and widen the detail popup Yes — Rocky typecheck, build, and headless-browser verification that checkbox clicks change the map selection without opening the popup and chip clicks open the correct warning's detail without changing the selection; deployed to VPS in release b5150ab on 2026-08-22
2026-08-22 56b8716 Replace single-warning Brīdinājumi selection with multi-select checkboxes and severity-ordered polygon compositing, and restyle chips as narrower, fully severity-colored, white-icon/white-text buttons with a ported Uiverse.io checkbox component Yes — Rocky typecheck, build, and headless-browser screenshot verification of multiple simultaneous checked warnings compositing correctly on the map; deployed to VPS in release b5150ab on 2026-08-22
2026-08-22 6185dbf Harden WarningService against partial upstream failures: bounded outage retry, per-warning fetch isolation, a live-API-verified CKAN polygon filter, and visible logging for a previously-silent vertex-order default Yes — Scala tests, Rocky full-stack rebuild verified against real live LVĢMC data (including a genuinely severe 52,345-point wind-warning polygon rendering correctly); deployed to VPS in release 6185dbf on 2026-08-22, verified against live data on the VPS itself
2026-08-23 c97875d Add draggable, resizable per-warning weather-symbol placement to Brīdinājumi, reusing Faktiskā's symbol set and image cache Yes — Rocky typecheck, build, and headless-browser mouse-drag/resize verification against fixture warning data (the live LVĢMC feed had zero active warnings at test time); confirmed the exported PNG excludes the preview-only drag/resize handles; deployed to VPS in release 138f57c8 on 2026-08-23
2026-08-23 893a09a Add fetch.lvgmc.OpenDataStationService: real, free, keyless current station observations from LVĢMC's open-data portal, scheduled alongside (not replacing) the private FTP feed Yes — Scala tests, manual sbt runMain fetch+write against the real feed on Rocky staging (1,577 rows, 33 cities), confirmed in weather via direct SQL, Faktiskā's 13 fixed stations rendering real temperatures with correct stale-fallback for the one uncovered city (Valmiera), and Kartes' aggregate query API returning sane blended values; deployed to VPS in release 138f57c8 on 2026-08-23
2026-08-23 cb25316 Stop generating synthetic weather data (remove dev/seed_weather.sql, the seed Compose service, and the README section describing it) now that real data is verified and this tool may plausibly be used against live broadcasts Yes — docker compose config valid, running Rocky stack unaffected (profile-gated service, no running containers touched); deployed to VPS in release 138f57c8 on 2026-08-23
2026-08-23 6ddfe73 Split scheduled jobs (ENABLE_SCHEDULED_JOBS vs. new ENABLE_LEGACY_PROVIDER_JOBS) so the working open-data station fetch runs without the FTP/Harmonie jobs' placeholder credentials crash-looping the whole app via parMapN Yes — Scala tests, Rocky rebuild deployed with the flag on: logs confirm only the safe jobs scheduled, FTP/Harmonie never attempted, app stayed up; deployed to VPS in release 138f57c8 on 2026-08-23
2026-08-23 39fcb3e Remove the dead METEO_* fetch path (fetch/csv/FetchService.scala, its empty test, and its env vars everywhere) — confirmed via git history it's the same underlying LVĢMC data as the FTP/open-data paths, just an earlier, superseded delivery mechanism that was never wired into anything running Yes — Scala tests (2 suites now, down from 3, the deleted one was empty); deployed to VPS in release 138f57c8 on 2026-08-23
2026-08-23 4036d244b92272 Add fetch.lvgmc.WaterTemperatureService: real, free, keyless water temperatures for all 6 Ūdens zones (2 coastal SEDUT stations for Jūra/Līcis, 4 inland WTEMD stations for Kurzeme/Zemgale/Vidzeme/Latgale), fetch-on-demand with a 15-minute in-memory cache mirroring WarningService rather than the persisted station-observation path, since Ūdens has never stored history Yes — Scala tests, Rocky rebuild, direct API curl confirming real values for all 6 zones; caught and fixed one station-selection error (Daugavpils only reports water level, not temperature — swapped for Ludza) via direct verification before committing; deployed to VPS in release 138f57c8 on 2026-08-23
2026-08-23 5a92465 Auto-populate Ūdens's min/max fields from /api/water-temperatures on load, mirroring Faktiskā's fetch/override/reset pattern, with per-zone manual-override tracking and an "Atiestatīt" reset button Yes — Rocky typecheck, build, and headless-browser verification (Playwright in Docker, since chromium-cli wasn't available in this environment) confirming auto-populated values match the API, manual edit correctly flips a zone to "Manuāli" and enables its reset button, and reset correctly restores the fetched value; noted along the way that /udens-temperatura 404s on a direct hit (no SPA fallback yet, consistent with the known limitation already listed) but works via client-side nav; deployed to VPS in release 138f57c8 on 2026-08-23
2026-08-23 9eca9eb Fix the one-station-per-zone design catching a real problem after deploy: every zone showed the same value duplicated as both min and max ("19...19"). Classified all 65 LVĢMC stations that report water temperature (56 inland WTEMD, 9 coastal SEDUT) into the 6 zones by geography and report the real min/max across each zone's currently-reporting stations, dropping readings over 12h stale so one stuck sensor can't skew a range Yes — Scala tests, Rocky rebuild, direct API curl confirming a genuine spread per zone (e.g. Vidzeme 11.418.7°C, traced the low point to a real fresh reading from a colder headwater station near Cēsis, not a stale-sensor artifact), and headless-browser screenshot confirming the UI renders the real spread (e.g. "11...19") with no console errors; deployed to VPS in release 138f57c8 on 2026-08-23
2026-08-23 69a5260 Bring architecture/workflow docs current with real-data ingestion (removed synthetic-seed staging descriptions, documented Ūdens auto-populate, the scheduler split) Yes — doc-only change, no rebuild needed
2026-08-23 138f57c8 Full go-live VPS release: deployed image weathertool:138f57c808631dc17396b22ac0305b670fbcf776 (everything from c97875d through 69a5260 — Brīdinājumi draggable symbol placement, real open-data station/water-temperature ingestion, synthetic-data removal, scheduler split, METEO_* deletion, real per-zone water-temperature ranges), flipped ENABLE_SCHEDULED_JOBS=true on the VPS now that the open-data path is proven, and wiped the VPS weather table's original 12,784 synthetic rows down to 0 (Rocky's own table was left untouched — separate decision, not requested for this release) Yes — exact release image smoke-tested on Rocky (healthy, real warnings/water-temperature data) before transfer; VPS PostgreSQL backed up to /srv/weathertool/backups/pre-real-data-release/ before the wipe; checksum verified on both ends; app service recreated without touching PostgreSQL/Authelia; confirmed healthy container, working loopback and public HTTPS (302 unauthenticated page, 401 unauthenticated API — matching established behavior), real warnings/water-temperature API responses, the scheduled open-data job firing on schedule and landing 1,577 real rows within ~2 minutes, and a real (non-Valmiera) city query returning genuine current temperatures with Valmiera correctly absent rather than falling back to wiped synthetic data
2026-08-23 744d706 Rework the visual palette: layered blue-gray background gradient (sampled tones, iterated visually against Home/Brīdinājumi/Kartes) replacing the near-flat pale tint, while keeping buttons/icons/links on the original saturated accent blue; removed the "Testa dati" badge, a synthetic-data leftover Yes — Rocky build and headless-browser screenshot comparison across Home, Brīdinājumi, and Kartes; deployed to VPS in release f78b0a26 on 2026-08-23
2026-08-23 a1c2fd5 Declutter Faktiskā: collapse "Jaunākās temperatūras" and "Kartes noformējums" by default behind toggles (with a manual-override-count badge so nothing's silently hidden), fix a real cascade bug forcing Faktiskā's resolution buttons into full-width left-aligned rows instead of a compact pair, restructure "Kartes noformējums" into distinct bordered cards, translate its still-English symbol/city-assignment block to Latvian ("Mākoņi pilsētām"), and collapse its city list to exceptions-only by default Yes — Rocky typecheck, build, and headless-browser screenshot verification of the collapse/expand toggles, the exceptions-only city list, and the button fix; deployed to VPS in release f78b0a26 on 2026-08-23
2026-08-23 5c00e48 Fix temperature numbers rendering visually too high in their map badges: canvas textBaseline: "middle" centers on font em-box metrics, not visible glyph ink, and digits have no descenders — centered on actual glyph bounds instead, matching the technique already used correctly in Ūdens Yes — Rocky build and headless-browser screenshot comparison of the map badges before/after; shared by Faktiskā and the older Kartes comparison map; deployed to VPS in release f78b0a26 on 2026-08-23
2026-08-23 59bb499 Promote Brīdinājumi from the "Vairāk" overflow menu into the always-visible header nav, and add a small icon to every visible nav item (reusing the same icons as their Home cards) Yes — Rocky build and headless-browser screenshot verification at 1900/1440/1200px widths, confirming no overflow and correct active/overflow-menu state; deployed to VPS in release f78b0a26 on 2026-08-23
2026-08-23 f78b0a26 Deploy the full frontend design pass (background/accent rework, Faktiskā decluttering, temperature-badge centering fix, header nav) to the VPS — no database or scheduler changes, application-only release Yes — exact release image smoke-tested on Rocky (bundle hash and headless-browser screenshot confirmed against the known-good local build) before transfer; checksum verified on both ends; app service recreated without touching PostgreSQL/Authelia; confirmed healthy container, matching bundle hash on the VPS, working loopback and public HTTPS (302 unauthenticated page, 401 unauthenticated API), and real warnings/water-temperature API responses
2026-08-23 d7439a2b Fresh-eyes review of the whole design pass turned up two real issues, fixed here: WindInputs had no Latvian branching and was showing "Wind direction"/"Wind speed"/"Gusts" in English right under the freshly-translated "Temperatūra un vējš" heading on Faktiskā (threaded the same productionTemplate flag MapView.tsx already uses elsewhere), and a confirmed-unreferenced .assignedSymbol CSS rule was removed. The review also flagged two things left alone: a pre-existing (not introduced this session) .symbolPalette/.selectedPreview/.currentSymbol cross-file class collision in weatherIcons.css currently masked by !important rather than actually resolved, and a naming nit on "Mākoņi pilsētām" (the section assigns arbitrary weather symbols, not just clouds) — both worth a look later, not blocking Yes — Rocky typecheck, build, and headless-browser screenshot confirming the wind fields now render in Latvian; deployed to VPS in release d7439a2b, exact release image smoke-tested (bundle hash matched) before transfer, checksum verified, container healthy, public HTTPS/API checks passing
2026-08-23 e446f1f Consolidate .symbolPalette/.selectedPreview/.currentSymbol (used by both Faktiskā's IconInputs.tsx and Brīdinājumi's Warnings.tsx) into one real definition in weatherIcons.css, the file both actually import, removing the dead/duplicate versions in mapGraphics.css and the !important size patches that were masking the collision. "Mākoņi pilsētām" naming confirmed intentional — the newsroom users are non-technical and the same people who'll use this long-term, so intuitive-but-imprecise beats literally-correct-but-jargony Yes — Rocky typecheck, build, headless-browser screenshot on Faktiskā (pixel-identical to before), and computed-style verification on both Faktiskā and Brīdinājumi confirming identical resolved CSS with no !important; caught one incidental fix for free (.currentSymbol.empty's Inter font was silently losing to the !important rule, normal cascade now applies it correctly); deployed to VPS in release e446f1ff, exact release image smoke-tested (bundle hash matched) before transfer, checksum verified, container healthy, public HTTPS/API checks passing
2026-08-23 2b5dff6 Make the weather table upsert non-destructive: ON CONFLICT DO UPDATE was a blind full-row overwrite, so open-data's always-null visibilityMin/dewPoint/sunDuration and always-empty phenomena would silently erase real FTP values whenever open-data's write landed later for the same row. Switched to COALESCE(excluded.field, weather.field), with NULLIF against an empty array specifically for phenomena since Scala's List[String] never maps to SQL NULL Yes — Scala tests; found while wiring up real FTP credentials, before any real dual-source writes had happened
2026-08-23 cab94d0 Fix a real UTC-vs-local timezone mismatch: the open-data portal's DATETIME is UTC, stored into weather.dateTime with no conversion, while FTP's "Laiks" column is already local and also stored as-is — silently present since open-data went live (893a09a), invisible until a second, correctly-labeled source existed alongside it. Fixed by converting to Europe/Riga at ingestion in OpenDataStationService Yes — Scala tests; verified live: at real local time 20:31 EEST the fix produces dateTime=20:00 instead of the previous 17:00, matching what FTP writes for the same real hour
2026-08-23 3315f00 Fix the same UTC-vs-local mismatch in Ūdens's observedAt display (WaterTemperatureService) — same open-data portal, same root cause, display-only (internal recency filtering was already self-consistent either way) Yes — Scala tests; verified live via /api/water-temperatures: observedAt now matches real local time
2026-08-23 3eddf95 Split ENABLE_LEGACY_PROVIDER_JOBS into independent ENABLE_LVGMC_FTP_JOBS/ENABLE_HARMONIE_JOBS — real LVGMC FTP credentials arrived today, real DMI HARMONIE credentials haven't, and the combined flag would have enabled both together, crash-looping the app on HARMONIE's still-placeholder values via parMapN Yes — Scala tests; caught before it could happen (the Grib job was ~15 min from its first scheduled run when noticed) and reverted within under a minute; re-verified after the fix that only "Fetch Weather Stations" scheduled, not "Fetch Grib"
2026-08-23 dc04f66 Fix deploy/vps/compose.yml hardcoding LVGMC_USER/PASSWORD/URL to inert placeholder strings directly in the file (unlike POSTGRES_*, which already read from .env.staging) — real credentials added to .env.staging alone would have had no effect until this switched to the same ${VAR} substitution pattern. Enabled ENABLE_LVGMC_FTP_JOBS on the VPS Yes — docker compose config syntax valid; VPS app container came up healthy with real credentials loaded (would have crashed immediately on missing-config if not)
2026-08-24 6b9c7cf Fix path traversal, auth-bypass, and DoS findings from an independent fresh-eyes security review (requested by the user specifically to get a second, skeptical pass against the local Rocky version): a live-verified CRITICAL — /api/debug/file/{fileName} allowed unauthenticated arbitrary file read, confirmed by reading /proc/self/environ and leaking LVGMC_PASSWORD/POSTGRES_PASSWORD into the review's own output — plus the same raw-filename pattern in /show/grib, /grib/binary-chunk, and /show/lvgmc-forecast. Added ValidateFileName (allowlist regex; rejects .. and separators including decoded %2f) and applied it everywhere a path segment reached a filesystem or FTP path; hardened ValidateInt to reject negatives; gated /api/fetch/lvgmc/stations behind ENABLE_LVGMC_FTP_JOBS (it could otherwise trigger a real, unauthenticated FTP login regardless of the flag) and stopped it leaking error.getMessage; added an explicit /api/* catch-all so an unmatched API route can never fall through to the SPA shell; capped binary-chunk read length at 64MB Yes — Scala tests, Rocky rebuild, live curl verification of %2f-encoded and literal traversal payloads (404), the gated FTP route (503 while the flag is off), an unmatched /api/* route (404, not index.html), and legitimate filenames/queries still returning correct 200s
2026-08-24 8c45d8d Fix SQL injection surfaced by the same security review: /api/query/city/.../{field}/... passed its field path segment unvalidated into PostgresService.query, which splices it into SQL via Fragment.const (unescaped) whenever the aggregate key is min/max/avg/sum/distinct, or whenever granularity is hour in the list branch. Added ValidateField, reusing the WeatherData.getKeys allowlist AggFieldList already applied to /query/country. AggregateKey values reaching Fragment.const elsewhere were already safe — they come from a closed ADT (AggregateKey.fromString), not raw input Yes — Scala tests, Rocky rebuild, live curl verification that SQL-injection payloads in the field segment 404 and a legitimate query still returns correct data
2026-08-24 (operational, no commit) Rotated Rocky's POSTGRES_PASSWORD following the security review, since the old value had been exposed into this session's context multiple times (two of my own sloppy shell commands, plus the /proc/self/environ read the review used as its traversal proof). Changed the role's actual password via ALTER ROLE against the running container (editing .env alone has no effect on an already-initialized PostgreSQL data directory), then updated .env and recreated the scala service. The Postgres container itself was also recreated as a side effect (its own POSTGRES_PASSWORD env interpolation changed too), which is harmless — that variable only takes effect on a fresh, empty data directory, not an existing one — but is worth knowing about Yes — clean scala container startup with no auth errors, and a real query (/api/query/latest-temperatures/Rīga) returning live data over the new password
2026-08-24 0be325f Deploy the security-review fixes (6b9c7cf, 8c45d8d) to the VPS as image weathertool:0be325fbbbf92b03bc8d574dc6f7b9449ef310ea; release b0b58d2 remains the immediate application rollback. Application-only release — PostgreSQL and Authelia were not restarted Yes — exact release image smoke-tested on Rocky against a throwaway local Postgres before transfer (traversal/injection payloads 404, gated FTP route 503, legitimate queries 200); checksum verified on both ends; app service recreated cleanly (healthy, no auth/DB errors in logs); loopback re-verification of the same traversal/injection/FTP-gate checks plus real /api/warnings and /api/water-temperatures responses; public HTTPS confirmed unchanged (302 unauthenticated page, 401 unauthenticated API)
2026-08-24 9bab93d A follow-up independent fresh-eyes review (requested specifically as a second pass, not a rerun of the same pentest) caught that the 6b9c7cf FTP auth-bypass fix was incomplete: only /api/fetch/lvgmc/stations (the one route the original review named) was gated behind ENABLE_LVGMC_FTP_JOBS. A sibling route, /api/show/lvgmc-forecast/{fileName}, calls the same fetch.fetchFile — a real, unauthenticated LVGMC FTP login — and had only gotten the traversal fix (ValidateFileName), not the auth-bypass fix, because the two routes were fixed along different mental categories (traversal batch vs. the one named auth-bypass fix) instead of by tracing every caller of the dangerous method. Gated this route the same way; traced every HTTP-reachable caller of fetch.fetchFile/fetchWeatherStations this time to confirm no others remain. Deployed to both Rocky and the VPS (image weathertool:9bab93daa6b3a0fb99165a8fac9accfa3a3ccbf1) immediately given this was live and exploitable in production Yes — Scala tests, Rocky rebuild and curl verification (both FTP routes 503 while the flag is off), VPS image smoke-tested against a throwaway local Postgres before transfer, checksum verified, app service recreated cleanly, loopback re-verification of both gated routes, public HTTPS unchanged (302/401)
2026-08-24 (review, no commit) A follow-up independent fresh-eyes review (explicitly scoped as a second, different pass — not a rerun of the original pentest) sanity-checked the 6b9c7cf/8c45d8d fixes and did a broader skeptical pass. Caught the incomplete FTP fix documented under 9bab93d above. Confirmed ValidateFileName's minor "." edge case is harmless (resolves to a directory, not exploitable) and found no other unvalidated Fragment.const usages. Surfaced several open architectural/testability findings now tracked in Phase 2/Phase 5 above: the parMapN scheduler crash-loop pattern (still fully live), per-request System.gc() in getBinaryChunk, PostgresService.query's unenforced field-routing invariant, no JDBC connection pooling, and several files under src/main/scala (DataServiceTest, FetchServiceTest ×2, OpenDataStationServiceTest, GribParserTest) that look like tests but are actually main-method scripts never run by sbt test — compiled straight into the production jar, at least one capable of writing to the real database or hitting live endpoints if run by mistake
2026-08-24 5b88e69 Add Gitea Actions CI: a repository-scoped weathertool-ci-rocky-01 runner on Rocky (isolated behind its own Docker-in-Docker daemon, mirroring the isolation pattern already proven by HOP's Forgejo runner on the same host — the two coexist without conflict, each an independent client registered to a different server) runs sbt test and the frontend typecheck/build/audit routine on every push/PR to codex/staging-baseline plus manual dispatch. Verified sbt test compiles and passes with no .env present at all (via a git archive HEAD dry run) before writing the workflow, since CI must never see real credentials. Repo mirrored to Gitea (bot/WeatherTool) as an additional remote alongside the existing rocky/origin ones — deliberately not replacing either, pending an open question about the GitHub origin (guntisdev/WeatherTool) that won't be resolved until 2026-08-26. See CONTINUOUS_INTEGRATION.md Correction: this row originally claimed the first run (ci.yml #1) verified green, based on a WebFetch summary of the Gitea Actions page that turned out to be simply wrong — reported as fact without visually checking it first. It actually failed, along with the next three runs, on three different real bugs (see 7e1b492/2c78ae8 below and CONTINUOUS_INTEGRATION.md). First actual green run was ci.yml #5, commit 2c78ae8
2026-08-24 7e1b4922c78ae8 Fix three real CI bugs surfaced by actually running the new workflow, found via user-provided screenshots of the Gitea Actions UI after the WebFetch-based "success" claim above turned out false: (1) the runner's HOME/cache-directory permission error, initially misdiagnosed as only disabling the optional actions/cache action type, actually blocked resolving actions/checkout at all — fixed with HOME=/data on the runner container (operational fix, in compose.yml, not this repo); (2) actions/checkout is a JavaScript action needing Node in the job container, which backend's hseeberger/scala-sbt image doesn't have — fixed with an apt-get install nodejs step before checkout (7e1b492); (3) that apt-installed Node was Debian bullseye's own stale default (Node 12), too old for actions/checkout@v4's modern JS — fixed by installing a real current Node 20 via NodeSource's setup script instead (2c78ae8). See CONTINUOUS_INTEGRATION.md for the full detail Yes — ci.yml #5 (commit 2c78ae8) verified green in the Gitea Actions UI via user-provided screenshot (2m24s), both jobs passing
2026-08-25 464e035 Add shared side padding (clamp(16px,3vw,48px)) to the one truly shared layout wrapper (.appContent) so full-bleed pages (Kartes, Stacijas, Apskats — the only three with no CSS of their own) get breathing room on the sides instead of running edge-to-edge, while staying full-width rather than a narrower centered column Yes — Rocky rebuild, headless-browser screenshot of Kartes before/after; confirmed no double-padding on pages that already set their own margins (Home)
2026-08-25 1e83e48 Fix a real CSS collision in Kartes' results grid (Result.css): a newer consolidated .item rule used background:var(--surface)!important to force out an older duplicate rule instead of removing it, leaving the old rule's padding/text-align/max-width still silently active underneath. Also removed dead frontend code found via a systematic check of every class in every stylesheet against actual .tsx usage — undercounted by a prior review by several: a whole leftover "Production layout" block in mapGraphics.css (9 classes, including a scoped selector that never matched anything in the current markup), unused .grid-1-1/.grid-1-2/.panel utility classes, dead calendar-navigation CSS, and dead commented-out JSX in DateList.tsx Yes — typecheck clean, production build succeeds, headless-browser screenshots of Kartes (including a real query result), Faktiskā, and Arhīvs show no visual regression
2026-08-25 e62895a Narrow Ūdens's six temperature cards so all fit on one row instead of wrapping — switched grid-template-columns to repeat(auto-fit, minmax(165px,1fr)). Caught a real bug while tuning the floor value: an earlier hardcoded repeat(6, minmax(130px,1fr)) pass produced 6 columns at every width from 10501900px, but at some in-between widths (e.g. 1300px) the resulting input boxes came out to ~38px — narrow enough that the native number-spinner arrows ate nearly all of it, clipping digits that were genuinely present in the DOM (confirmed via inputValue()) to nothing visible. Fixed by hiding the native spinner arrows and switching to auto-fit, which can't produce that in-between too-narrow zone the way a hand-rolled breakpoint ladder can Yes — a programmatic sweep of 14 viewport widths (3751900px) reading each input's real computed width and value, not just screenshots, confirmed every width keeps a comfortably legible input
2026-08-25 e1ce153 Give Ūdens's resolution-choice buttons the same visual language as the temperature cards next to them (border, radius, soft glass background) instead of the app's generic flat button style, after measuring (via Playwright bounding boxes, not eyeballing) that the perceived "buttons sit higher" was a visual-weight mismatch, not an actual coordinate offset — the tops were already pixel-identical Yes — Rocky rebuild, headless-browser screenshot confirmation
2026-08-25 07b16ac Source Valmiera's Faktiskā temperature from Priekuļi as a temporary substitute, since LVĢMC's open-data station list has no station in or near Valmiera at all. Priekuļi chosen by measuring against the app's own calibrated map coordinates (~56px away vs. the next-closest candidate's ~81px), not a guess — same pattern as Stende/Zīlāni substituting for Talsi/Jēkabpils. Implemented client-side in Faktiskā's fetch only; Valmiera's marker position and on-map label are untouched. Also fixes a second symptom for free: the weather-symbol picker only lists cities with a value, so Valmiera never appeared there either. Provisional pending direct confirmation from LVĢMC Yes — typecheck clean, production build succeeds, headless-browser checks confirm Valmiera's field shows Priekuļi's real live value (fresh, not flagged manual/stale) and now appears in the weather-symbol exceptions list (13 cities, was 12)
2026-08-25 4c40c24 Add click-to-edit temperature badges directly on the Faktiskā map, as a fast path for the common "one or two values need a tweak" case instead of always opening the 13-field "Rādīt stacijas" panel. Mirrors Brīdinājumi's existing canvas-coordinate pattern exactly (getBoundingClientRect scaling) rather than inventing a new one, hit-testing against Faktiskā's already-fixed per-resolution marker positions. The floating input is a plain DOM element positioned over the canvas, never drawn onto it, so it stays preview-only like Brīdinājumi's drag handles. Reuses the existing updateValue function via a new onEditValue callback prop, so the "Rādīt stacijas" panel's override tracking/reset/manual-count badge all stay in sync automatically. Kept the panel rather than removing it after this shipped — it's the only place that surfaces missing/stale data and offers a reset, neither of which click-to-edit does Yes — end-to-end Playwright test: hover shows a pointer cursor, click shows a correctly positioned input prefilled with the real value, Enter commits and the canvas redraws, the change appears in the panel as a manual override, Escape cancels without committing, works correctly at both export resolutions, no console errors, Kartes (a different MapView mode) unaffected
2026-08-25 829319e Polish hover/focus states on inputs and checkboxes (matching the checkbox/button interaction language already established elsewhere), restyle Brīdinājumi's warning checkbox to match, and drop a stray hardcoded Inter font override on Faktiskā's reset-temperature button Made directly by the user, included in the next release
2026-08-25 a6b2b84 Fix Faktiskā showing blank temperatures for a stretch after every new hour starts, reported live by the user on both Rocky and the VPS. Root cause: queryLatestTemperatures picked the single most-recent row per city with no regard for whether that row's tempAvg was actually populated — the open-data source publishes fields on different schedules within an hour (snow arrives well before the hourly temperature aggregate), so the freshest row routinely has a real timestamp but a still-null tempAvg while the previous hour's row has good data. Confirmed live on both Rocky and the VPS before fixing: every city's newest row had tempAvg null but a real snowAvg=0. Fixed by requiring tempAvg IS NOT NULL in the query, so it finds the newest row that actually has a value instead of a fresher partial one. Isolated to Faktiskā — Kartes aggregates over a range (one partial hour doesn't blank the result) and Ūdens is a separate data source entirely, which is why only Faktiskā showed the symptom Yes — confirmed the root cause directly (upstream open-data API check, full-row DB inspection on both Rocky and VPS) before writing the fix, not assumed; compiled; deployed to both Rocky and the VPS following VPS_RELEASE_RUNBOOK.md (which itself got a real bug fixed while running it for the first time — the checksum file recorded a full local path instead of a bare filename, failing verification on the VPS); confirmed all 13 Faktiskā stations show real 15:00 data on Rocky via headless browser, confirmed the same live on the VPS via curl, public HTTPS gate unchanged (302/401)
2026-08-25 001b014 Deploy everything accumulated since the last VPS push (9bab93d) as image weathertool:001b014932a1c5e28080dd057613f8bb807a3b81: the frontend padding/cleanup/Ūdens-layout/button-styling pass, the Valmiera→Priekuļi substitution, and click-to-edit on Faktiskā. The Gitea CI setup itself doesn't affect the deployed app. Release b0b58d2 remains the rollback target Yes — exact release image smoke-tested on Rocky against a throwaway local Postgres before transfer (traversal/injection payloads 404, gated FTP routes 503, /faktiska and /udens-temperatura direct hits 200); confirmed the built image's CSS bundle genuinely contains tonight's changes (not a stale cache hit) before shipping; checksum verified on both ends; app service recreated cleanly (healthy, schedulers registered, no errors in logs); loopback re-verification of the same security/routing checks plus confirmed Priekuļi has real live data server-side; public HTTPS unchanged (302/401). Browser-level verification of Valmiera/click-to-edit specifically relied on Rocky's already-passing headless-browser test of the identical code, since the VPS app port is loopback-only by design and not reachable for a direct browser check from this session
2026-08-24 2c44888b0b58d2 Replace the hardcoded per-route static-file list (/station, /cities, /latvia, /database, /harmonie, /lvgmc-forecast, /bridinajumi) with a general SPA fallback: real files serve as-is, anything else falls back to index.html so the SolidJS router handles it client-side — fixes a real, user-reported bug where a direct hit (e.g. a browser refresh) on /faktiska or /udens-temperatura 404ed instead of loading the app, since those two routes were never added to the old list. A missing file under /assets specifically still 404s properly rather than silently serving HTML, so a stale tab after a future deploy gets a clean error instead of a confusing JS parse failure Yes — first attempt silently no-op'd because the fix was built into a git archive HEAD image before being committed (classic mistake, caught immediately by re-testing and finding identical old behavior); after committing, verified via curl on all previously-working routes, both previously-broken routes, a missing asset (404), a real asset (200), and /api (200), then a full headless-browser render check (zero console errors, real data, correct nav state) on a genuine direct hit — not just HTTP status codes; deployed to VPS in release b0b58d2, verified the same way through both the loopback port and the public domain (Authelia gate still correctly redirects unauthenticated requests)