WeatherTool/docs/UPDATE_ROADMAP.md

WeatherTool update roadmap

This document tracks proposed WeatherTool improvements. Work should be delivered in small, reviewable phases rather than as one large rewrite. Each phase should leave the application runnable and independently testable.

Environments and workflow

• Windows development: source editing and Git workflow only; Docker runtime testing is performed on Rocky.
• Rocky staging: production-like Docker deployment at http://192.168.1.101:9190.
• Git over SSH: Windows pushes reviewed commits to a private bare repository on Rocky; the Rocky staging checkout pulls those commits and rebuilds.
• Temporary VPS staging: release ef64895 is publicly operational behind Cloudflare strict TLS, Nginx, and Authelia with a limited 14-day synthetic dataset.
• Workplace production: remains separate until changes are reviewed, tested, and explicitly approved for workplace use.

Do not synchronize .env, database directories, generated dependencies, build output, or provider credentials between machines.

Working principles

1. Make one coherent change at a time.
2. Record the existing behavior before intentionally changing it.
3. Keep dependency updates separate from UI redesign and functional changes.
4. Review and commit on Windows, then deploy and test the same commit on Rocky staging.
5. Use synthetic or sanitized data outside the workplace environment.
6. Never enable external provider schedules with placeholder credentials.
7. Do not connect staging to workplace services without explicit authorization.

Phase 0 — Reproducible development baseline

Status: in progress

• Review backend, frontend, deployment, and security structure.
• Run the project locally through Docker Desktop.
• Add configurable host port and scheduled-job switch.
• Add deterministic synthetic station data for the repository station set and the Faktiskā-required Valmiera position.
• Create an isolated Rocky Linux staging deployment.
• Keep staging PostgreSQL private to its Compose network.
• Commit the baseline changes and establish the Git-over-SSH workflow.
• Document normal build, seed, frontend deploy, source rollback, and current operational limitations. (Database backup/restore remains pending.)
• Capture representative screenshots and expected API responses.

Phase 1 — Behavior discovery and bug inventory

Status: in progress

• Walk through every page with synthetic data.
• Document the current understood purpose of each workflow; validate it with workplace users over time.
• Separate analytical dashboard features from broadcast-graphic authoring tools on the city-results map.
• Record unclear controls, missing units, broken states, and layout problems.
• Fix the atmPressire frontend field typo.
• Fix shifted Database export columns caused by duplicated tempMax.
• Correct malformed integer route handling.
• Add consistent loading, empty, and error states. (City selection empty state completed.)

Phase 2 — Test safety net

Status: in progress

• Add backend route tests for representative station and country queries.
• Add database integration tests for aggregation and export behavior.
• Restore and expand CSV parser tests.
• Add GRIB parser boundary and malformed-file tests.
• Add security tests for invalid fields, filenames, offsets, lengths, and date ranges.
• Add frontend type checking and critical workflow smoke tests.
• Run tests automatically before staging deployment.

Phase 3 — Dependency modernization

Status: in progress — frontend maintenance complete; backend maintenance pending

The first observed frontend install reported 15 vulnerabilities: 1 critical, 10 high, 3 moderate, and 1 low. The frontend dependency tree was reviewed in controlled groups, obsolete packages were removed, and both the complete and production-only npm audits now report zero known vulnerabilities at commit de6f279.

• Capture and review the full npm audit report.
• Establish that the production dependency audit is primarily blocked by solid-js@1.9.4 resolving vulnerable seroval@1.2.0; confirm that the full audit also contains development-tool advisories.
• Update direct frontend dependencies in controlled groups.
• Replace or remove obsolete frontend packages where appropriate.
• Add a committed TypeScript typecheck command and validate the updated frontend with a clean npm ci, typecheck, and production build.
• Resolve the remaining transitive build-tool advisories after reviewing the proposed npm audit fix changes; verify full and production-only audits at zero.
• Build, smoke-test, checksum, transfer, and deploy a commit-addressed VPS release containing de6f279 or later without restarting PostgreSQL or Authelia. (ef64895 deployed 2026-08-22.)
• Visually compare every page and representative exported PNG in the deployed dependency-maintenance release.
• Update Scala within the supported 2.13 line before considering larger migration.
• Update http4s, Doobie, Circe, Cats Effect, Logback, and test libraries in compatible groups.
• Replace release-candidate dependencies with stable releases where possible.
• Update Docker base images deliberately and pin reproducible versions.
• Verify database compatibility and generated artifacts after every group.

Dependency changes must not be combined with a visual redesign unless a package migration strictly requires it.

Phase 4 — Security hardening

Status: pending

• Rotate and remove the API key exposed in a source comment.
• Remove credentials from connection-error messages.
• Protect or remove debug and administrative endpoints.
• Convert state-changing GET routes to appropriate methods.
• Introduce closed, validated weather-field and aggregation types.
• Eliminate raw user-controlled SQL identifiers.
• Validate and constrain filenames, resolved paths, offsets, and byte lengths.
• Add query-range, response-size, request-rate, and timeout limits.
• Restrict CORS to intended origins.
• Define authentication and authorization requirements for workplace deployment.

Phase 5 — Runtime reliability and operations

Status: pending

• Manage custom executors as resources and close them cleanly.
• Remove explicit System.gc() calls.
• Supervise scheduled jobs independently instead of recursively restarting the application.
• Add application health and readiness endpoints.
• Add structured logging without leaking secrets.
• Define PostgreSQL and GRIB-data backup/restore procedures.
• Add container resource limits and deployment health checks.
• Document monitoring, update, rollback, and incident procedures.

Phase 6 — Information architecture and UI redesign

Status: pending

• Identify primary user roles and their most frequent tasks.
• Separate historical analysis, live station monitoring, database inspection, HARMONIE visualization, and broadcast graphics.
• Replace technical/internal labels with task-oriented language.
• Replace the copy/paste weather-character workflow with a direct visual font-glyph picker, bulk assignment, and city exceptions.
• Explain and visually separate manual wind and weather-icon inputs from queried data.
• Add units, legends, contextual help, and clear date semantics.
• Establish the first responsive layout, typography, spacing, and component-system foundation.
• Add an explicit PNG download workflow for broadcast map assets.
• Implement the fixed Faktiskā station set, latest-temperature loading, manual overrides, and locked 3840×1440 export.
• Make Faktiskā weather-symbol selection manual while keeping placement automatic and badge-relative.
• Replace Daira font glyphs with normalized 256×256 transparent image assets across the editor and canvas renderers.
• Add a separate Latvian-named Ūdens temperatūra workspace with fixed manual fields.
• Validate independent Ūdens temperatūra exports at 1920×1080 and 3840×1440 against the production specification.
• Bundle the confirmed Monda Regular/Bold files and replace temporary Rubik rendering.
• Perform pixel-level Monda comparison against the authoritative production masters.
• Test target resolutions and real workplace display conditions.
• Check keyboard navigation, contrast, focus states, and screen-reader labeling.

Phase 7 — Real data and production readiness

Status: pending

• Confirm the actual workplace deployment topology and current deployed commit.
• Obtain authorized development credentials or representative fixtures.
• Validate LVGMC station and forecast CSV ingestion.
• Validate DMI STAC discovery and EDR GRIB downloads.
• Test scheduled ingestion failure and recovery behavior.
• Rehearse deployment and rollback using sanitized data.
• Obtain technical and operational review before workplace rollout.

Phase 8 — Temporary VPS user acceptance

Status: in progress

• Document the proposed isolated VPS topology and trusted Rocky-to-VPS release flow.
• Choose laikapstak.li and auth.laikapstak.li, create their proxied Cloudflare DNS records, and activate Cloudflare delegation.
• Record a read-only VPS inventory before provisioning and confirm ports, networks, storage, and capacity do not collide with existing services.
• Provision a dedicated /srv/weathertool tree, Compose project, private network, database storage, and loopback-only application and Authelia ports.
• Configure Authelia with one temporary shared account, Argon2id password storage, rate limiting, and temporary IP bans.
• Add an exact-host Cloudflare origin certificate and Nginx authorization routing without disrupting the existing HOP site.
• Build and verify commit-addressed images on Rocky, transfer and checksum-verify them on the VPS, and update only the application service. Current full-SHA image: weathertool:ef6489549b990864c59a720f2ba5b25f5caaa4f1; PostgreSQL and Authelia were not restarted.
• Complete UAT verification: public authentication and application health pass; direct-origin blocking, logout, throttling, PNG downloads, backups, logs, and rollback remain.
• Run the month-long user test with manual releases and record feedback before any workplace-production decision.

Known current limitations

• The fixed production PNG bases, browser/social metadata, and reviewed frontend dependency updates are deployed in UAT release ef64895; newsroom workflow validation is in progress.

• Link-preview crawlers cannot authenticate through Authelia; the final Nginx policy must deliberately expose only the minimum preview metadata/assets if WhatsApp previews are required.

• Staging uses synthetic PostgreSQL station data.

• LVGMC forecast CSV fixtures are not yet available.

• HARMONIE GRIB fixtures are not yet available.

• Scheduled provider downloads are disabled in development and staging.

• Existing automated test coverage is minimal.

• Direct refreshes on newer frontend routes can return 404 until the backend gains a general SPA fallback.

• Full Docker build context scanning on Rocky can fail on the container-owned postgres/ bind directory; do not loosen its permissions.

Change log

Record completed work here by date and commit after the Git workflow is established.

Date	Commit	Summary	Verified on Rocky
2026-08-18	b1fff67	Docker development baseline, isolated staging, scheduler switch, and synthetic station data	Yes
2026-08-18	bb1d9fc	Home/UI foundation, map preview layout, friendly empty state, and PNG export	Yes
2026-08-18	67decb3	Adaptive temperature badges and title/source overlays	Yes
2026-08-18	82199b8	Separate City Analysis and Faktiskā workflows with direct symbol assignment	Yes
2026-08-18	0c78d3c	Optical centering for bundled weather-font glyphs	Pending Rocky verification
2026-08-19	e726291	Fixed Faktiskā workflow with latest observations, manual overrides, and production-size export	Yes
2026-08-19	5913217	Add Valmiera to the deterministic Faktiskā development data	Yes
2026-08-19	632f377	Arrange manual Faktiskā wind controls horizontally	Yes
2026-08-19	340c370–3aba9b4	Anchor and visually calibrate manually selected Faktiskā symbols	Yes
2026-08-19	1868041–6c9290b	Add Ūdens temperatūra, bundle and load Monda, fix overlay scaling, and center its locked nameplate	Yes
2026-08-19	0d641bd	Replace runtime Daira glyph rendering with normalized transparent image assets	Yes
2026-08-20	754415a	Match editable title/source overlays to production-safe right margins	Yes
2026-08-20	16a8c68	Deploy approved production map bases, normalized branding assets, favicon, Apple icon, and social metadata to public UAT	Yes — Rocky build and VPS/browser smoke checks
2026-08-21	5007517–de6f279	Update Solid runtime and frontend build tooling, add type checking, remove obsolete packages, prune the lockfile, and resolve all npm advisories	Yes — clean npm ci, typecheck, production build, full audit, and production-only audit
2026-08-22	ef64895	Deploy the reviewed zero-advisory frontend dependency state as an immutable full-SHA VPS release while leaving PostgreSQL, Authelia, and HOP uninterrupted	Yes — isolated Rocky smoke test, dual-host checksum, matching image ID, container health, loopback, authentication-gate, and HOP checks