Add isolated staging and synthetic weather workflow

docs/UPDATE_ROADMAP.md

@@ -0,0 +1,151 @@
+# WeatherTool update roadmap
+
+This document tracks proposed WeatherTool improvements. Work should be delivered in small, reviewable phases rather than as one large rewrite. Each phase should leave the application runnable and independently testable.
+
+## Environments and workflow
+
+- **Windows development:** primary source-editing and UI-review workspace.
+- **Rocky staging:** production-like Docker deployment at `http://192.168.1.101:9190`.
+- **Git over SSH:** Windows pushes reviewed commits to a private bare repository on Rocky; the Rocky staging checkout pulls those commits and rebuilds.
+- **Production:** remains separate until changes are reviewed, tested, and explicitly approved for workplace use.
+
+Do not synchronize `.env`, database directories, generated dependencies, build output, or provider credentials between machines.
+
+## Working principles
+
+1. Make one coherent change at a time.
+2. Record the existing behavior before intentionally changing it.
+3. Keep dependency updates separate from UI redesign and functional changes.
+4. Test on Windows, then deploy the same commit to Rocky staging.
+5. Use synthetic or sanitized data outside the workplace environment.
+6. Never enable external provider schedules with placeholder credentials.
+7. Do not connect staging to workplace services without explicit authorization.
+
+## Phase 0 — Reproducible development baseline
+
+Status: in progress
+
+- [x] Review backend, frontend, deployment, and security structure.
+- [x] Run the project locally through Docker Desktop.
+- [x] Add configurable host port and scheduled-job switch.
+- [x] Add deterministic synthetic station data for all 33 stations.
+- [x] Create an isolated Rocky Linux staging deployment.
+- [x] Keep staging PostgreSQL private to its Compose network.
+- [ ] Commit the baseline changes and establish the Git-over-SSH workflow.
+- [ ] Document normal build, seed, deploy, backup, and rollback commands.
+- [ ] Capture representative screenshots and expected API responses.
+
+## Phase 1 — Behavior discovery and bug inventory
+
+Status: pending
+
+- [ ] Walk through every page with synthetic data.
+- [ ] Document the actual purpose and intended users of each workflow.
+- [ ] Separate analytical dashboard features from broadcast-graphic authoring tools.
+- [ ] Record unclear controls, missing units, broken states, and layout problems.
+- [ ] Fix the `atmPressire` frontend field typo.
+- [ ] Fix shifted Database export columns caused by duplicated `tempMax`.
+- [ ] Correct malformed integer route handling.
+- [ ] Add consistent loading, empty, and error states.
+
+## Phase 2 — Test safety net
+
+Status: pending
+
+- [ ] Add backend route tests for representative station and country queries.
+- [ ] Add database integration tests for aggregation and export behavior.
+- [ ] Restore and expand CSV parser tests.
+- [ ] Add GRIB parser boundary and malformed-file tests.
+- [ ] Add security tests for invalid fields, filenames, offsets, lengths, and date ranges.
+- [ ] Add frontend type checking and critical workflow smoke tests.
+- [ ] Run tests automatically before staging deployment.
+
+## Phase 3 — Dependency modernization
+
+Status: pending
+
+The first observed frontend install reported 15 vulnerabilities: 1 critical, 10 high, 3 moderate, and 1 low. Exact advisories must be reviewed before choosing upgrades.
+
+- [ ] Capture and review the full npm audit report.
+- [ ] Update direct frontend dependencies in controlled groups.
+- [ ] Replace or remove obsolete frontend packages where appropriate.
+- [ ] Rebuild and visually compare every page after frontend upgrades.
+- [ ] Update Scala within the supported 2.13 line before considering larger migration.
+- [ ] Update http4s, Doobie, Circe, Cats Effect, Logback, and test libraries in compatible groups.
+- [ ] Replace release-candidate dependencies with stable releases where possible.
+- [ ] Update Docker base images deliberately and pin reproducible versions.
+- [ ] Verify database compatibility and generated artifacts after every group.
+
+Dependency changes must not be combined with a visual redesign unless a package migration strictly requires it.
+
+## Phase 4 — Security hardening
+
+Status: pending
+
+- [ ] Rotate and remove the API key exposed in a source comment.
+- [ ] Remove credentials from connection-error messages.
+- [ ] Protect or remove debug and administrative endpoints.
+- [ ] Convert state-changing `GET` routes to appropriate methods.
+- [ ] Introduce closed, validated weather-field and aggregation types.
+- [ ] Eliminate raw user-controlled SQL identifiers.
+- [ ] Validate and constrain filenames, resolved paths, offsets, and byte lengths.
+- [ ] Add query-range, response-size, request-rate, and timeout limits.
+- [ ] Restrict CORS to intended origins.
+- [ ] Define authentication and authorization requirements for workplace deployment.
+
+## Phase 5 — Runtime reliability and operations
+
+Status: pending
+
+- [ ] Manage custom executors as resources and close them cleanly.
+- [ ] Remove explicit `System.gc()` calls.
+- [ ] Supervise scheduled jobs independently instead of recursively restarting the application.
+- [ ] Add application health and readiness endpoints.
+- [ ] Add structured logging without leaking secrets.
+- [ ] Define PostgreSQL and GRIB-data backup/restore procedures.
+- [ ] Add container resource limits and deployment health checks.
+- [ ] Document monitoring, update, rollback, and incident procedures.
+
+## Phase 6 — Information architecture and UI redesign
+
+Status: pending
+
+- [ ] Identify primary user roles and their most frequent tasks.
+- [ ] Separate historical analysis, live station monitoring, database inspection, HARMONIE visualization, and broadcast graphics.
+- [ ] Replace technical/internal labels with task-oriented language.
+- [ ] Replace the character-based weather-icon entry workflow with a visual picker or automatic mapping.
+- [ ] Explain or automate manual wind and weather-icon inputs.
+- [ ] Add units, legends, contextual help, and clear date semantics.
+- [ ] Establish a responsive layout, typography, spacing, and component system.
+- [ ] Design explicit export/download workflows for broadcast assets.
+- [ ] Test target resolutions and real workplace display conditions.
+- [ ] Check keyboard navigation, contrast, focus states, and screen-reader labeling.
+
+## Phase 7 — Real data and production readiness
+
+Status: pending
+
+- [ ] Confirm the actual workplace deployment topology and current deployed commit.
+- [ ] Obtain authorized development credentials or representative fixtures.
+- [ ] Validate LVGMC station and forecast CSV ingestion.
+- [ ] Validate DMI STAC discovery and EDR GRIB downloads.
+- [ ] Test scheduled ingestion failure and recovery behavior.
+- [ ] Rehearse deployment and rollback using sanitized data.
+- [ ] Obtain technical and operational review before workplace rollout.
+
+## Known current limitations
+
+- Staging uses synthetic PostgreSQL station data.
+- LVGMC forecast CSV fixtures are not yet available.
+- HARMONIE GRIB fixtures are not yet available.
+- Scheduled provider downloads are disabled in development and staging.
+- Existing automated test coverage is minimal.
+- The current UI combines analysis and broadcast-authoring concepts without explanation.
+
+## Change log
+
+Record completed work here by date and commit after the Git workflow is established.
+
+| Date | Commit | Summary | Verified on Rocky |
+|---|---|---|---|
+| 2026-08-18 | pending | Docker development baseline, isolated staging, scheduler switch, and synthetic station data | Yes |