diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 0000000..fa28652 --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,85 @@ +# Architecture and data flow + +## System overview + +WeatherTool is a Scala backend, SolidJS frontend, and PostgreSQL database used to inspect Latvian weather observations and prepare map-based newsroom graphics. + +```text +External providers Local/staging workflow +LVGMC station FTP/CSV ----+ Synthetic SQL seed +LVGMC forecast CSV -------+ | +DMI HARMONIE APIs --------+ v + v PostgreSQL + Scala ingestion | + | | + +---------> Scala/http4s API + | + v + SolidJS web UI + | + v + Canvas preview and PNG export +``` + +Staging currently exercises the PostgreSQL → API → frontend path with synthetic observations. It does not emulate LVGMC forecast files or HARMONIE GRIB files. + +## Components + +### Scala backend + +- Serves JSON API routes and the built frontend from `web/dist`. +- Queries and aggregates station observations in PostgreSQL. +- Contains ingestion/parsing code for LVGMC station and forecast data. +- Contains HARMONIE discovery, download, GRIB parsing, and rendering support. +- Starts scheduled ingestion/cleanup tasks unless `ENABLE_SCHEDULED_JOBS=false`. + +### SolidJS frontend + +- Queries the backend using the same origin in production builds. +- Provides Stations, City Analysis, Faktiskā, Latvia overview, Database, HARMONIE, and LVGMC forecast workspaces. +- Uses HTML canvas for broadcast maps and PNG export. +- Uses the bundled `ltv_meteo.otf` font for weather symbols; the glyphs are font characters, not emoji. + +### PostgreSQL + +- Stores normalized weather observations used by Stations, City Analysis, Faktiskā, and Latvia overview. +- Is accessible only inside the isolated Compose network in staging. +- Uses a bind-mounted `postgres/` runtime directory in the current Compose configuration. + +## Main observation-data flow + +1. An ingestion job parses provider station data, or the development seed inserts synthetic rows. +2. Observations are stored in the `weather` table. +3. The frontend requests an API route containing cities, time range, granularity, field, and aggregate key. +4. The backend performs the database query and returns JSON. +5. The frontend renders cards, lists, charts, or canvas graphics. +6. Broadcast maps can be downloaded as PNG files entirely in the browser. + +## Broadcast-map flow + +### City Analysis temperature maps + +Queried city values are placed at predefined coordinates on one of four map backgrounds: + +- `map_1920x1080` +- `map_1920x1080_wind` +- `map_3840x1440` +- `map_3840x1440_wind` + +The operator can add title/source overlays and export the rendered canvas. + +### Faktiskā + +Faktiskā queries temperature values, then lets the operator assign bundled weather-font glyphs to selected cities. One symbol can be applied to all cities and exceptions adjusted individually. Temperature badges, symbols, title, and source are composed into the downloaded PNG. + +## What the repository does not provide by itself + +- Authorized provider accounts and credentials. +- Workplace deployment topology or authentication requirements. +- Real LVGMC forecast CSV fixtures for development. +- Real HARMONIE GRIB fixtures for development. +- A complete automated test suite. +- Production monitoring, backups, incident handling, or a reviewed security boundary. + +See `UPDATE_ROADMAP.md` before considering workplace deployment. + diff --git a/docs/DEVELOPMENT_AND_STAGING.md b/docs/DEVELOPMENT_AND_STAGING.md new file mode 100644 index 0000000..c46bea1 --- /dev/null +++ b/docs/DEVELOPMENT_AND_STAGING.md @@ -0,0 +1,134 @@ +# Development and staging workflow + +## Environments + +| Environment | Purpose | Address | +|---|---|---| +| Windows | Source editing, local Docker build, and initial testing | `http://localhost:9190` | +| Rocky Linux | Production-like isolated staging | `http://192.168.1.101:9190` | +| Workplace production | Out of scope until reviewed and approved | Not documented here | + +Rocky checkout: + +```text +/home/sandbox/Documents/projects/WeatherTool +``` + +Private Rocky bare Git remote: + +```text +/home/sandbox/Documents/git/WeatherTool.git +``` + +Development branch: + +```text +codex/staging-baseline +``` + +## Safety boundaries + +- Do not commit `.env`, credentials, database data, `web/node_modules`, or `web/dist`. +- Keep `ENABLE_SCHEDULED_JOBS=false` with placeholder or development credentials. +- Do not publish the staging PostgreSQL port. +- Do not run `npm audit fix` without reviewing the proposed dependency changes. +- Do not change ownership or permissions of the container-managed `postgres/` directory as a deployment workaround. +- Do not use the current staging procedure as an undocumented workplace-production procedure. + +## Windows development + +Build the frontend using the project Node container: + +```powershell +docker compose run --rm node +``` + +Start or update the complete local stack when backend or Compose changes require it: + +```powershell +docker compose up --build -d +``` + +Seed deterministic observations: + +```powershell +docker compose --profile tools run --rm seed +``` + +Inspect status and logs: + +```powershell +docker compose ps +docker compose logs --tail 100 scala +``` + +## Commit and transfer workflow + +On Windows, review and commit a coherent change: + +```powershell +git status --short +git diff --check +git add +git commit -m "Describe the coherent change" +git push rocky codex/staging-baseline +``` + +On Rocky, update the staging checkout: + +```bash +cd /home/sandbox/Documents/projects/WeatherTool +git pull --ff-only +``` + +## Rocky frontend-only deployment + +For frontend-only changes, rebuild the bind-mounted production assets without restarting Scala or PostgreSQL: + +```bash +cd /home/sandbox/Documents/projects/WeatherTool +docker compose run --rm node +docker compose ps +curl -I http://localhost:9190/ +``` + +Use a hard browser refresh after deployment. + +## Rocky full-stack deployment + +Only use a full Compose rebuild when backend, dependency, image, or Compose changes require it: + +```bash +cd /home/sandbox/Documents/projects/WeatherTool +docker compose up --build -d +docker compose ps +``` + +Known limitation: the repository currently contains the bind-mounted `postgres/` data directory. On Rocky, Docker BuildKit can fail while scanning that container-owned directory even when it is listed in `.dockerignore`. Do not loosen database-directory permissions. The durable fix is to move database storage outside the build context or use a named volume. + +## Verification + +At minimum, verify: + +1. `docker compose ps` reports PostgreSQL healthy and Scala running. +2. `/` returns HTTP 200. +3. The home navigation loads. +4. City Analysis can query synthetic observations. +5. Each map resolution renders and downloads. +6. Faktiskā assigns font symbols, applies one to all, supports exceptions, and downloads a PNG. +7. Browser developer tools show no new runtime errors. + +Known limitation: direct browser refreshes on newer client-side routes such as `/faktiska` can return 404 because the backend static-route list does not yet provide a general SPA fallback. Navigate from the home page until that backend behavior is fixed. + +## Rollback + +Prefer a normal Git revert rather than manually copying old files: + +1. Identify the faulty commit with `git log --oneline`. +2. On the development checkout, run `git revert `. +3. Build and test the reverted state locally. +4. Push the revert commit. +5. Pull it on Rocky and run the appropriate frontend-only or full-stack deployment command. + +Database rollback is a separate operation and must not be inferred from a source rollback. Database backup and restore procedures are not yet defined. + diff --git a/docs/PRODUCT_WORKFLOWS.md b/docs/PRODUCT_WORKFLOWS.md new file mode 100644 index 0000000..14124ed --- /dev/null +++ b/docs/PRODUCT_WORKFLOWS.md @@ -0,0 +1,86 @@ +# Product workflows + +This document records the current understanding of the visible workspaces. It should be updated when workplace users clarify the real operational purpose. + +## Home + +Entry point and navigation overview. It identifies the development dataset and links to each workspace. + +## Stations + +Purpose: inspect station-focused observations, map values, and open detailed charts for a selected station. + +Current status: inherited workflow; still requires detailed workplace validation and UI review. + +## City Analysis + +Purpose: compare historical observations across selected cities. + +The operator chooses: + +- cities; +- start and end time; +- weather field; +- aggregate key; +- granularity. + +Results can be viewed as grid/list data or rendered into the familiar temperature-map formats. Weather-symbol authoring does not belong in this workflow. + +## Faktiskā + +Purpose: prepare the daily **faktiskā laika ziņu karte**—an actual-weather newsroom graphic containing: + +- title; +- source; +- temperatures; +- weather conditions represented by the bundled weather-font glyphs; +- 1920×1080 or 3840×1440 output; +- optional wind layout where required. + +Current operator flow: + +1. Select cities. +2. Select resolution and optional wind layout. +3. Choose the temperature field, calculation, and time range. +4. Load map data. +5. Enter title and source. +6. Select a weather glyph. +7. Apply it to all selected cities when appropriate. +8. Assign different glyphs to the exceptions. +9. Review the canvas and download PNG. + +The weather symbols are not automatically derived from the historical temperature query. They remain editorial/manual inputs until a reliable phenomena-data mapping is designed and validated. + +## Latvia overview + +Purpose: aggregate selected fields across Latvia for a selected time range. + +Current status: inherited workflow; terminology, units, and workplace use still need validation. + +## Data archive + +Purpose: inspect available database dates and individual stored records/files. + +Current status: inherited technical/administrative workflow. Access control and export correctness require review before production use. + +## HARMONIE + +Purpose: inspect and render forecast-model GRIB fields. + +Development limitation: no representative HARMONIE GRIB fixtures are currently included, and valid external-provider access is not configured in staging. + +## LVGMC forecast + +Purpose: display prepared LVGMC forecast products and broadcast tables. + +Development limitation: representative forecast CSV fixtures are not currently included, and valid external-provider access is not configured in staging. + +## Decisions still required + +- Exact workplace meaning and default query for Faktiskā temperatures. +- Which cities belong to each newsroom graphic by default. +- Default titles and source labels. +- Whether wind layouts belong to Faktiskā, City Analysis, or both. +- Whether weather conditions can eventually be populated automatically and which provider field is authoritative. +- Required authentication and role separation for analytical, production, and administrative workspaces. + diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..ff61c07 --- /dev/null +++ b/docs/README.md @@ -0,0 +1,28 @@ +# WeatherTool project documentation + +This directory contains the working documentation for the WeatherTool modernization effort. The repository-root `README.md` is preserved as the original project overview; these documents describe the reviewed code, current staging environment, and changes being developed. + +## Current status + +- Development source is edited and tested on Windows. +- A production-like staging copy runs through Docker Compose on Rocky Linux at `http://192.168.1.101:9190`. +- Staging uses deterministic synthetic station observations for all 33 stations. +- Scheduled external-provider jobs are disabled in development and staging. +- PostgreSQL is private to the project Compose network; only the Scala application publishes a host port. +- UI modernization is in progress. City Analysis retains temperature-map outputs, while Faktiskā is a separate temperature-and-weather-symbol production workflow. +- This is not yet approved or hardened for workplace production. + +## Documents + +- [Architecture and data flow](ARCHITECTURE.md) — components, data sources, data flow, and repository layout. +- [Development and staging](DEVELOPMENT_AND_STAGING.md) — Git workflow, local build, Rocky deployment, synthetic data, verification, and rollback. +- [Product workflows](PRODUCT_WORKFLOWS.md) — the intended purpose and current status of each visible workspace. +- [Update roadmap](UPDATE_ROADMAP.md) — phased technical, security, dependency, testing, and UI work. + +## Documentation rules + +1. Do not put passwords, API keys, workplace URLs, or production data in Git. +2. Document the behavior that exists separately from behavior that is proposed. +3. Update these documents in the same commit when a change alters deployment, data flow, or a user workflow. +4. Keep synthetic/development instructions clearly distinguished from workplace production procedures. + diff --git a/docs/UPDATE_ROADMAP.md b/docs/UPDATE_ROADMAP.md index 474ac32..753d506 100644 --- a/docs/UPDATE_ROADMAP.md +++ b/docs/UPDATE_ROADMAP.md @@ -32,7 +32,7 @@ Status: in progress - [x] Create an isolated Rocky Linux staging deployment. - [x] Keep staging PostgreSQL private to its Compose network. - [x] Commit the baseline changes and establish the Git-over-SSH workflow. -- [ ] Document normal build, seed, deploy, backup, and rollback commands. +- [x] Document normal build, seed, frontend deploy, source rollback, and current operational limitations. (Database backup/restore remains pending.) - [ ] Capture representative screenshots and expected API responses. ## Phase 1 — Behavior discovery and bug inventory @@ -40,7 +40,7 @@ Status: in progress Status: in progress - [ ] Walk through every page with synthetic data. -- [ ] Document the actual purpose and intended users of each workflow. +- [x] Document the current understood purpose of each workflow; validate it with workplace users over time. - [x] Separate analytical dashboard features from broadcast-graphic authoring tools on the city-results map. - [ ] Record unclear controls, missing units, broken states, and layout problems. - [ ] Fix the `atmPressire` frontend field typo. @@ -113,7 +113,7 @@ Status: pending - [ ] Identify primary user roles and their most frequent tasks. - [ ] Separate historical analysis, live station monitoring, database inspection, HARMONIE visualization, and broadcast graphics. - [ ] Replace technical/internal labels with task-oriented language. -- [ ] Replace the character-based weather-icon entry workflow with a visual picker or automatic mapping. +- [x] Replace the copy/paste weather-character workflow with a direct visual font-glyph picker, bulk assignment, and city exceptions. - [x] Explain and visually separate manual wind and weather-icon inputs from queried data. - [ ] Add units, legends, contextual help, and clear date semantics. - [x] Establish the first responsive layout, typography, spacing, and component-system foundation. @@ -140,7 +140,8 @@ Status: pending - HARMONIE GRIB fixtures are not yet available. - Scheduled provider downloads are disabled in development and staging. - Existing automated test coverage is minimal. -- The current UI combines analysis and broadcast-authoring concepts without explanation. +- Direct refreshes on newer frontend routes can return 404 until the backend gains a general SPA fallback. +- Full Docker build context scanning on Rocky can fail on the container-owned `postgres/` bind directory; do not loosen its permissions. ## Change log @@ -149,3 +150,7 @@ Record completed work here by date and commit after the Git workflow is establis | Date | Commit | Summary | Verified on Rocky | |---|---|---|---| | 2026-08-18 | `b1fff67` | Docker development baseline, isolated staging, scheduler switch, and synthetic station data | Yes | +| 2026-08-18 | `bb1d9fc` | Home/UI foundation, map preview layout, friendly empty state, and PNG export | Yes | +| 2026-08-18 | `67decb3` | Adaptive temperature badges and title/source overlays | Yes | +| 2026-08-18 | `82199b8` | Separate City Analysis and Faktiskā workflows with direct symbol assignment | Yes | +| 2026-08-18 | `0c78d3c` | Optical centering for bundled weather-font glyphs | Pending Rocky verification |