Revert VPS FTP after it crash-looped the app; document the diagnosis

The ltv account fails to authenticate from the VPS specifically
(works fine from Rocky with the same credentials, network path to
ftp.meteo.lv:21 confirmed fine from the VPS too) — narrowed to either
a password transcription error in .env.staging or LVGMC IP-allowlisting
the account, neither confirmable without inspecting the password.
Caught via the overnight watch this was set up for rather than
assumed safe; reverted ENABLE_LVGMC_FTP_JOBS to false on the VPS
immediately once the failure was confirmed real, restoring stable
operation. FTP stays enabled on Rocky, where it works.
This commit is contained in:
b0txec
2026-08-23 22:14:58 +03:00
parent 9a6cb30552
commit 1086c8b548
3 changed files with 49 additions and 13 deletions
+39 -9
View File
@@ -7,7 +7,7 @@ This document tracks proposed WeatherTool improvements. Work should be delivered
- **Windows source workspace:** source editing, review, and Git operations only; do not install dependencies, compile, build, run, or test here.
- **Rocky development and staging:** the sole compile, build, development-runtime, and test environment, with production-like Docker staging at `http://192.168.1.101:9190`.
- **Git over SSH:** Windows pushes reviewed commits to a private bare repository on Rocky; the Rocky staging checkout pulls those commits and rebuilds.
- **Ubuntu VPS deployment:** release `3eddf95` is publicly operational behind Cloudflare strict TLS, Nginx, and Authelia, ingesting real LVĢMC open-data station/water-temperature observations on a schedule, plus real LVGMC FTP station data as of 2026-08-23 (`ENABLE_LVGMC_FTP_JOBS=true`, real credentials); the `weather` table was wiped and re-populated fresh once the UTC/local timezone mismatch between the two sources was fixed. The VPS does not compile or build the project.
- **Ubuntu VPS deployment:** release `3eddf95` is publicly operational behind Cloudflare strict TLS, Nginx, and Authelia, ingesting real LVĢMC open-data station/water-temperature observations on a schedule; the `weather` table was wiped and re-populated fresh once the UTC/local timezone mismatch between sources was fixed. FTP (`ENABLE_LVGMC_FTP_JOBS`) was briefly enabled 2026-08-23 but reverted to `false` the same night after the `ltv` account failed to authenticate from the VPS specifically (crash-looped the app in-process until caught and reverted) — real credentials work fine from Rocky, not yet from the VPS; see the changelog for full diagnosis. The VPS does not compile or build the project.
- **Workplace production:** remains separate until changes are reviewed, tested, and explicitly approved for workplace use.
Do not synchronize `.env`, database directories, generated dependencies, build output, or provider credentials between machines.
@@ -241,14 +241,44 @@ running in parallel until each real source is proven, not cut over in one step.
had no effect until this was switched to the same `${VAR}` substitution
pattern. VPS now runs release `3eddf95`/`dc04f66` with real FTP
credentials (added directly on the VPS by the user, never typed into
this session) and `ENABLE_LVGMC_FTP_JOBS=true`; watching the container
logs for VPS's first scheduled FTP fetch specifically (not just the
open-data job, which fires more often and was initially mistaken for
it on one premature check) to confirm real rows land before leaving it
unattended overnight — outbound FTP from the VPS network is a real,
non-hypothetical risk worth checking, not something safe to assume
just because it worked from Rocky. **Result pending as of this
writing — update this line once confirmed.**
this session) and `ENABLE_LVGMC_FTP_JOBS=true`. Good thing this was
watched instead of assumed safe: the VPS's first scheduled FTP fetch
(19:13 UTC) failed with `Failed to login to FTP server`, and because
`parMapN` cancels every sibling task the instant one throws, this
cascaded all the way up through `Main.scala`'s top-level
`handleErrorWith` — which tears down and rebuilds the *entire* app
(DB connections, HTTP server, schedulers) and retries every ~5s. Not a
Docker-level restart (`RestartCount` stayed `0`, since the JVM process
itself never exited) but a real in-process crash-loop on the public
site, repeating every ~2 minutes as the FTP schedule re-fired. Reverted
`ENABLE_LVGMC_FTP_JOBS` to `false` on the VPS immediately (the user was
already asleep; this is undoing a flag flipped minutes earlier as part
of the same authorized change, not new destructive action, so didn't
wait) — confirmed stable afterward, no fatal errors, public site
responding normally again.
Diagnosed before reverting further: from the VPS, `ftp.meteo.lv:21` is
reachable and returns a normal vsFTPd banner, and an anonymous login
attempt correctly gets rejected (`530 Login incorrect`) — so this is
**not** a network/firewall problem, outbound FTP works fine from the
VPS. The real `ltv` account specifically fails to authenticate from
the VPS despite working from Rocky a couple of hours earlier with the
same credentials. Two candidate causes, neither confirmed (didn't
inspect the actual password value to check):
1. A transcription error when the password was typed into the VPS's
`.env.staging` (manual entry, real risk for a long decoded string).
2. LVGMC IP-allowlists the `ltv` account to specific source addresses
— common for partner/business FTP accounts — and the VPS's outbound
IP isn't on that list even though Rocky's apparently is. The VPS's
outbound IP is `57.128.250.38`, in case it needs to be given to the
LVGMC contact for allowlisting.
**Next step (morning, user): double-check the password in
`/srv/weathertool/.env.staging` on the VPS character-for-character
against the decoded value; if it's correct, ask the LVGMC contact
whether the `ltv` account is IP-restricted and whether the VPS's
outbound IP needs adding.** FTP stays on for Rocky (working there);
VPS runs open-data only until this is resolved and re-verified the
same careful way — watch-and-confirm, not assume-and-move-on.
- [x] Researched DMI HARMONIE credentials while waiting on the above and
confirmed live (not just from search results, which claimed a specific
date that wasn't independently verified): DMI's Forecast Data EDR and