Fix a real UTC-vs-local timezone mismatch in open-data station timestamps
Found while wiring up real FTP credentials: the open-data portal's DATETIME field is UTC, but was being stored into weather.dateTime as-is with no conversion — while the private FTP feed's "Laiks" column is already Latvia local time and also stored as-is. Both paths write the same column, so the table has been silently holding two timelines 2-3h apart since open-data went live, invisible only because FTP was never actually running with real data until today. Verified live: at real local time 20:31 EEST, the fix now correctly produces dateTime=20:00 instead of 17:00, matching what FTP writes for the same real hour.
This commit is contained in:
@@ -11,7 +11,7 @@ import org.typelevel.log4cats.Logger
|
||||
import org.typelevel.log4cats.slf4j.Slf4jLogger
|
||||
import parse.csv.{WeatherData, WeatherStationData}
|
||||
|
||||
import java.time.LocalDateTime
|
||||
import java.time.{LocalDateTime, ZoneId, ZoneOffset}
|
||||
import scala.util.Try
|
||||
|
||||
object OpenDataStationService {
|
||||
@@ -27,6 +27,14 @@ object OpenDataStationService {
|
||||
final class OpenDataStationService private (logger: Logger[IO]) {
|
||||
private val pageSize = 5000
|
||||
|
||||
// The open-data portal's DATETIME field is UTC (verified live against real
|
||||
// clock time 2026-08-23); the private FTP feed's "Laiks" column is already
|
||||
// Latvia local time with no conversion needed. Both write into the same
|
||||
// weather.dateTime column, so this path converts to match — otherwise the
|
||||
// two sources silently occupy two different, 2-3h-offset timelines in the
|
||||
// same table instead of one.
|
||||
private val rigaZone = ZoneId.of("Europe/Riga")
|
||||
|
||||
private val observationsUrl = sys.env.getOrElse(
|
||||
"LVGMC_OPENDATA_OBSERVATIONS_URL",
|
||||
"https://data.gov.lv/dati/api/3/action/datastore_search?resource_id=17460efb-ae99-4d1d-8144-1068f184b05f"
|
||||
@@ -113,7 +121,7 @@ final class OpenDataStationService private (logger: Logger[IO]) {
|
||||
city <- stationCities.get(stationId)
|
||||
abbreviation <- cursor.downField("ABBREVIATION").as[String].toOption
|
||||
datetimeStr <- cursor.downField("DATETIME").as[String].toOption
|
||||
timestamp <- Try(LocalDateTime.parse(datetimeStr)).toOption
|
||||
timestamp <- Try(LocalDateTime.parse(datetimeStr).atZone(ZoneOffset.UTC).withZoneSameInstant(rigaZone).toLocalDateTime).toOption
|
||||
value <- cursor.downField("VALUE").as[Double].toOption
|
||||
} yield ObservationCell(city, timestamp, abbreviation, value)
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user