Revive project at new apartment — fixes, SSID update, gitignore
WiFi/firmware: - SSID updated to botnet in both node and ble_node config.h - Stale coordinator comment removed from node/config.h Coordinator fixes: - BLE-only nodes now visible in dashboard sidebar and node detail (build_nodes was querying beacon_events only; BLE nodes have no beacons) - ble_events and heartbeat_events added to pruning cycle - ble_events table added to ensure_schema() in dashboard - confidence column dropped from beacon_events (always 'high', never queried) - Ingestor commits batched per packet instead of per store call - wal_autocheckpoint=500 added to ingestor DB connection (writer was missing it) - python3 -u added to both service ExecStart lines (stdout was buffered, logs not appearing in journalctl) Repo hygiene: - .gitignore added (events.db, __pycache__, build artifacts) - events.db removed from git tracking - README updated: new apartment, node table, active investigations, BLE observations labelled as previous-location data, status checklist Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
+20
@@ -0,0 +1,20 @@
|
|||||||
|
# Database — lives on the Orange Pi only, never committed
|
||||||
|
coordinator/events.db
|
||||||
|
coordinator/events.db-shm
|
||||||
|
coordinator/events.db-wal
|
||||||
|
|
||||||
|
# Python
|
||||||
|
__pycache__/
|
||||||
|
*.pyc
|
||||||
|
*.pyo
|
||||||
|
|
||||||
|
# Arduino build artifacts
|
||||||
|
build/
|
||||||
|
*.elf
|
||||||
|
*.bin
|
||||||
|
*.map
|
||||||
|
|
||||||
|
# Editor / OS
|
||||||
|
.DS_Store
|
||||||
|
*.swp
|
||||||
|
*.swo
|
||||||
@@ -32,19 +32,19 @@ This is a learning/research project covering distributed systems, event-driven a
|
|||||||
### Active nodes
|
### Active nodes
|
||||||
|
|
||||||
| node_id | MAC | Firmware | Location |
|
| node_id | MAC | Firmware | Location |
|
||||||
|----------|-------------------|----------|------------------------------|
|
|----------|-------------------|----------|------------------|
|
||||||
| F68D6E30 | 44:1b:f6:8d:6e:30 | WiFi | room 1 (permanent) |
|
| A1D658D4 | e0:72:a1:d6:58:d4 | WiFi | unplaced |
|
||||||
| A1D658D4 | e0:72:a1:d6:58:d4 | WiFi | room 2 (permanent) |
|
| A1D6F190 | e0:72:a1:d6:f1:90 | WiFi | unplaced |
|
||||||
| A1D700C4 | e0:72:a1:d7:00:c4 | BLE | room 3 (permanent) |
|
| A1D700C4 | e0:72:a1:d7:00:c4 | BLE | unplaced |
|
||||||
| A1D6F190 | e0:72:a1:d6:f1:90 | BLE | dev machine (desk, USB) |
|
| F68D6E30 | 44:1b:f6:8d:6e:30 | BLE | unplaced |
|
||||||
|
|
||||||
WiFi nodes stay on their existing firmware. BLE nodes are reflashed with `firmware/ble_node/`.
|
All four nodes reflashed 2026-05-21 (new apartment, SSID updated to `botnet`). WiFi: A1D658D4, A1D6F190. BLE: A1D700C4, F68D6E30.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Network
|
## Network
|
||||||
|
|
||||||
- **ESP32 nodes connect to:** WiFi SSID `sandbox`
|
- **ESP32 nodes connect to:** WiFi SSID `botnet`
|
||||||
- **Coordinator IP:** `192.168.1.133` (Orange Pi, production)
|
- **Coordinator IP:** `192.168.1.133` (Orange Pi, production)
|
||||||
- **UDP port:** `5005`
|
- **UDP port:** `5005`
|
||||||
- **Dashboard port:** `8080`
|
- **Dashboard port:** `8080`
|
||||||
@@ -106,9 +106,12 @@ ssh root@192.168.1.133 'journalctl -u dashboard -f'
|
|||||||
```
|
```
|
||||||
esp32_cluster/
|
esp32_cluster/
|
||||||
├── firmware/
|
├── firmware/
|
||||||
│ └── node/
|
│ ├── node/
|
||||||
│ ├── node.ino # ESP32 node firmware
|
│ │ ├── node.ino # ESP32 WiFi node firmware
|
||||||
│ └── config.h # WiFi creds, coordinator IP, scan/probe config
|
│ │ └── config.h # WiFi creds, coordinator IP, scan/probe config
|
||||||
|
│ └── ble_node/
|
||||||
|
│ ├── ble_node.ino # ESP32 BLE node firmware
|
||||||
|
│ └── config.h # BLE scan/flush/dedup config
|
||||||
├── coordinator/
|
├── coordinator/
|
||||||
│ ├── udp_ingest.py # UDP listener — receives packets, validates, writes to SQLite
|
│ ├── udp_ingest.py # UDP listener — receives packets, validates, writes to SQLite
|
||||||
│ ├── dashboard.py # FastAPI app — reads SQLite, serves UI + SSE stream
|
│ ├── dashboard.py # FastAPI app — reads SQLite, serves UI + SSE stream
|
||||||
@@ -316,6 +319,7 @@ The ESP32 radio is shared between WiFi and BLE via the hardware coexistence cont
|
|||||||
| 0x0499 | Ruuvi Innovations |
|
| 0x0499 | Ruuvi Innovations |
|
||||||
| 0x0157 | Polar Electro |
|
| 0x0157 | Polar Electro |
|
||||||
| 0x00BD | Fitbit |
|
| 0x00BD | Fitbit |
|
||||||
|
| 0x0057 | Harman International (JBL, AKG) |
|
||||||
| 0x0171 | Amazon |
|
| 0x0171 | Amazon |
|
||||||
|
|
||||||
Apple type byte (first byte of `mfr_data` when `mfr_id` = 76):
|
Apple type byte (first byte of `mfr_data` when `mfr_id` = 76):
|
||||||
@@ -459,13 +463,10 @@ To avoid investigating our own infrastructure, these SSIDs and BSSIDs are ours:
|
|||||||
|
|
||||||
| SSID | Band | Purpose |
|
| SSID | Band | Purpose |
|
||||||
|-----------|--------|--------------------------------|
|
|-----------|--------|--------------------------------|
|
||||||
| `sandbox` | 2.4GHz | Main network, nodes connect here|
|
| `botnet` | 2.4GHz | Main network — nodes connect here, IoT devices |
|
||||||
| `botnet` | 2.4GHz | IoT devices |
|
|
||||||
| `pronet` | 5GHz | Main 5GHz network |
|
| `pronet` | 5GHz | Main 5GHz network |
|
||||||
| `mango` | 2.4GHz | Secondary network |
|
| `mango` | 2.4GHz | Secondary network |
|
||||||
|
|
||||||
`sandbox` BSSID `1C:3B:F3:9C:AC:30` appearing in deauth/impersonation data is expected — our own nodes briefly deauth from it during channel hopping and reconnect.
|
|
||||||
|
|
||||||
### OUI lookup
|
### OUI lookup
|
||||||
|
|
||||||
The `oui.txt` file is the IEEE public OUI database (39,171 entries as of download). It maps the first 3 bytes of a real MAC to a manufacturer name. Used in the Clients tab. Refresh it occasionally by re-running `deploy.sh` after downloading a fresh copy from `https://standards-oui.ieee.org/oui/oui.txt`.
|
The `oui.txt` file is the IEEE public OUI database (39,171 entries as of download). It maps the first 3 bytes of a real MAC to a manufacturer name. Used in the Clients tab. Refresh it occasionally by re-running `deploy.sh` after downloading a fresh copy from `https://standards-oui.ieee.org/oui/oui.txt`.
|
||||||
@@ -474,22 +475,58 @@ The `oui.txt` file is the IEEE public OUI database (39,171 entries as of downloa
|
|||||||
|
|
||||||
## Active investigations
|
## Active investigations
|
||||||
|
|
||||||
### Sustained deauth attack on Tuya device
|
### Living Room speaker
|
||||||
~~A persistent automated deauth flood has been running since **2026-04-03**~~
|
SSID: `Living Room speaker.n078` · BSSID: `FA:8F:CA:76:06:B2` · ch 6 · OPEN · RSSI -57 dBm.
|
||||||
|
|
||||||
**Status as of 2026-04-25: attack has wound down.** Down from hundreds of frames/day to 1–7 frames/day. The direct target `38:2C:E5:7E:77:1D` is no longer being hit. Two of the four attacker MACs (`82:4E:66:47:09:C1`, `62:87:CB:38:2C:20`) have gone silent. `42:8C:46:6E:12:9C` and `62:D9:AA:E8:B4:09` still appear occasionally at very low volume. Conclusion: whoever was running the tool either captured the WPA2 handshake they needed or moved away. Keep in watch list for re-activation.
|
Spotted again on first scan at new apartment (2026-05-21). Same BSSID as previous location — either the device moved with us, or a neighbour in the new building has the same model. Broadcasting an open unauthenticated provisioning hotspot. Pending: connect via Parrot laptop + Alfa adapter, nmap the interface.
|
||||||
|
|
||||||
Attacker src MACs: `82:4E:66:47:09:C1`, `42:8C:46:6E:12:9C`, `62:D9:AA:E8:B4:09`, `62:87:CB:38:2C:20`
|
### Previous location — deauth investigation (archived)
|
||||||
|
A sustained deauth flood targeting a Tuya device ran 2026-04-03 to ~2026-04-25, then wound down. Attacker MACs: `82:4E:66:47:09:C1`, `42:8C:46:6E:12:9C`, `62:D9:AA:E8:B4:09`, `62:87:CB:38:2C:20`. Not expected to reappear at new location.
|
||||||
### Living Room speaker (own device)
|
|
||||||
SSID: `Living Room speaker.n078` · BSSID: `FA:8F:CA:76:06:B2` · ch 6 · OPEN · RSSI -54 dBm (bathroom, same apartment).
|
|
||||||
|
|
||||||
Broadcasting an open unauthenticated hotspot. Next step: connect via Parrot laptop + Alfa adapter, nmap the provisioning interface, document what is exposed. Not yet started.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## BLE observations and notes
|
## BLE observations and notes
|
||||||
|
|
||||||
|
> Data below is from the **previous location** (2026-04-26 to 2026-05-02, 6 days). Retained as reference — some devices and patterns will differ at the new apartment.
|
||||||
|
|
||||||
|
### BLE data summary (previous location)
|
||||||
|
|
||||||
|
Data collected from BLE launch (2026-04-26) to project shutdown (2026-05-02) — 6 days, 2 nodes (A1D700C4, F68D6E30).
|
||||||
|
|
||||||
|
| Metric | Value |
|
||||||
|
|--------|-------|
|
||||||
|
| Total BLE events | 1,086,897 |
|
||||||
|
| Unique MACs observed | 89,818 |
|
||||||
|
| Public (stable) MACs | 1,392 |
|
||||||
|
| Random (rotating) MACs | 88,426 |
|
||||||
|
| Named devices identified | 1,486 |
|
||||||
|
| Peak day | 2026-04-29 — 209,589 events, 18,308 unique MACs |
|
||||||
|
| Node A1D700C4 (room 3) | 675,951 events, 85,535 unique MACs |
|
||||||
|
| Node F68D6E30 (room 1) | 411,086 events, 46,906 unique MACs |
|
||||||
|
|
||||||
|
**Manufacturer breakdown (top by event count):**
|
||||||
|
|
||||||
|
| Company ID | Vendor | Events | Unique MACs |
|
||||||
|
|------------|--------|--------|-------------|
|
||||||
|
| 0x004C | Apple | 686,109 | 67,987 |
|
||||||
|
| 0x0075 | Samsung | 143,520 | 957 |
|
||||||
|
| 0x0006 | Microsoft | 53,794 | 2,336 |
|
||||||
|
| 0x0057 | Harman/JBL | 27,930 | 47 |
|
||||||
|
|
||||||
|
Apple accounts for ~63% of all BLE traffic. The high Apple unique MAC count is mostly FindMy/AirTag MAC rotation — individual physical devices inflate the count significantly.
|
||||||
|
|
||||||
|
**Apple type byte breakdown:**
|
||||||
|
|
||||||
|
| Type | Description | Events | Unique MACs |
|
||||||
|
|------|-------------|--------|-------------|
|
||||||
|
| `0x12` | FindMy / AirTag (MAC rotates per advertisement) | 271,336 | 26,137 |
|
||||||
|
| `0x10` | Proximity Pair (iPhone advertising nearby) | 228,086 | 28,992 |
|
||||||
|
| `0x16` | Nearby Info (iPhone battery/status) | 78,848 | 4,375 |
|
||||||
|
| `0x06` | Unknown type — 1 stable MAC, continuous all-day | 33,306 | 3 |
|
||||||
|
| `0x07` | HomeKit accessory | 32,774 | 4,449 |
|
||||||
|
| `0x09` | AirPods (in use / case open) | 22,447 | 1,852 |
|
||||||
|
| `0x0c` | AirPods case / Watch (lid open or unlocked) | 11,420 | 2,800 |
|
||||||
|
|
||||||
### Heap baseline
|
### Heap baseline
|
||||||
|
|
||||||
BLE nodes floor out at **109–127K free heap** vs 222K+ for WiFi nodes. The ~100K difference is the BLE stack overhead (BLEDevice + BLEScan + coexistence controller). This is expected and stable — no downward drift observed after 21 hours. The WiFi-only queue flush model keeps heap use predictable.
|
BLE nodes floor out at **109–127K free heap** vs 222K+ for WiFi nodes. The ~100K difference is the BLE stack overhead (BLEDevice + BLEScan + coexistence controller). This is expected and stable — no downward drift observed after 21 hours. The WiFi-only queue flush model keeps heap use predictable.
|
||||||
@@ -517,16 +554,29 @@ Most common types seen in practice:
|
|||||||
| `09`/`0a`/`0f` | AirPods (various gen) | Seen when case is open or buds are in use |
|
| `09`/`0a`/`0f` | AirPods (various gen) | Seen when case is open or buds are in use |
|
||||||
| `02` | iBeacon | Retail/venue tracking beacons |
|
| `02` | iBeacon | Retail/venue tracking beacons |
|
||||||
|
|
||||||
### Identified permanent BLE neighbours (as of 2026-04-26)
|
### Identified permanent BLE neighbours (as of 2026-05-02)
|
||||||
|
|
||||||
| Device | MAC type | Pattern | Notes |
|
| Device | MAC type | Pattern | Notes |
|
||||||
|--------|----------|---------|-------|
|
|--------|----------|---------|-------|
|
||||||
| Sony WH-1000XM5 | Random (rotates ~15min) | All day + late night | Likely 1 device; user wears them most of the day, leaves connected overnight |
|
| Sony WH-1000XM5 | Random (rotates ~15min) | Most days, all hours | 327 unique MACs observed across 6 days — confirmed 1 device by rotation pattern |
|
||||||
| B&W PX5 headphones | Public — stable | Weekday afternoons ~12:00–18:00 | Consistent with WFH office hours |
|
| Sony WH-1000XM4 | Random (rotates ~15min) | Most days | 57 unique MACs — distinct neighbour from the XM5 user |
|
||||||
| ELK-BLEDOM LED strip | Public — stable | Evenings | BLE-controlled RGB LED strip, cheap Chinese controller |
|
| B&W PX5 headphones | Public — stable | Sunday Apr 26, RSSI ~−85–99 | Appeared on a Sunday rather than expected weekday pattern; may also be present on weekdays at lower RSSI |
|
||||||
| Audio-Technica ATH-M50xBT2 | Public — stable | Afternoon sessions | Professional wireless headphones |
|
| ELK-BLEDOM LED strip | Public — stable | Apr 26 evenings only | Sporadic; appeared one evening then went quiet |
|
||||||
| `0x5148` device (c2:fe:68:e8:01:a6) | Public — stable | Persistent all day | Unknown company ID, payload ASCII "364656", best RSSI −56 dBm — very close, identity unknown |
|
| Audio-Technica ATH-M50xBT2 | Public — stable (2 MACs) | Multiple days, afternoon sessions | Two separate units or one with two stable MACs |
|
||||||
| "net" (80:3e:4f:1b:4f:e3) | Public — stable | 07:00–20:00 | Short device name, likely a smart home hub or IoT device |
|
| Govee H6609 LED strip (`Govee_H6609_4D1A`) | Random — stable | Multiple days, persistent | BLE-controlled RGB strip, Govee controller in nearby unit |
|
||||||
|
| Creative Bowie MA10 speaker | Public — stable (41:aa:66:90:99:d7) | All 6 days, evenings | Consistent presence; best RSSI −82 dBm |
|
||||||
|
| Sennheiser MOMENTUM 4 | Public — stable (80:c3:ba:86:18:e9) | Multiple days | Professional over-ear headphones |
|
||||||
|
| JBL speaker cluster | Mixed (Clip 5, Flip 5 ×2, Charge 5/6) | Occasional, low RSSI ~−93–99 | Multiple different JBL models across different days — building has several JBL users |
|
||||||
|
| `0x5148` device (c2:fe:68:e8:01:a6) | Public — stable | All 6 days, all hours | Unknown company ID 0x5148, payload fixed ASCII "364656", best RSSI −55 dBm — very close, identity still unknown |
|
||||||
|
| "net" (80:3e:4f:1b:4f:e3) | Public — stable | Apr 26/28/29 only, very weak −96–100 | Appears to have moved or powered off; was previously more consistent |
|
||||||
|
|
||||||
|
### Samsung SmartTag cluster
|
||||||
|
|
||||||
|
Three stable public Samsung MACs (`8c:79:f5:a6:dd:c3`, `68:72:c3:bd:ed:6f`, `7c:64:56:84:e0:b3`) generate 30K–36K events each across all 6 days of BLE data. All share the same mfr_data prefix (`42040180...`) — Samsung's Galaxy Find Network advertisement format (type byte `0x42`). These are Samsung SmartTag2 trackers belonging to one or more neighbours; stable public MACs make them directly trackable. RSSI −73 to −100 dBm suggests they are in a nearby unit or parking area below.
|
||||||
|
|
||||||
|
### Apple type 0x06 — persistent unknown device
|
||||||
|
|
||||||
|
MAC `c7:73:2d:eb:1a:b1` (random but stable) generates 30,765 events across all 6 days with consistent daily volume of 3K–5K events. Apple type byte `0x06` does not appear in Apple's published proximity pairing spec; this is an undocumented continuity or accessory advertisement type. The MAC has not rotated over 6 days, which is unusual for a random-type address. Likely a HomeKit accessory or Apple TV in a neighbouring unit that advertises continuously. Best RSSI −83 dBm.
|
||||||
|
|
||||||
### Parking structure BLE
|
### Parking structure BLE
|
||||||
|
|
||||||
@@ -548,9 +598,11 @@ For reference during future BLE correlation work, the known open networks in ran
|
|||||||
|
|
||||||
## Current status
|
## Current status
|
||||||
|
|
||||||
|
**Project revived 2026-05-21** — new apartment, fresh start. All four nodes reflashed and online, coordinator running on Orange Pi, database clean.
|
||||||
|
|
||||||
- [x] Arduino CLI installed, ESP32 core configured
|
- [x] Arduino CLI installed, ESP32 core configured
|
||||||
- [x] Node firmware: beacon scan + probe sniffing
|
- [x] Node firmware: beacon scan + probe sniffing
|
||||||
- [x] Four ESP32-S3 nodes flashed and running
|
- [x] Four ESP32-S3 nodes flashed and running (2× WiFi, 2× BLE)
|
||||||
- [x] UDP ingestor (`udp_ingest.py`) — beacon + probe routing with field validation
|
- [x] UDP ingestor (`udp_ingest.py`) — beacon + probe routing with field validation
|
||||||
- [x] Web dashboard: Feed, Networks, Cross-node, Clients, Alerts, Search, Presence, Node detail
|
- [x] Web dashboard: Feed, Networks, Cross-node, Clients, Alerts, Search, Presence, Node detail
|
||||||
- [x] OUI vendor lookup in Clients view (vendor grouping + randomized MAC detection)
|
- [x] OUI vendor lookup in Clients view (vendor grouping + randomized MAC detection)
|
||||||
@@ -580,9 +632,16 @@ For reference during future BLE correlation work, the known open networks in ran
|
|||||||
- [x] BLE firmware (`firmware/ble_node/`) — passive scan, dedup, mfr data parsing, heartbeat
|
- [x] BLE firmware (`firmware/ble_node/`) — passive scan, dedup, mfr data parsing, heartbeat
|
||||||
- [x] Coordinator BLE support — `ble_events` table, ingest handler, `ble_drops` in heartbeat
|
- [x] Coordinator BLE support — `ble_events` table, ingest handler, `ble_drops` in heartbeat
|
||||||
- [x] Dashboard BLE tab — devices table, manufacturer breakdown, live feed
|
- [x] Dashboard BLE tab — devices table, manufacturer breakdown, live feed
|
||||||
|
- [x] BLE nodes now visible in dashboard sidebar and node detail (previously invisible — heartbeat-only nodes not queried)
|
||||||
|
- [x] `ble_events` and `heartbeat_events` added to pruning cycle (previously accumulated unbounded)
|
||||||
|
- [x] `ble_events` table added to `ensure_schema()` in dashboard (previously only created by ingestor)
|
||||||
|
- [x] Ingestor commits batched per packet instead of per store call; `wal_autocheckpoint=500` added to writer connection
|
||||||
|
- [x] Python stdout unbuffered in both service files (`python3 -u`) — logs now appear in journalctl in real time
|
||||||
|
- [x] `confidence` column dropped from `beacon_events` (was always `"high"`, never queried)
|
||||||
- [ ] Surface assoc events in dashboard (Search results or dedicated view)
|
- [ ] Surface assoc events in dashboard (Search results or dedicated view)
|
||||||
- [ ] Scan interval control from dashboard
|
- [ ] Scan interval control from dashboard
|
||||||
- [ ] Dedicated AP for nodes to isolate reconnect churn from regular network traffic
|
- [ ] Dedicated AP for nodes to isolate reconnect churn from regular network traffic
|
||||||
|
- [ ] Place and label nodes in new apartment rooms
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|||||||
+88
-10
@@ -196,7 +196,9 @@ def build_nodes() -> list[dict]:
|
|||||||
now = datetime.datetime.utcnow().timestamp()
|
now = datetime.datetime.utcnow().timestamp()
|
||||||
if _nodes_cache and (now - _nodes_cache[0]) < _NODES_CACHE_TTL:
|
if _nodes_cache and (now - _nodes_cache[0]) < _NODES_CACHE_TTL:
|
||||||
return _nodes_cache[1]
|
return _nodes_cache[1]
|
||||||
rows = query("""
|
|
||||||
|
# WiFi nodes: beacon-derived stats
|
||||||
|
beacon_rows = query("""
|
||||||
SELECT
|
SELECT
|
||||||
node_id,
|
node_id,
|
||||||
MAX(received_at) AS last_beacon,
|
MAX(received_at) AS last_beacon,
|
||||||
@@ -204,9 +206,9 @@ def build_nodes() -> list[dict]:
|
|||||||
MIN(received_at) AS first_seen
|
MIN(received_at) AS first_seen
|
||||||
FROM beacon_events
|
FROM beacon_events
|
||||||
GROUP BY node_id
|
GROUP BY node_id
|
||||||
ORDER BY node_id ASC
|
|
||||||
""")
|
""")
|
||||||
# Latest heartbeat per node
|
|
||||||
|
# Latest heartbeat per node — covers ALL node types (WiFi and BLE)
|
||||||
hb_rows = query("""
|
hb_rows = query("""
|
||||||
SELECT node_id, received_at AS last_heartbeat,
|
SELECT node_id, received_at AS last_heartbeat,
|
||||||
uptime_ms, free_heap, wifi_rssi
|
uptime_ms, free_heap, wifi_rssi
|
||||||
@@ -214,6 +216,21 @@ def build_nodes() -> list[dict]:
|
|||||||
WHERE id IN (SELECT MAX(id) FROM heartbeat_events GROUP BY node_id)
|
WHERE id IN (SELECT MAX(id) FROM heartbeat_events GROUP BY node_id)
|
||||||
""")
|
""")
|
||||||
hb_map = {r["node_id"]: r for r in hb_rows}
|
hb_map = {r["node_id"]: r for r in hb_rows}
|
||||||
|
|
||||||
|
node_map: dict = {r["node_id"]: r for r in beacon_rows}
|
||||||
|
|
||||||
|
# BLE-only nodes have no beacon events — add them from heartbeat data
|
||||||
|
for node_id, hb in hb_map.items():
|
||||||
|
if node_id not in node_map:
|
||||||
|
node_map[node_id] = {
|
||||||
|
"node_id": node_id,
|
||||||
|
"last_beacon": None,
|
||||||
|
"total_events": 0,
|
||||||
|
"first_seen": hb.get("last_heartbeat"),
|
||||||
|
}
|
||||||
|
|
||||||
|
rows = sorted(node_map.values(), key=lambda r: r["node_id"])
|
||||||
|
|
||||||
for r in rows:
|
for r in rows:
|
||||||
hb = hb_map.get(r["node_id"], {})
|
hb = hb_map.get(r["node_id"], {})
|
||||||
r["last_heartbeat"] = hb.get("last_heartbeat")
|
r["last_heartbeat"] = hb.get("last_heartbeat")
|
||||||
@@ -221,11 +238,11 @@ def build_nodes() -> list[dict]:
|
|||||||
r["free_heap"] = hb.get("free_heap")
|
r["free_heap"] = hb.get("free_heap")
|
||||||
r["wifi_rssi"] = hb.get("wifi_rssi")
|
r["wifi_rssi"] = hb.get("wifi_rssi")
|
||||||
r["last_seen"] = r["last_heartbeat"] or r["last_beacon"]
|
r["last_seen"] = r["last_heartbeat"] or r["last_beacon"]
|
||||||
# Online if heartbeat within 30s; fall back to beacon within 60s if no heartbeat yet
|
|
||||||
if r["last_heartbeat"]:
|
if r["last_heartbeat"]:
|
||||||
r["status"] = _node_status(r["last_heartbeat"], threshold=30)
|
r["status"] = _node_status(r["last_heartbeat"], threshold=30)
|
||||||
else:
|
else:
|
||||||
r["status"] = _node_status(r["last_beacon"], threshold=60)
|
r["status"] = _node_status(r["last_beacon"], threshold=60)
|
||||||
|
|
||||||
_nodes_cache = (now, rows)
|
_nodes_cache = (now, rows)
|
||||||
return rows
|
return rows
|
||||||
|
|
||||||
@@ -973,6 +990,7 @@ def build_search(q: str) -> dict:
|
|||||||
|
|
||||||
|
|
||||||
def build_node_detail(node_id: str) -> dict | None:
|
def build_node_detail(node_id: str) -> dict | None:
|
||||||
|
# WiFi node path — beacon-derived stats
|
||||||
rows = query("""
|
rows = query("""
|
||||||
SELECT
|
SELECT
|
||||||
node_id,
|
node_id,
|
||||||
@@ -988,9 +1006,51 @@ def build_node_detail(node_id: str) -> dict | None:
|
|||||||
WHERE node_id = ?
|
WHERE node_id = ?
|
||||||
GROUP BY node_id
|
GROUP BY node_id
|
||||||
""", (node_id,))
|
""", (node_id,))
|
||||||
if not rows:
|
|
||||||
return None
|
if rows:
|
||||||
detail = rows[0]
|
detail = rows[0]
|
||||||
|
detail["node_type"] = "wifi"
|
||||||
|
else:
|
||||||
|
# BLE node path — stats from ble_events
|
||||||
|
ble_rows = query("""
|
||||||
|
SELECT
|
||||||
|
node_id,
|
||||||
|
MIN(received_at) AS first_seen,
|
||||||
|
MAX(received_at) AS last_seen,
|
||||||
|
COUNT(*) AS total_events,
|
||||||
|
COUNT(DISTINCT mac) AS unique_macs,
|
||||||
|
AVG(rssi) AS avg_rssi,
|
||||||
|
MIN(rssi) AS min_rssi,
|
||||||
|
MAX(rssi) AS max_rssi
|
||||||
|
FROM ble_events
|
||||||
|
WHERE node_id = ?
|
||||||
|
GROUP BY node_id
|
||||||
|
""", (node_id,))
|
||||||
|
|
||||||
|
if ble_rows:
|
||||||
|
detail = ble_rows[0]
|
||||||
|
else:
|
||||||
|
# Node exists (heartbeats only) but no scan data yet
|
||||||
|
hb_check = query(
|
||||||
|
"SELECT node_id FROM heartbeat_events WHERE node_id = ? LIMIT 1", (node_id,)
|
||||||
|
)
|
||||||
|
if not hb_check:
|
||||||
|
return None
|
||||||
|
detail = {
|
||||||
|
"node_id": node_id,
|
||||||
|
"first_seen": None,
|
||||||
|
"last_seen": None,
|
||||||
|
"total_events": 0,
|
||||||
|
"unique_macs": 0,
|
||||||
|
"avg_rssi": None,
|
||||||
|
"min_rssi": None,
|
||||||
|
"max_rssi": None,
|
||||||
|
}
|
||||||
|
|
||||||
|
detail["node_type"] = "ble"
|
||||||
|
detail["unique_ssids"] = 0
|
||||||
|
detail["unique_bssids"] = 0
|
||||||
|
|
||||||
hb_rows = query("""
|
hb_rows = query("""
|
||||||
SELECT received_at AS last_heartbeat, uptime_ms, free_heap, wifi_rssi
|
SELECT received_at AS last_heartbeat, uptime_ms, free_heap, wifi_rssi
|
||||||
FROM heartbeat_events
|
FROM heartbeat_events
|
||||||
@@ -1005,8 +1065,8 @@ def build_node_detail(node_id: str) -> dict | None:
|
|||||||
if detail["last_heartbeat"]:
|
if detail["last_heartbeat"]:
|
||||||
detail["status"] = _node_status(detail["last_heartbeat"], threshold=30)
|
detail["status"] = _node_status(detail["last_heartbeat"], threshold=30)
|
||||||
else:
|
else:
|
||||||
detail["status"] = _node_status(detail["last_seen"], threshold=60)
|
detail["status"] = _node_status(detail.get("last_seen"), threshold=60)
|
||||||
detail["avg_rssi"] = round(detail["avg_rssi"], 1) if detail["avg_rssi"] else None
|
detail["avg_rssi"] = round(detail["avg_rssi"], 1) if detail.get("avg_rssi") else None
|
||||||
detail["events"] = build_events(limit=EVENT_CAP, node_id=node_id)
|
detail["events"] = build_events(limit=EVENT_CAP, node_id=node_id)
|
||||||
return detail
|
return detail
|
||||||
|
|
||||||
@@ -1019,7 +1079,7 @@ def ensure_schema():
|
|||||||
CREATE TABLE IF NOT EXISTS beacon_events (
|
CREATE TABLE IF NOT EXISTS beacon_events (
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT, received_at TEXT NOT NULL,
|
id INTEGER PRIMARY KEY AUTOINCREMENT, received_at TEXT NOT NULL,
|
||||||
node_id TEXT, node_ts INTEGER, ssid TEXT, bssid TEXT,
|
node_id TEXT, node_ts INTEGER, ssid TEXT, bssid TEXT,
|
||||||
rssi INTEGER, channel INTEGER, encryption TEXT, importance TEXT, confidence TEXT
|
rssi INTEGER, channel INTEGER, encryption TEXT, importance TEXT
|
||||||
)
|
)
|
||||||
""")
|
""")
|
||||||
conn.execute("""
|
conn.execute("""
|
||||||
@@ -1063,6 +1123,20 @@ def ensure_schema():
|
|||||||
rssi INTEGER
|
rssi INTEGER
|
||||||
)
|
)
|
||||||
""")
|
""")
|
||||||
|
conn.execute("""
|
||||||
|
CREATE TABLE IF NOT EXISTS ble_events (
|
||||||
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
|
received_at TEXT NOT NULL,
|
||||||
|
node_id TEXT,
|
||||||
|
node_ts INTEGER,
|
||||||
|
mac TEXT,
|
||||||
|
addr_type TEXT,
|
||||||
|
name TEXT,
|
||||||
|
rssi INTEGER,
|
||||||
|
mfr_id INTEGER,
|
||||||
|
mfr_data TEXT
|
||||||
|
)
|
||||||
|
""")
|
||||||
conn.executescript("""
|
conn.executescript("""
|
||||||
CREATE INDEX IF NOT EXISTS idx_beacon_received_at ON beacon_events (received_at);
|
CREATE INDEX IF NOT EXISTS idx_beacon_received_at ON beacon_events (received_at);
|
||||||
CREATE INDEX IF NOT EXISTS idx_beacon_node_id ON beacon_events (node_id);
|
CREATE INDEX IF NOT EXISTS idx_beacon_node_id ON beacon_events (node_id);
|
||||||
@@ -1083,6 +1157,10 @@ def ensure_schema():
|
|||||||
CREATE INDEX IF NOT EXISTS idx_assoc_node_id ON assoc_events (node_id);
|
CREATE INDEX IF NOT EXISTS idx_assoc_node_id ON assoc_events (node_id);
|
||||||
CREATE INDEX IF NOT EXISTS idx_assoc_src ON assoc_events (src);
|
CREATE INDEX IF NOT EXISTS idx_assoc_src ON assoc_events (src);
|
||||||
CREATE INDEX IF NOT EXISTS idx_assoc_bssid ON assoc_events (bssid);
|
CREATE INDEX IF NOT EXISTS idx_assoc_bssid ON assoc_events (bssid);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_ble_received_at ON ble_events (received_at);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_ble_node_id ON ble_events (node_id);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_ble_mac ON ble_events (mac);
|
||||||
|
CREATE INDEX IF NOT EXISTS idx_ble_mfr_id ON ble_events (mfr_id);
|
||||||
""")
|
""")
|
||||||
conn.commit()
|
conn.commit()
|
||||||
|
|
||||||
@@ -1094,7 +1172,7 @@ CHECKPOINT_INTERVAL_SECS = 30 * 60 # force WAL checkpoint every 30 minutes
|
|||||||
def prune_old_events() -> None:
|
def prune_old_events() -> None:
|
||||||
"""Delete events older than HOT_DAYS from all event tables, then checkpoint."""
|
"""Delete events older than HOT_DAYS from all event tables, then checkpoint."""
|
||||||
cutoff = _hot_cutoff()
|
cutoff = _hot_cutoff()
|
||||||
tables = ["beacon_events", "probe_events", "deauth_events", "assoc_events"]
|
tables = ["beacon_events", "probe_events", "deauth_events", "assoc_events", "ble_events", "heartbeat_events"]
|
||||||
with get_conn() as conn:
|
with get_conn() as conn:
|
||||||
for table in tables:
|
for table in tables:
|
||||||
conn.execute(f"DELETE FROM {table} WHERE received_at < ?", (cutoff,))
|
conn.execute(f"DELETE FROM {table} WHERE received_at < ?", (cutoff,))
|
||||||
|
|||||||
@@ -5,7 +5,7 @@ After=network.target
|
|||||||
[Service]
|
[Service]
|
||||||
Type=simple
|
Type=simple
|
||||||
WorkingDirectory=/opt/esp32_cluster/coordinator
|
WorkingDirectory=/opt/esp32_cluster/coordinator
|
||||||
ExecStart=/usr/bin/python3 dashboard.py
|
ExecStart=/usr/bin/python3 -u dashboard.py
|
||||||
Restart=always
|
Restart=always
|
||||||
RestartSec=5
|
RestartSec=5
|
||||||
StandardOutput=journal
|
StandardOutput=journal
|
||||||
|
|||||||
Binary file not shown.
@@ -21,6 +21,7 @@ DB_PATH = os.path.join(os.path.dirname(__file__), "events.db")
|
|||||||
def init_db(path: str) -> sqlite3.Connection:
|
def init_db(path: str) -> sqlite3.Connection:
|
||||||
conn = sqlite3.connect(path, check_same_thread=False)
|
conn = sqlite3.connect(path, check_same_thread=False)
|
||||||
conn.execute("PRAGMA journal_mode=WAL")
|
conn.execute("PRAGMA journal_mode=WAL")
|
||||||
|
conn.execute("PRAGMA wal_autocheckpoint=500")
|
||||||
conn.execute("""
|
conn.execute("""
|
||||||
CREATE TABLE IF NOT EXISTS beacon_events (
|
CREATE TABLE IF NOT EXISTS beacon_events (
|
||||||
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
||||||
@@ -32,8 +33,7 @@ def init_db(path: str) -> sqlite3.Connection:
|
|||||||
rssi INTEGER,
|
rssi INTEGER,
|
||||||
channel INTEGER,
|
channel INTEGER,
|
||||||
encryption TEXT,
|
encryption TEXT,
|
||||||
importance TEXT,
|
importance TEXT
|
||||||
confidence TEXT
|
|
||||||
)
|
)
|
||||||
""")
|
""")
|
||||||
conn.execute("""
|
conn.execute("""
|
||||||
@@ -139,8 +139,8 @@ def init_db(path: str) -> sqlite3.Connection:
|
|||||||
def store_beacon(conn: sqlite3.Connection, ev: dict, received_at: str):
|
def store_beacon(conn: sqlite3.Connection, ev: dict, received_at: str):
|
||||||
conn.execute("""
|
conn.execute("""
|
||||||
INSERT INTO beacon_events
|
INSERT INTO beacon_events
|
||||||
(received_at, node_id, node_ts, ssid, bssid, rssi, channel, encryption, importance, confidence)
|
(received_at, node_id, node_ts, ssid, bssid, rssi, channel, encryption, importance)
|
||||||
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||||
""", (
|
""", (
|
||||||
received_at,
|
received_at,
|
||||||
ev.get("node_id"),
|
ev.get("node_id"),
|
||||||
@@ -151,7 +151,6 @@ def store_beacon(conn: sqlite3.Connection, ev: dict, received_at: str):
|
|||||||
ev.get("ch"),
|
ev.get("ch"),
|
||||||
ev.get("enc"),
|
ev.get("enc"),
|
||||||
ev.get("imp"),
|
ev.get("imp"),
|
||||||
ev.get("conf"),
|
|
||||||
))
|
))
|
||||||
conn.commit()
|
conn.commit()
|
||||||
|
|
||||||
@@ -174,7 +173,6 @@ def store_heartbeat(conn: sqlite3.Connection, ev: dict, received_at: str):
|
|||||||
ev.get("assoc_drops"),
|
ev.get("assoc_drops"),
|
||||||
ev.get("ble_drops"),
|
ev.get("ble_drops"),
|
||||||
))
|
))
|
||||||
conn.commit()
|
|
||||||
|
|
||||||
|
|
||||||
def store_probe(conn: sqlite3.Connection, ev: dict, received_at: str):
|
def store_probe(conn: sqlite3.Connection, ev: dict, received_at: str):
|
||||||
@@ -191,7 +189,6 @@ def store_probe(conn: sqlite3.Connection, ev: dict, received_at: str):
|
|||||||
ev.get("rssi"),
|
ev.get("rssi"),
|
||||||
ev.get("imp"),
|
ev.get("imp"),
|
||||||
))
|
))
|
||||||
conn.commit()
|
|
||||||
|
|
||||||
# ─── Validation ──────────────────────────────────────────────────────────────
|
# ─── Validation ──────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
@@ -244,7 +241,6 @@ def store_deauth(conn: sqlite3.Connection, ev: dict, received_at: str):
|
|||||||
ev.get("reason"),
|
ev.get("reason"),
|
||||||
ev.get("rssi"),
|
ev.get("rssi"),
|
||||||
))
|
))
|
||||||
conn.commit()
|
|
||||||
|
|
||||||
|
|
||||||
def validate_deauth(ev: dict) -> str | None:
|
def validate_deauth(ev: dict) -> str | None:
|
||||||
@@ -279,7 +275,6 @@ def store_assoc(conn: sqlite3.Connection, ev: dict, received_at: str):
|
|||||||
ev.get("ssid", ""),
|
ev.get("ssid", ""),
|
||||||
ev.get("rssi"),
|
ev.get("rssi"),
|
||||||
))
|
))
|
||||||
conn.commit()
|
|
||||||
|
|
||||||
|
|
||||||
def validate_assoc(ev: dict) -> str | None:
|
def validate_assoc(ev: dict) -> str | None:
|
||||||
@@ -312,7 +307,6 @@ def store_ble(conn: sqlite3.Connection, ev: dict, received_at: str):
|
|||||||
mfr_id if mfr_id is not None and mfr_id >= 0 else None,
|
mfr_id if mfr_id is not None and mfr_id >= 0 else None,
|
||||||
ev.get("mfr_data", ""),
|
ev.get("mfr_data", ""),
|
||||||
))
|
))
|
||||||
conn.commit()
|
|
||||||
|
|
||||||
|
|
||||||
def validate_ble(ev: dict) -> str | None:
|
def validate_ble(ev: dict) -> str | None:
|
||||||
@@ -438,6 +432,8 @@ def handle_packet(data: bytes, addr: tuple, conn: sqlite3.Connection):
|
|||||||
color = IMP_COLOR.get(imp, "")
|
color = IMP_COLOR.get(imp, "")
|
||||||
print(f"{color}[{received_at}] {node} {ssid:<32} {bssid} ch{ch:<3} {rssi:>4}dBm {enc}{RESET}")
|
print(f"{color}[{received_at}] {node} {ssid:<32} {bssid} ch{ch:<3} {rssi:>4}dBm {enc}{RESET}")
|
||||||
|
|
||||||
|
conn.commit()
|
||||||
|
|
||||||
# ─── Main ────────────────────────────────────────────────────────────────────
|
# ─── Main ────────────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
def main():
|
def main():
|
||||||
|
|||||||
@@ -5,7 +5,7 @@ After=network.target
|
|||||||
[Service]
|
[Service]
|
||||||
Type=simple
|
Type=simple
|
||||||
WorkingDirectory=/opt/esp32_cluster/coordinator
|
WorkingDirectory=/opt/esp32_cluster/coordinator
|
||||||
ExecStart=/usr/bin/python3 udp_ingest.py
|
ExecStart=/usr/bin/python3 -u udp_ingest.py
|
||||||
Restart=always
|
Restart=always
|
||||||
RestartSec=5
|
RestartSec=5
|
||||||
StandardOutput=journal
|
StandardOutput=journal
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
#pragma once
|
#pragma once
|
||||||
|
|
||||||
// ─── WiFi (for UDP transport only) ───────────────────────────────────────────
|
// ─── WiFi (for UDP transport only) ───────────────────────────────────────────
|
||||||
#define WIFI_SSID "sandbox"
|
#define WIFI_SSID "botnet"
|
||||||
#define WIFI_PASSWORD "Jaunsgads11!!"
|
#define WIFI_PASSWORD "Jaunsgads11!!"
|
||||||
|
|
||||||
// ─── Coordinator ─────────────────────────────────────────────────────────────
|
// ─── Coordinator ─────────────────────────────────────────────────────────────
|
||||||
|
|||||||
@@ -1,12 +1,10 @@
|
|||||||
#pragma once
|
#pragma once
|
||||||
|
|
||||||
// ─── WiFi ────────────────────────────────────────────────────────────────────
|
// ─── WiFi ────────────────────────────────────────────────────────────────────
|
||||||
#define WIFI_SSID "sandbox"
|
#define WIFI_SSID "botnet"
|
||||||
#define WIFI_PASSWORD "Jaunsgads11!!"
|
#define WIFI_PASSWORD "Jaunsgads11!!"
|
||||||
|
|
||||||
// ─── Coordinator ─────────────────────────────────────────────────────────────
|
// ─── Coordinator ─────────────────────────────────────────────────────────────
|
||||||
// This PC (192.168.1.101) runs coordinator.py during the test phase.
|
|
||||||
// Will change to Orange Pi (192.168.1.133) when we migrate.
|
|
||||||
#define COORDINATOR_IP "192.168.1.133"
|
#define COORDINATOR_IP "192.168.1.133"
|
||||||
#define COORDINATOR_PORT 5005
|
#define COORDINATOR_PORT 5005
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user