bot/esp32-cluster
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix dashboard latency and add DB pruning

Browse Source 
- Wrap all build_* route calls in asyncio.to_thread() so SQLite queries
  run in a thread pool instead of blocking the event loop. Fixes tab
  switch freezes and SSE stream stalls.
- Add 60s TTL caches to alerts, sessions, presence, and cross-node
  endpoints (previously uncached, queried on every request).
- Scope alerts queries (top targets, devices, reason breakdown, heatmap)
  to last 7 days instead of all-time full table scans.
- Add composite indexes: (bssid, received_at) on beacon_events and
  deauth_events, (src_mac, received_at) on probe_events.
- Add background pruning task: runs every 6 hours, deletes events older
  than HOT_DAYS from all tables, followed by WAL checkpoint.
- Document the full diagnosis and fixes in README.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
bot
2026-04-06 13:01:13 +03:00
parent fee23f0f66
commit 36c7248d0e
2 changed files with 241 additions and 74 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+74 -9
View File
@@ -25,15 +25,18 @@ This is a learning/research project covering distributed systems, event-driven a
- **Coordinator:** Orange Pi at `192.168.1.133`, runs 24/7 as systemd services
- **Dev machine:** This PC at `192.168.1.101` — coding and flashing only
- **Flashing:** Arduino CLI on this PC
- **Pen testing:** Parrot OS laptop at `192.168.1.124` (user: `keny`) — active investigation rig
- **WiFi adapter:** Alfa MT7612U (`wlx00c0cab67193`) — monitor mode + packet injection, plugged into Parrot laptop
- **Pen testing tools:** airgeddon at `/home/keny/Documents/github_tools/airgeddon`
### Active nodes
| node_id | MAC | Location |
|----------|-------------------|-------------------------|
| F68D6E30 | 44:1b:f6:8d:6e:30 | room 1 (permanent) |
| A1D658D4 | e0:72:a1:d6:58:d4 | room 2 (permanent) |
| A1D700C4 | e0:72:a1:d7:00:c4 | dev machine /dev/ttyACM2|
| A1D6F190 | e0:72:a1:d6:f1:90 | dev machine /dev/ttyACM1|
| node_id | MAC | Location |
|----------|-------------------|------------------------------|
| F68D6E30 | 44:1b:f6:8d:6e:30 | room 1 (permanent) |
| A1D658D4 | e0:72:a1:d6:58:d4 | room 2 (permanent) |
| A1D700C4 | e0:72:a1:d7:00:c4 | room 3 (permanent) |
| A1D6F190 | e0:72:a1:d6:f1:90 | dev machine (desk, USB) |
---
@@ -43,6 +46,7 @@ This is a learning/research project covering distributed systems, event-driven a
- **Coordinator IP:** `192.168.1.133` (Orange Pi, production)
- **UDP port:** `5005`
- **Dashboard port:** `8080`
- **Parrot laptop:** `192.168.1.124` — SSH as `keny`, passwordless sudo configured
---
@@ -345,12 +349,39 @@ The **Bursts** section is the anomaly detector. A single unique source MAC gener
Multi-node confirmation (`node_count > 1`, highlighted red) significantly raises confidence — it means the source is physically close and strong, not a distant weak signal.
### Our own networks
To avoid investigating our own infrastructure, these SSIDs and BSSIDs are ours:
| SSID | Band | Purpose |
|-----------|--------|--------------------------------|
| `sandbox` | 2.4GHz | Main network, nodes connect here|
| `botnet` | 2.4GHz | IoT devices |
| `pronet` | 5GHz | Main 5GHz network |
| `mango` | 2.4GHz | Secondary network |
`sandbox` BSSID `1C:3B:F3:9C:AC:30` appearing in deauth/impersonation data is expected — our own nodes briefly deauth from it during channel hopping and reconnect.
### OUI lookup
The `oui.txt` file is the IEEE public OUI database (39,171 entries as of download). It maps the first 3 bytes of a real MAC to a manufacturer name. Used in the Clients tab. Refresh it occasionally by re-running `deploy.sh` after downloading a fresh copy from `https://standards-oui.ieee.org/oui/oui.txt`.
---
## Active investigations
### Sustained deauth attack on Tuya device
A persistent automated deauth flood has been running since **2026-04-03**, targeting `38:2C:E5:7E:77:1D` (Tuya Smart Inc. device). Four spoofed source MACs fire simultaneously roughly every hour, all using reason code 2, impersonating real AP BSSIDs in the building. All 4 nodes confirm it. Consistent with an automated WPA2 handshake capture tool. No action taken yet — being passively monitored.
Attacker src MACs: `82:4E:66:47:09:C1`, `42:8C:46:6E:12:9C`, `62:D9:AA:E8:B4:09`, `62:87:CB:38:2C:20`
### Living Room speaker (own device)
SSID: `Living Room speaker.n078` · BSSID: `FA:8F:CA:76:06:B2` · ch 6 · OPEN · RSSI -54 dBm (bathroom, same apartment).
Broadcasting an open unauthenticated hotspot. Next step: connect via Parrot laptop + Alfa adapter, nmap the provisioning interface, document what is exposed. Not yet started.
---
## Current status
- [x] Arduino CLI installed, ESP32 core configured
@@ -375,13 +406,47 @@ The `oui.txt` file is the IEEE public OUI database (39,171 entries as of downloa
- [x] Queue drop counters — tracked per queue in firmware, reported in heartbeat, stored in DB
- [x] SQLite indexes — received_at, node_id, bssid, src_mac, src, dst across all event tables
- [x] Dashboard query caps — 300-row limits on Clients, Networks, Cross-node to keep UI responsive
- [ ] Attack session reconstruction — group deauth events into discrete sessions by source/time
- [ ] Surface assoc events in dashboard (Sessions tab or Search results)
- [ ] DB pruning / retention policy (events.db grows indefinitely)
- [x] Sessions tab — deauth events grouped into sessions by source MAC + 2-min gap, sortable, expandable rows
- [x] Dashboard performance overhaul — fixed tab switch latency and Networks tab failing to load (see below)
- [x] DB pruning / retention policy — background task deletes events older than HOT_DAYS every 6 hours
- [ ] Surface assoc events in dashboard (Search results or dedicated view)
- [ ] Scan interval control from dashboard
---
## Dashboard performance overhaul (2026-04-06)
### Problem
Tab switches took up to a minute to load. The Networks tab failed to load entirely. The SSE live feed would stall during tab switches. The root cause was three compounding issues:
**1. Blocking the async event loop.**
FastAPI routes are `async def`, but the SQLite calls (`sqlite3` module) are fully synchronous. When called directly inside `async def`, they block uvicorn's entire event loop — meaning while one slow query runs, the server cannot serve any other request, including the SSE stream. This is why everything froze together.
**2. Expensive endpoints had no caching.**
`/api/alerts`, `/api/sessions`, `/api/presence`, and `/api/cross-node` ran full database queries on every single request, with no result caching. The 2-second debounce on the frontend meant these were being called repeatedly.
`build_sessions()` was the worst: it pulled every deauth event from the last 30 days and grouped them into sessions in a Python loop — O(N) in Python on every call, with no cache. Under a sustained deauth flood (thousands of events/day), this was very slow.
`build_alerts()` ran several queries against the **entire** `deauth_events` table with no time cutoff — top targets, top targeted devices, reason breakdown, activity heatmap all scanned all-time data.
**3. Unbounded database growth.**
No pruning meant every query got slower every day as the tables grew. The database had also developed B-tree corruption (double-referenced pages, out-of-order rowids), likely from WAL journal not checkpointing cleanly under sustained write load. The database was rebuilt clean.
### Fixes applied
- **`asyncio.to_thread()`** — all `build_*` calls in every route are now dispatched to a thread pool executor. The event loop stays free to handle SSE and other requests while queries run in the background. The SSE generator's inline `query()` calls were also fixed the same way.
- **Caching added** — alerts (60s TTL), sessions (60s), presence (60s), cross-node (60s) now cache results in memory. The existing nodes/networks/clients caches were kept. Tab switches hit the cache on repeated loads rather than re-querying the DB.
- **Time bounds on alerts queries** — top targets, top targeted devices, reason breakdown, and activity heatmap are now scoped to the last 7 days instead of all-time. Still meaningful, no longer scanning the full table history.
- **Composite indexes added** — added `(bssid, received_at)` on `beacon_events` and `deauth_events`, and `(src_mac, received_at)` on `probe_events`. Queries that filter by time and aggregate by BSSID or MAC now use a single composite index instead of two separate ones.
- **Background pruning task** — at startup, a background coroutine runs every 6 hours and deletes rows older than `HOT_DAYS` (30 days) from all event tables, followed by a WAL checkpoint. The database will no longer grow indefinitely.
---
## Python dependencies
```