esp32-cluster/README.md

ESP32 WiFi Recon Cluster

What this is

A distributed WiFi reconnaissance system using ESP32-S3 nodes and an Orange Pi as coordinator. The goal is passive RF awareness — mapping what is happening in the surrounding WiFi environment without transmitting or interacting with any networks.

Each ESP32 node scans the WiFi environment, builds structured events (not raw data), and ships them over UDP to a coordinator. The coordinator stores events to SQLite. A separate web dashboard reads the database and displays everything in real time via SSE.

The system is designed around a clear separation of concerns: nodes are dumb sensors, the Orange Pi does all storage and analysis, the browser does all rendering. No logic lives on the nodes beyond capturing and formatting events.

Practical use cases this has already demonstrated:

• Mapping every network and client device in a city-center apartment building
• Detecting sustained deauth flood attacks targeting specific devices (confirmed live)
• Reconstructing device travel history from probe request SSID lists
• Identifying device types by OUI (phones, IoT devices, laptops, cars, smart home hardware)
• Correlating attack victims across multiple impersonated APs using deauth dst MAC

This is a learning/research project covering distributed systems, event-driven architecture, sensor fusion, and detection engineering concepts.

---

Hardware

• Nodes: ESP32-S3 N16R8 (4 active, scattered across apartment)
• Coordinator: Orange Pi at 192.168.1.133, runs 24/7 as systemd services
• Dev machine: This PC at 192.168.1.101 — coding and flashing only
• Flashing: Arduino CLI on this PC
• Pen testing: Parrot OS laptop at 192.168.1.124 (user: keny) — active investigation rig
• WiFi adapter: Alfa MT7612U (wlx00c0cab67193) — monitor mode + packet injection, plugged into Parrot laptop
• Pen testing tools: airgeddon at /home/keny/Documents/github_tools/airgeddon

Active nodes

node_id	MAC	Firmware	Location
A1D658D4	e0:72:a1:d6:58:d4	WiFi	unplaced
A1D6F190	e0:72:a1:d6:f1:90	WiFi	unplaced
A1D700C4	e0:72:a1:d7:00:c4	BLE	unplaced
F68D6E30	44:1b:f6:8d:6e:30	BLE	unplaced

All four nodes reflashed 2026-05-21 (new apartment, SSID updated to botnet). WiFi: A1D658D4, A1D6F190. BLE: A1D700C4, F68D6E30.

---

Network

• ESP32 nodes connect to: WiFi SSID botnet
• Coordinator IP: 192.168.1.133 (Orange Pi, production)
• UDP port: 5005
• Dashboard port: 8080
• Parrot laptop: 192.168.1.124 — SSH as keny, passwordless sudo configured

---

Architecture

[WiFi nodes]  --UDP 5005--> [udp_ingest.py] --write--> [events.db (SQLite)]
[BLE nodes]   --UDP 5005-+                                     |
                                                               |
[Browser] <--HTTP/SSE 8080-- [dashboard.py] ------read--------+

Both node types share the same UDP ingestor and database. They are distinguished by event type field. BLE nodes send ble_adv and heartbeat events; WiFi nodes send beacon, probe, deauth, assoc, and heartbeat events.

The two coordinator processes are intentionally separate. The ingestor has one job: receive UDP, write to DB. The dashboard has one job: read DB, serve UI. They communicate only through the database. Each is independently restartable without affecting the other.

---

Development workflow

All coding is done on this PC. The Orange Pi only runs the deployed files — never edit directly on it.

[This PC] → edit code → rsync → [Orange Pi] → restart services

Sync to Orange Pi

bash /home/sandbox/Documents/projects/esp32_cluster/deploy.sh

This rsyncs coordinator/ to /opt/esp32_cluster/coordinator/ on the Orange Pi, excluding events.db (the Pi keeps its own database).

Restart services after deploy

ssh root@192.168.1.133 'systemctl restart udp_ingest dashboard'

View live logs on Orange Pi

# Ingestor (UDP events coming in)
ssh root@192.168.1.133 'journalctl -u udp_ingest -f'

# Dashboard
ssh root@192.168.1.133 'journalctl -u dashboard -f'

---

Project structure

esp32_cluster/
├── firmware/
│   ├── node/
│   │   ├── node.ino              # ESP32 WiFi node firmware
│   │   └── config.h              # WiFi creds, coordinator IP, scan/probe config
│   └── ble_node/
│       ├── ble_node.ino          # ESP32 BLE node firmware
│       └── config.h              # BLE scan/flush/dedup config
├── coordinator/
│   ├── udp_ingest.py             # UDP listener — receives packets, validates, writes to SQLite
│   ├── dashboard.py              # FastAPI app — reads SQLite, serves UI + SSE stream
│   ├── templates/
│   │   └── dashboard.html        # Dashboard skeleton HTML
│   ├── static/
│   │   ├── dashboard.css         # All dashboard styles
│   │   └── dashboard.js          # All dashboard logic (vanilla JS)
│   ├── oui.txt                   # IEEE OUI database for vendor lookup (39k entries)
│   ├── udp_ingest.service        # systemd service file for ingestor
│   ├── dashboard.service         # systemd service file for dashboard
│   └── events.db                 # SQLite database — on Orange Pi only, not synced
├── deploy.sh                     # rsync script: this PC → Orange Pi
├── starting_plan.md              # Original architecture design doc
└── README.md                     # This file

---

Firmware

Each node:

• Connects to WiFi sandbox
• Scans for nearby APs every 15 seconds (active scan, includes hidden SSIDs)
• Hops across all 13 2.4 GHz channels (300ms dwell per channel) while sniffing
• Captures probe request frames passively (promiscuous mode) between scans
• Captures deauth and disassoc frames (0xC0 / 0xA0)
• Captures association and reassociation request frames (0x00 / 0x20) with SSID IE parsing
• Sends a heartbeat packet every 10 seconds (uptime, free heap, WiFi RSSI to router, queue drop counters)
• Sends all events as UDP JSON packets to the coordinator
• node_id is derived from bytes 2–5 of the ESP32's base MAC address
• Events are buffered in FreeRTOS queues and flushed once per scan cycle when back on the home channel
• If a deauth with reason code 2 is seen at RSSI ≥ -60 dBm, an alert flag triggers an immediate flush on the home channel without waiting for the next scan

Event formats

Beacon event (AP discovered):

{
  "node_id": "F68D6E30",
  "ts":      12345,
  "type":    "beacon",
  "conf":    "high",
  "imp":     "normal",
  "ssid":    "NetworkName",
  "bssid":   "AA:BB:CC:DD:EE:FF",
  "rssi":    -65,
  "ch":      6,
  "enc":     "WPA2"
}

Probe event (client device searching for a network):

{
  "node_id": "F68D6E30",
  "ts":      12345,
  "type":    "probe",
  "conf":    "high",
  "imp":     "normal",
  "src_mac": "AA:BB:CC:DD:EE:FF",
  "ssid":    "HomeNetwork",
  "rssi":    -55
}

Deauth / Disassoc event (management frame captured in promiscuous mode):

{
  "node_id": "F68D6E30",
  "ts":      12345,
  "type":    "deauth",
  "src":     "AA:BB:CC:DD:EE:FF",
  "dst":     "11:22:33:44:55:66",
  "bssid":   "AA:BB:CC:DD:EE:FF",
  "reason":  7,
  "rssi":    -61
}

• type is deauth (0xC0) or disassoc (0xA0)
• dst = FF:FF:FF:FF:FF:FF means broadcast deauth — a classic deauth flood signature
• reason is the 802.11 reason code (7 = class 3 frame received from non-associated station, common in attack tools)
• Captured passively in the same promiscuous callback as probe requests, flushed to coordinator after each scan cycle

Assoc / Reassoc event (client joining a network):

{
  "node_id": "F68D6E30",
  "ts":      12345,
  "type":    "assoc",
  "src":     "AA:BB:CC:DD:EE:FF",
  "bssid":   "11:22:33:44:55:66",
  "ssid":    "NetworkName",
  "rssi":    -55
}

• type is assoc (0x00) or reassoc (0x20)
• SSID is parsed from the first Information Element in the frame body (IE tag 0x00)
• Captures real client→AP join events, not just passive probe searches

Heartbeat event (node health, sent every 10 seconds):

{
  "node_id":      "F68D6E30",
  "ts":           12345,
  "type":         "heartbeat",
  "uptime_ms":    123456,
  "free_heap":    245000,
  "wifi_rssi":    -62,
  "probe_drops":  0,
  "deauth_drops": 0,
  "assoc_drops":  0
}

• probe_drops, deauth_drops, assoc_drops count events lost due to queue overflow since boot. Non-zero values indicate the node is seeing more traffic than the queue sizes can absorb.

• imp: high if RSSI >= -50, low if <= -80, else normal

• ssid in probe events is the network the device is searching for — empty means wildcard (any network)

• Probe dedup: same MAC+SSID suppressed for 30 seconds to avoid flooding

• wifi_rssi in heartbeat is the node's own signal to the sandbox AP (not a scanned network)

• A node is considered online if a heartbeat arrived within the last 30 seconds; falls back to beacon within 60 seconds if no heartbeat has been received yet

Flashing a node

Requirements: Arduino CLI v1.4.1, ESP32 core v3.3.7, FQBN esp32:esp32:esp32s3, user in dialout group.

# Compile
arduino-cli compile --fqbn esp32:esp32:esp32s3 /home/sandbox/Documents/projects/esp32_cluster/firmware/node/

# Flash — adjust port as needed (check with: ls /dev/ttyACM*)
arduino-cli upload --fqbn esp32:esp32:esp32s3 --port /dev/ttyACM0 \
  /home/sandbox/Documents/projects/esp32_cluster/firmware/node/

The same binary works on every node — node_id is auto-derived from the MAC, no per-node config needed.

Check group membership with groups. You should see dialout.

config.h reference

Constant	Default	Purpose
WIFI_SSID	sandbox	WiFi network to connect to
COORDINATOR_IP	192.168.1.133	Orange Pi address
COORDINATOR_PORT	5005	UDP port
SCAN_INTERVAL_MS	15000	How often to run a beacon scan (ms)
HEARTBEAT_INTERVAL_MS	10000	How often to send a heartbeat (ms)
PROBE_SNIFF	1	Enable/disable probe/deauth/assoc sniffing (1/0)
PROBE_DEDUP_SECS	30	Suppress duplicate probe MAC+SSID (secs)
DEDUP_CACHE_SIZE	64	Number of MAC+SSID pairs tracked for dedup
PROBE_QUEUE_SIZE	128	Max probe events buffered between flushes
DEAUTH_QUEUE_SIZE	128	Max deauth/disassoc events buffered
ASSOC_QUEUE_SIZE	32	Max assoc/reassoc events buffered
HOP_DWELL_MS	300	Ms to dwell on each channel while hopping (13ch × 300ms ≈ 4s)
RSSI_ALERT_THRESHOLD	-60	RSSI floor for immediate-flush deauth alert (dBm)

---

BLE Firmware

BLE nodes run firmware/ble_node/. Same hardware (ESP32-S3), same transport (UDP JSON to coordinator), different job.

What BLE nodes do

• Connect to WiFi sandbox for UDP transport only (no WiFi scanning)
• Run a passive BLE scan continuously (5-second windows, auto-restart)
• Parse each advertisement: MAC, address type, device name, manufacturer specific data (company ID + payload)
• Dedup by MAC within a 30-second window to avoid flooding the coordinator
• Buffer events in a FreeRTOS queue, flush every 5 seconds
• Send a heartbeat every 10 seconds (same format as WiFi nodes, adds ble_drops field)

The ESP32 radio is shared between WiFi and BLE via the hardware coexistence controller. WiFi only transmits briefly during UDP flushes, so BLE scan coverage is near-continuous.

BLE event format

{
  "node_id":  "A1D700C4",
  "ts":       12345,
  "type":     "ble_adv",
  "mac":      "AA:BB:CC:DD:EE:FF",
  "addr_type": 1,
  "name":     "DeviceName",
  "rssi":     -65,
  "mfr_id":   76,
  "mfr_data": "1201..."
}

• addr_type: 0 = public (real, stable MAC), 1 = random (rotates periodically)
• mfr_id: 16-bit little-endian Bluetooth company ID, -1 if the advertisement has no manufacturer data
• mfr_data: hex string of the manufacturer payload after the 2-byte company ID, up to 16 bytes

Known company IDs

ID (hex)	Vendor
0x004C	Apple (AirTags, AirPods, iBeacon, FindMy, HomeKit)
0x0075	Samsung (SmartTag, Galaxy devices)
0x0006	Microsoft (Swift Pair)
0x00E0	Google
0x0059	Nordic Semiconductor (common in IoT devices)
0x0138	Garmin
0x0499	Ruuvi Innovations
0x0157	Polar Electro
0x00BD	Fitbit
0x0057	Harman International (JBL, AKG)
0x0171	Amazon

Apple type byte (first byte of mfr_data when mfr_id = 76):

Byte	Device
02	iBeacon
05	AirDrop
07	HomeKit
09	AirPods
0a	AirPods Pro
0f	AirPods 3rd gen
12	FindMy / AirTag
15	Proximity Pair

Flashing a BLE node

Same requirements as WiFi nodes. Flash firmware/ble_node/ instead of firmware/node/:

# Compile
arduino-cli compile --fqbn esp32:esp32:esp32s3 /home/sandbox/Documents/projects/esp32_cluster/firmware/ble_node/

# Flash
arduino-cli upload --fqbn esp32:esp32:esp32s3 --port /dev/ttyACM0 \
  /home/sandbox/Documents/projects/esp32_cluster/firmware/ble_node/

config.h reference (BLE node)

Constant	Default	Purpose
BLE_SCAN_DURATION_SECS	5	Passive scan window length
BLE_FLUSH_INTERVAL_MS	5000	How often to send queued events to coordinator
HEARTBEAT_INTERVAL_MS	10000	Heartbeat cadence
BLE_DEDUP_SECS	30	Suppress same MAC within this window
BLE_DEDUP_CACHE_SIZE	300	Dedup ring-buffer entries
BLE_QUEUE_SIZE	256	Max events buffered between flushes

---

Orange Pi — running services

Services are managed by systemd and start automatically on boot.

# Status
ssh root@192.168.1.133 'systemctl status udp_ingest dashboard'

# Restart both
ssh root@192.168.1.133 'systemctl restart udp_ingest dashboard'

# Stop
ssh root@192.168.1.133 'systemctl stop udp_ingest dashboard'

Service files are in coordinator/ and deployed to /etc/systemd/system/ on the Orange Pi. The coordinator files live at /opt/esp32_cluster/coordinator/ on the Orange Pi.

---

Dashboard

Open http://192.168.1.133:8080 in your browser.

Views

• Feed — live beacon event stream across all nodes, capped at 100 rows, newest on top
• Networks — deduplicated AP table (by BSSID): times seen, best/avg RSSI, channel, encryption, node count. Sortable by any column. Each row has a ▶ button to open the RSSI history chart for that network.
• RSSI history chart — opens from any Networks row. Shows signal strength over time per node (1h / 2h / 6h / 24h selectable). Each node gets its own coloured line. Useful for spotting interference patterns, seeing how signal fluctuates by time of day, and comparing which node consistently hears a given AP better. Updates live as new scans arrive.
• Cross-node — side-by-side per-node RSSI for every AP. Shows which node is physically closer to each network. Filter to multi-node only to focus on confirmed cross-node observations.
• Clients — probe request data: unique MACs, vendor (OUI lookup), what SSIDs they're searching for, signal, which nodes saw them.
• Alerts — deauth/disassoc frame detection with four sections:
  • 1h Summary strip — total frames, deauth vs disassoc counts, unique BSSIDs and sources in the last hour
  • Most Impersonated Networks — APs whose BSSID is being spoofed as the deauth source, ranked by frame count
  • Most Targeted Devices — actual victim devices (by dst MAC), ranked by frames received across all impersonated APs
  • Detected Bursts — fires when ≥10 frames for the same BSSID within 5 minutes. Highlighted red when confirmed by 2+ nodes. Reason code 2 at volume is the primary attack indicator.
  • Raw Feed — last 100 deauth/disassoc events with full src/dst/BSSID/reason/RSSI detail.
• Search — cross-table lookup by any identifier. Supports MAC address, SSID, BSSID, and vendor name (e.g. "Raspberry", "Tuya", "Intel"). Returns all matching data across beacon, probe, and deauth tables in labelled sections. Vendor name search works by matching against the OUI database and finding all devices with those MAC prefixes.
• Presence — three sections driven entirely by probe data:
  • Present Now — real (non-randomized) MACs seen 2+ times in the last hour. Filters out the city-center noise of transient devices.
  • New Arrivals — devices and networks first seen in the last 24 hours, split into two side-by-side tables.
  • Regulars — devices and networks seen on 2 or more distinct calendar days. Takes at least two days of data to populate.
• Node detail — click a node in the sidebar for stats: total events, unique SSIDs/BSSIDs, RSSI range, first/last seen, uptime, free heap, AP signal, last 100 events.

Node detail

Click any node in the sidebar to open the detail view. Ten stat cards arranged in a 2×5 grid show: status, total events, unique SSIDs, unique BSSIDs, avg RSSI, first seen, last seen, uptime, free heap, and AP signal. Below the cards is a table of the last 100 events for that node.

Sidebar

Always visible: node list with online/offline status dot, last seen time, event count, and a compact heartbeat line (uptime · free heap · AP signal). Nodes are sorted alphabetically by node ID so the order stays fixed as more nodes are added. A node is considered online if a heartbeat was received within the last 30 seconds. Falls back to beacon timestamp (60s threshold) if no heartbeat has been received yet.

UI

Font: Roboto (Google Fonts). Chosen for readability at small sizes — the previous monospace font felt too thin at the smaller label sizes used throughout the dashboard.

Live updates

SSE stream pushes new beacon events as they arrive. The feed updates immediately. All other views (sidebar, networks, chart, cross-node, presence, alerts) refresh on a 2-second debounce so a single scan burst doesn't flood the server with requests.

---

Observations and notes

Hidden networks

Networks with a blank SSID but valid BSSID, signal, and channel are hidden networks — APs configured not to broadcast their name. They are not hidden from passive scanners. The encryption showing as OPEN is a firmware parsing artifact; the network itself may be encrypted. The BSSID OUI still identifies the manufacturer.

Probe requests and MAC randomization

Modern phones and laptops (iOS since ~2020, Android since ~2021) randomize their MAC address for probe requests. A randomized MAC has bit 1 of the first byte set — the dashboard detects this and labels it "Randomized MAC" instead of doing a meaningless OUI lookup.

What probe data tells you:

• Directed probes (with SSID): the device has connected to that network in the past
• Wildcard probes (empty SSID): the device is searching for any available network
• Probe data maps client devices, not infrastructure — complementary to beacon scan data

Deauth frame noise vs real attacks

The raw deauth feed in the Alerts tab will fill up constantly — deauth and disassoc frames are a normal part of WiFi operation (devices disconnecting, APs doing band steering, power saving). This is expected and not cause for concern.

The Bursts section is the anomaly detector. A single unique source MAC generating 10+ deauth frames for the same BSSID within 5 minutes is not normal operation — it indicates a deauth flood, the standard precursor to a WPA2 handshake capture attack. The attacker forces connected clients to disconnect, captures the 4-way handshake when they reconnect, and then cracks it offline.

Multi-node confirmation (node_count > 1, highlighted red) significantly raises confidence — it means the source is physically close and strong, not a distant weak signal.

Node dropout — AP overwhelm on simultaneous reconnect

Observed 2026-04-06: three of four nodes dropped off the dashboard after being power cycled individually at different times. Symptoms included reason-15 deauth floods (4-way handshake timeout) from the affected nodes captured by the surviving node, and nodes getting stuck in a reconnect loop without ever successfully sending data to the coordinator.

Root cause: the sandbox AP was overwhelmed by multiple nodes power cycling at slightly different times and hammering it with simultaneous association requests. Each node hops channels every ~4 seconds, briefly dropping and reconnecting to sandbox on every cycle. When several nodes do this at the same time after a staggered power cycle, the AP's association table fills with stale entries and stops completing WPA2 handshakes for new clients.

Confirmed not a firmware issue: all four nodes ran for 21 hours continuously without a single drop when power cycled simultaneously (clean start, all connect at once). Heap stayed stable across all nodes throughout (234K–247K, no drift).

Mitigation: power cycle all nodes at the same time rather than individually, so they all connect fresh together rather than in a staggered loop. Long-term fix is a dedicated AP for the nodes only, so their reconnect churn is isolated from regular network traffic and the AP is never competing with other clients.

Our own networks

To avoid investigating our own infrastructure, these SSIDs and BSSIDs are ours:

SSID	Band	Purpose
botnet	2.4GHz	Main network — nodes connect here, IoT devices
pronet	5GHz	Main 5GHz network
mango	2.4GHz	Secondary network

OUI lookup

The oui.txt file is the IEEE public OUI database (39,171 entries as of download). It maps the first 3 bytes of a real MAC to a manufacturer name. Used in the Clients tab. Refresh it occasionally by re-running deploy.sh after downloading a fresh copy from https://standards-oui.ieee.org/oui/oui.txt.

---

Active investigations

Living Room speaker

SSID: Living Room speaker.n078 · BSSID: FA:8F:CA:76:06:B2 · ch 6 · OPEN · RSSI -57 dBm.

Spotted again on first scan at new apartment (2026-05-21). Same BSSID as previous location — either the device moved with us, or a neighbour in the new building has the same model. Broadcasting an open unauthenticated provisioning hotspot. Pending: connect via Parrot laptop + Alfa adapter, nmap the interface.

Previous location — deauth investigation (archived)

A sustained deauth flood targeting a Tuya device ran 2026-04-03 to ~2026-04-25, then wound down. Attacker MACs: 82:4E:66:47:09:C1, 42:8C:46:6E:12:9C, 62:D9:AA:E8:B4:09, 62:87:CB:38:2C:20. Not expected to reappear at new location.

---

BLE observations and notes

Data below is from the previous location (2026-04-26 to 2026-05-02, 6 days). Retained as reference — some devices and patterns will differ at the new apartment.

BLE data summary (previous location)

Data collected from BLE launch (2026-04-26) to project shutdown (2026-05-02) — 6 days, 2 nodes (A1D700C4, F68D6E30).

Metric	Value
Total BLE events	1,086,897
Unique MACs observed	89,818
Public (stable) MACs	1,392
Random (rotating) MACs	88,426
Named devices identified	1,486
Peak day	2026-04-29 — 209,589 events, 18,308 unique MACs
Node A1D700C4 (room 3)	675,951 events, 85,535 unique MACs
Node F68D6E30 (room 1)	411,086 events, 46,906 unique MACs

Manufacturer breakdown (top by event count):

Company ID	Vendor	Events	Unique MACs
0x004C	Apple	686,109	67,987
0x0075	Samsung	143,520	957
0x0006	Microsoft	53,794	2,336
0x0057	Harman/JBL	27,930	47

Apple accounts for ~63% of all BLE traffic. The high Apple unique MAC count is mostly FindMy/AirTag MAC rotation — individual physical devices inflate the count significantly.

Apple type byte breakdown:

Type	Description	Events	Unique MACs
0x12	FindMy / AirTag (MAC rotates per advertisement)	271,336	26,137
0x10	Proximity Pair (iPhone advertising nearby)	228,086	28,992
0x16	Nearby Info (iPhone battery/status)	78,848	4,375
0x06	Unknown type — 1 stable MAC, continuous all-day	33,306	3
0x07	HomeKit accessory	32,774	4,449
0x09	AirPods (in use / case open)	22,447	1,852
0x0c	AirPods case / Watch (lid open or unlocked)	11,420	2,800

Heap baseline

BLE nodes floor out at 109–127K free heap vs 222K+ for WiFi nodes. The ~100K difference is the BLE stack overhead (BLEDevice + BLEScan + coexistence controller). This is expected and stable — no downward drift observed after 21 hours. The WiFi-only queue flush model keeps heap use predictable.

MAC randomization and device counting

BLE unique MAC counts are misleading without context:

• Apple FindMy / AirTag rotates MAC on every advertisement interval (seconds). One physical AirTag generates hundreds of apparent "unique" MACs per day. The mfr_data payload is more useful for identity than the MAC.
• Sony WH-1000XM5 rotates MAC every ~15 minutes. One device appears as ~28 different MACs over a day. Identify by device name (LE_WH-1000XM5) rather than MAC.
• Public MACs (addr_type = public) are stable and reliably trackable (B&W PX5, ELK-BLEDOM, ATH-M50xBT2).
• Apple Proximity Pair / Nearby Info are iPhones advertising presence — high volume, all randomized.

Apple type byte reference (first byte of mfr_data when mfr_id = 76)

Most common types seen in practice:

Byte	Type	Notes
10	Proximity Pair	iPhone/Watch advertising nearby — very common
12	FindMy / AirTag	MAC rotates every advertisement; best RSSI seen: −38 dBm
16	Nearby Info	iPhone battery/status broadcast
0c	AirPods case / Watch	Seen when case lid is open or Watch is unlocked
07	HomeKit	Smart home accessory (lights, locks, sensors)
09/0a/0f	AirPods (various gen)	Seen when case is open or buds are in use
02	iBeacon	Retail/venue tracking beacons

Identified permanent BLE neighbours (as of 2026-05-02)

Device	MAC type	Pattern	Notes
Sony WH-1000XM5	Random (rotates ~15min)	Most days, all hours	327 unique MACs observed across 6 days — confirmed 1 device by rotation pattern
Sony WH-1000XM4	Random (rotates ~15min)	Most days	57 unique MACs — distinct neighbour from the XM5 user
B&W PX5 headphones	Public — stable	Sunday Apr 26, RSSI ~−85–99	Appeared on a Sunday rather than expected weekday pattern; may also be present on weekdays at lower RSSI
ELK-BLEDOM LED strip	Public — stable	Apr 26 evenings only	Sporadic; appeared one evening then went quiet
Audio-Technica ATH-M50xBT2	Public — stable (2 MACs)	Multiple days, afternoon sessions	Two separate units or one with two stable MACs
Govee H6609 LED strip (Govee_H6609_4D1A)	Random — stable	Multiple days, persistent	BLE-controlled RGB strip, Govee controller in nearby unit
Creative Bowie MA10 speaker	Public — stable (41:aa:66:90:99:d7)	All 6 days, evenings	Consistent presence; best RSSI −82 dBm
Sennheiser MOMENTUM 4	Public — stable (80:c3:ba:86:18:e9)	Multiple days	Professional over-ear headphones
JBL speaker cluster	Mixed (Clip 5, Flip 5 ×2, Charge 5/6)	Occasional, low RSSI ~−93–99	Multiple different JBL models across different days — building has several JBL users
0x5148 device (c2:fe:68:e8:01:a6)	Public — stable	All 6 days, all hours	Unknown company ID 0x5148, payload fixed ASCII "364656", best RSSI −55 dBm — very close, identity still unknown
"net" (80:3e:4f:1b:4f:e3)	Public — stable	Apr 26/28/29 only, very weak −96–100	Appears to have moved or powered off; was previously more consistent

Samsung SmartTag cluster

Three stable public Samsung MACs (8c:79:f5:a6:dd:c3, 68:72:c3:bd:ed:6f, 7c:64:56:84:e0:b3) generate 30K–36K events each across all 6 days of BLE data. All share the same mfr_data prefix (42040180...) — Samsung's Galaxy Find Network advertisement format (type byte 0x42). These are Samsung SmartTag2 trackers belonging to one or more neighbours; stable public MACs make them directly trackable. RSSI −73 to −100 dBm suggests they are in a nearby unit or parking area below.

Apple type 0x06 — persistent unknown device

MAC c7:73:2d:eb:1a:b1 (random but stable) generates 30,765 events across all 6 days with consistent daily volume of 3K–5K events. Apple type byte 0x06 does not appear in Apple's published proximity pairing spec; this is an undocumented continuity or accessory advertisement type. The MAC has not rotated over 6 days, which is unusual for a random-type address. Likely a HomeKit accessory or Apple TV in a neighbouring unit that advertises continuously. Best RSSI −83 dBm.

Parking structure BLE

The same parking structure that generates hundreds of 70mai dashcam WiFi beacons also contains AirTags. The FindMy/AirTag type (0x12) consistently hits −38 dBm best RSSI — some are close enough to be in vehicles parked right outside. At least one is persistent across multiple scans, suggesting a parked vehicle with an AirTag rather than a passing pedestrian.

WiFi-derived context: notable open networks

For reference during future BLE correlation work, the known open networks in range:

SSID	Device	Notes
Living Room speaker.n078	Unknown speaker	Unauthenticated provisioning hotspot, RSSI −38 dBm, same building. Pending active investigation.
yeelink-light-strip2_miap7AB7	Xiaomi Yeelight LED strip	Stuck in setup mode for 8+ days, open
xiaomi-fryer-maf65_mibt962C	Xiaomi smart air fryer	Open setup AP
zhimi-airp-mb5_mibt5FF0	Xiaomi Mi Air Purifier MB5	Open setup AP
REOLINK-HHRYzJCESD-2.4G	Reolink IP camera	Permanent neighbour's camera, 8 days uptime

---

Current status

Project revived 2026-05-21 — new apartment, fresh start. All four nodes reflashed and online, coordinator running on Orange Pi, database clean.

• Arduino CLI installed, ESP32 core configured
• Node firmware: beacon scan + probe sniffing
• Four ESP32-S3 nodes flashed and running (2× WiFi, 2× BLE)
• UDP ingestor (udp_ingest.py) — beacon + probe routing with field validation
• Web dashboard: Feed, Networks, Cross-node, Clients, Alerts, Search, Presence, Node detail
• OUI vendor lookup in Clients view (vendor grouping + randomized MAC detection)
• Dashboard split into HTML / CSS / JS (no monolithic file)
• Deployed to Orange Pi as systemd services (runs 24/7)
• All four nodes sending to Orange Pi, confirmed live
• Node heartbeat every 10s — uptime, free heap, AP signal, drives online/offline status
• RSSI history chart per network (canvas, per-node coloured lines, 1h/2h/6h/24h range)
• Presence tab — Present Now, New Arrivals, Regulars (fixed key mismatch bug)
• Deauth/disassoc frame detection — burst detection with multi-node confirmation, Alerts tab
• Alerts tab — most impersonated networks, most targeted devices, burst detection, raw feed
• Search tab — cross-table lookup by MAC, SSID, BSSID, or vendor name
• Node detail view — 2×5 stat cards with animated border, fixed alphabetical sidebar order
• Channel hopping — 13 channels at 300ms dwell, flush on homeChannel pass
• Assoc/reassoc frame capture — client join events with SSID IE parsing
• RSSI-triggered alert flush — immediate homeChannel flush on close-range deauth attack signature
• Queue drop counters — tracked per queue in firmware, reported in heartbeat, stored in DB
• SQLite indexes — received_at, node_id, bssid, src_mac, src, dst across all event tables
• Dashboard query caps — 300-row limits on Clients, Networks, Cross-node to keep UI responsive
• Sessions tab — deauth events grouped into sessions by source MAC + 2-min gap, sortable, expandable rows
• Dashboard performance overhaul — fixed tab switch latency and Networks tab failing to load (see below)
• DB pruning / retention policy — background task deletes events older than HOT_DAYS (7 days) every 6 hours
• Alerts tab — dominant reason code added to Most Impersonated Networks and Most Targeted Devices tables
• Sessions tab — dominant reason code + description shown per session row and in expanded detail panel
• SSE reconnect bug fix — multiple stale EventSource instances were accumulating on reconnect, flooding /api/nodes on tab re-open; fixed with proper close-before-reconnect and in-flight guard
• WAL corruption fix — recurring DB corruption under sustained write load; fixed with RESTART checkpointing every 30 minutes and reduced retention to 7 days (see below)
• BLE firmware (firmware/ble_node/) — passive scan, dedup, mfr data parsing, heartbeat
• Coordinator BLE support — ble_events table, ingest handler, ble_drops in heartbeat
• Dashboard BLE tab — devices table, manufacturer breakdown, live feed
• BLE nodes now visible in dashboard sidebar and node detail (previously invisible — heartbeat-only nodes not queried)
• ble_events and heartbeat_events added to pruning cycle (previously accumulated unbounded)
• ble_events table added to ensure_schema() in dashboard (previously only created by ingestor)
• Ingestor commits batched per packet instead of per store call; wal_autocheckpoint=500 added to writer connection
• Python stdout unbuffered in both service files (python3 -u) — logs now appear in journalctl in real time
• confidence column dropped from beacon_events (was always "high", never queried)
• Surface assoc events in dashboard (Search results or dedicated view)
• Scan interval control from dashboard
• Dedicated AP for nodes to isolate reconnect churn from regular network traffic
• Place and label nodes in new apartment rooms

---

Dashboard performance overhaul (2026-04-06)

Problem

Tab switches took up to a minute to load. The Networks tab failed to load entirely. The SSE live feed would stall during tab switches. The root cause was three compounding issues:

1. Blocking the async event loop. FastAPI routes are async def, but the SQLite calls (sqlite3 module) are fully synchronous. When called directly inside async def, they block uvicorn's entire event loop — meaning while one slow query runs, the server cannot serve any other request, including the SSE stream. This is why everything froze together.

2. Expensive endpoints had no caching. /api/alerts, /api/sessions, /api/presence, and /api/cross-node ran full database queries on every single request, with no result caching. The 2-second debounce on the frontend meant these were being called repeatedly.

build_sessions() was the worst: it pulled every deauth event from the last 30 days and grouped them into sessions in a Python loop — O(N) in Python on every call, with no cache. Under a sustained deauth flood (thousands of events/day), this was very slow.

build_alerts() ran several queries against the entire deauth_events table with no time cutoff — top targets, top targeted devices, reason breakdown, activity heatmap all scanned all-time data.

3. Unbounded database growth. No pruning meant every query got slower every day as the tables grew. The database had also developed B-tree corruption (double-referenced pages, out-of-order rowids), likely from WAL journal not checkpointing cleanly under sustained write load. The database was rebuilt clean.

Fixes applied

• asyncio.to_thread() — all build_* calls in every route are now dispatched to a thread pool executor. The event loop stays free to handle SSE and other requests while queries run in the background. The SSE generator's inline query() calls were also fixed the same way.

• Caching added — alerts (60s TTL), sessions (60s), presence (60s), cross-node (60s) now cache results in memory. The existing nodes/networks/clients caches were kept. Tab switches hit the cache on repeated loads rather than re-querying the DB.

• Time bounds on alerts queries — top targets, top targeted devices, reason breakdown, and activity heatmap are now scoped to the last 7 days instead of all-time. Still meaningful, no longer scanning the full table history.

• Composite indexes added — added (bssid, received_at) on beacon_events and deauth_events, and (src_mac, received_at) on probe_events. Queries that filter by time and aggregate by BSSID or MAC now use a single composite index instead of two separate ones.

• Background pruning task — at startup, a background coroutine runs every 6 hours and deletes rows older than HOT_DAYS from all event tables. HOT_DAYS was later reduced from 30 to 7 — see WAL corruption fix below.

---

WAL corruption fix (2026-04-08)

Problem

The SQLite database corrupted twice within 3 days of operation, both times producing database disk image is malformed errors. The dashboard would start returning partial or no results on some endpoints, and the SSE stream would crash. A full DB rebuild was required each time.

Root cause

SQLite in WAL (Write-Ahead Log) mode works by writing all changes to a separate WAL file first, then periodically folding those changes back into the main database file — a process called checkpointing. The default checkpoint mode is PASSIVE, which only checkpoints what it can without blocking any active readers. It gives up immediately if a reader is present.

With 4 nodes generating hundreds of events per minute and the dashboard constantly reading (SSE polling every 2 seconds, plus user interactions), there is almost never a clean window for a passive checkpoint to complete. The WAL file accumulates uncommitted state indefinitely, grows too large, and eventually corrupts the main database file. The prune task added in the earlier performance overhaul also used PASSIVE mode, so it had the same problem.

A second contributing factor: HOT_DAYS was set to 30 days. At the observed data rate (~500k beacon events/day, ~115k probe events/day), the database would grow to several gigabytes before the first prune cycle deleted anything, making the WAL problem worse over time.

Fixes applied

• RESTART checkpointing — replaced PASSIVE with PRAGMA wal_checkpoint(RESTART) in a new checkpoint_wal() function. RESTART waits for all current readers to finish, then checkpoints all WAL frames into the main DB file and resets the WAL back to the beginning. This guarantees the WAL is fully flushed rather than partially checkpointed.

• Dedicated checkpoint task — a new background coroutine runs checkpoint_wal() every 30 minutes, independent of the prune cycle. The prune task also calls it after deleting rows, so a large delete is always followed by a full WAL flush.

• wal_autocheckpoint=500 — set on every connection so SQLite's own automatic checkpoint triggers at ~500 pages (~2MB) instead of the default 1000 pages (~4MB). This gives more frequent small opportunities to checkpoint between the 30-minute forced runs.

• HOT_DAYS reduced from 30 to 7 — at current data rates, 7 days of retention keeps the database under ~200MB permanently. Queries stay fast, the WAL stays manageable, and 7 days of history is sufficient for all dashboard views and active investigations.

---

Python dependencies

fastapi
uvicorn

pip install fastapi uvicorn