Files
esp32-cluster/README.md
T

301 lines
12 KiB
Markdown
Raw Normal View History

2026-04-03 14:38:38 +03:00
# ESP32 WiFi Recon Cluster
## What this is
A distributed WiFi reconnaissance system using ESP32-S3 nodes and an Orange Pi as coordinator.
Each ESP32 node scans the WiFi environment, builds structured events (not raw data), and ships them over UDP to a coordinator. The coordinator stores events to SQLite. A separate web dashboard reads the database and displays everything in real time via SSE.
This is a learning/research project covering distributed systems, event-driven architecture, sensor fusion, and detection engineering concepts.
---
## Hardware
- **Nodes:** ESP32-S3 N16R8 (2 active, scaling later)
- **Coordinator:** Orange Pi at `192.168.1.133`, runs 24/7 as systemd services
- **Dev machine:** This PC at `192.168.1.101` — coding and flashing only
- **Flashing:** Arduino CLI on this PC
### Active nodes
| node_id | MAC | Port |
|----------|-------------------|--------------|
| F68D6E30 | 44:1b:f6:8d:6e:30 | /dev/ttyACM0 |
| A1D6F190 | e0:72:a1:d6:f1:90 | /dev/ttyACM1 |
| A1D658D4 | e0:72:a1:d6:58:d4 | /dev/ttyACM2 |
---
## Network
- **ESP32 nodes connect to:** WiFi SSID `sandbox`
- **Coordinator IP:** `192.168.1.133` (Orange Pi, production)
- **UDP port:** `5005`
- **Dashboard port:** `8080`
---
## Architecture
```
[ESP32 nodes] --UDP 5005--> [udp_ingest.py] --write--> [events.db (SQLite)]
|
[Browser] <--HTTP/SSE 8080-- [dashboard.py] ------read--------+
```
The two coordinator processes are intentionally separate. The ingestor has one job: receive UDP, write to DB. The dashboard has one job: read DB, serve UI. They communicate only through the database. Each is independently restartable without affecting the other.
---
## Development workflow
**All coding is done on this PC.** The Orange Pi only runs the deployed files — never edit directly on it.
```
[This PC] → edit code → rsync → [Orange Pi] → restart services
```
### Sync to Orange Pi
```bash
bash /home/sandbox/Documents/projects/esp32_cluster/deploy.sh
```
This rsyncs `coordinator/` to `/opt/esp32_cluster/coordinator/` on the Orange Pi, excluding `events.db` (the Pi keeps its own database).
### Restart services after deploy
```bash
ssh root@192.168.1.133 'systemctl restart udp_ingest dashboard'
```
### View live logs on Orange Pi
```bash
# Ingestor (UDP events coming in)
ssh root@192.168.1.133 'journalctl -u udp_ingest -f'
# Dashboard
ssh root@192.168.1.133 'journalctl -u dashboard -f'
```
---
## Project structure
```
esp32_cluster/
├── firmware/
│ └── node/
│ ├── node.ino # ESP32 node firmware
│ └── config.h # WiFi creds, coordinator IP, scan/probe config
├── coordinator/
│ ├── udp_ingest.py # UDP listener — receives packets, validates, writes to SQLite
│ ├── dashboard.py # FastAPI app — reads SQLite, serves UI + SSE stream
│ ├── templates/
│ │ └── dashboard.html # Dashboard skeleton HTML
│ ├── static/
│ │ ├── dashboard.css # All dashboard styles
│ │ └── dashboard.js # All dashboard logic (vanilla JS)
│ ├── oui.txt # IEEE OUI database for vendor lookup (39k entries)
│ ├── udp_ingest.service # systemd service file for ingestor
│ ├── dashboard.service # systemd service file for dashboard
│ └── events.db # SQLite database — on Orange Pi only, not synced
├── deploy.sh # rsync script: this PC → Orange Pi
├── starting_plan.md # Original architecture design doc
└── README.md # This file
```
---
## Firmware
Each node:
- Connects to WiFi `sandbox`
- Scans for nearby APs every 15 seconds (active scan, includes hidden SSIDs)
- Captures probe request frames passively (promiscuous mode) between scans
- Sends a heartbeat packet every 10 seconds (uptime, free heap, WiFi RSSI to router)
- Sends beacon, probe, and heartbeat events as UDP JSON packets to the coordinator
- `node_id` is derived from bytes 25 of the ESP32's base MAC address
### Event formats
**Beacon event** (AP discovered):
```json
{
"node_id": "F68D6E30",
"ts": 12345,
"type": "beacon",
"conf": "high",
"imp": "normal",
"ssid": "NetworkName",
"bssid": "AA:BB:CC:DD:EE:FF",
"rssi": -65,
"ch": 6,
"enc": "WPA2"
}
```
**Probe event** (client device searching for a network):
```json
{
"node_id": "F68D6E30",
"ts": 12345,
"type": "probe",
"conf": "high",
"imp": "normal",
"src_mac": "AA:BB:CC:DD:EE:FF",
"ssid": "HomeNetwork",
"rssi": -55
}
```
**Heartbeat event** (node health, sent every 10 seconds):
```json
{
"node_id": "F68D6E30",
"ts": 12345,
"type": "heartbeat",
"uptime_ms": 123456,
"free_heap": 245000,
"wifi_rssi": -62
}
```
- `imp`: `high` if RSSI >= -50, `low` if <= -80, else `normal`
- `ssid` in probe events is the network the device is searching for — empty means wildcard (any network)
- Probe dedup: same MAC+SSID suppressed for 30 seconds to avoid flooding
- `wifi_rssi` in heartbeat is the node's own signal to the `sandbox` AP (not a scanned network)
- A node is considered **online** if a heartbeat arrived within the last 30 seconds; falls back to beacon within 60 seconds if no heartbeat has been received yet
### Flashing a node
Requirements: Arduino CLI v1.4.1, ESP32 core v3.3.7, FQBN `esp32:esp32:esp32s3`, user in `dialout` group.
```bash
# Compile
arduino-cli compile --fqbn esp32:esp32:esp32s3 /home/sandbox/Documents/projects/esp32_cluster/firmware/node/
# Flash — adjust port as needed (check with: ls /dev/ttyACM*)
arduino-cli upload --fqbn esp32:esp32:esp32s3 --port /dev/ttyACM0 \
/home/sandbox/Documents/projects/esp32_cluster/firmware/node/
```
The same binary works on every node — `node_id` is auto-derived from the MAC, no per-node config needed.
> Check group membership with `groups`. You should see `dialout`.
### config.h reference
| Constant | Default | Purpose |
|-------------------|------------------|-------------------------------------------|
| `WIFI_SSID` | `sandbox` | WiFi network to connect to |
| `COORDINATOR_IP` | `192.168.1.133` | Orange Pi address |
| `COORDINATOR_PORT`| `5005` | UDP port |
| `SCAN_INTERVAL_MS`| `15000` | How often to run a beacon scan (ms) |
| `PROBE_SNIFF` | `1` | Enable/disable probe sniffing (1/0) |
| `PROBE_DEDUP_SECS`| `30` | Suppress duplicate probe MAC+SSID (secs) |
| `PROBE_QUEUE_SIZE`| `32` | Max probe events buffered between scans |
---
## Orange Pi — running services
Services are managed by systemd and start automatically on boot.
```bash
# Status
ssh root@192.168.1.133 'systemctl status udp_ingest dashboard'
# Restart both
ssh root@192.168.1.133 'systemctl restart udp_ingest dashboard'
# Stop
ssh root@192.168.1.133 'systemctl stop udp_ingest dashboard'
```
Service files are in `coordinator/` and deployed to `/etc/systemd/system/` on the Orange Pi.
The coordinator files live at `/opt/esp32_cluster/coordinator/` on the Orange Pi.
---
## Dashboard
Open `http://192.168.1.133:8080` in your browser.
### Views
- **Feed** — live beacon event stream across all nodes, capped at 100 rows, newest on top
- **Networks** — deduplicated AP table (by BSSID): times seen, best/avg RSSI, channel, encryption, node count. Sortable by any column. Each row has a ▶ button to open the RSSI history chart for that network.
- **RSSI history chart** — opens from any Networks row. Shows signal strength over time per node (1h / 2h / 6h / 24h selectable). Each node gets its own coloured line. Useful for spotting interference patterns, seeing how signal fluctuates by time of day, and comparing which node consistently hears a given AP better. Updates live as new scans arrive.
- **Cross-node** — side-by-side per-node RSSI for every AP. Shows which node is physically closer to each network. Filter to multi-node only to focus on confirmed cross-node observations.
- **Clients** — probe request data: unique MACs, vendor (OUI lookup), what SSIDs they're searching for, signal, which nodes saw them.
- **Presence** — three sections driven entirely by probe data:
- *Present Now* — real (non-randomized) MACs seen 2+ times in the last hour. Filters out the city-center noise of transient devices.
- *New Arrivals* — devices and networks first seen in the last 24 hours, split into two side-by-side tables.
- *Regulars* — devices and networks seen on 2 or more distinct calendar days. Takes at least two days of data to populate.
2026-04-03 14:38:38 +03:00
- **Node detail** — click a node in the sidebar for stats: total events, unique SSIDs/BSSIDs, RSSI range, first/last seen, uptime, free heap, AP signal, last 100 events.
### Sidebar
Always visible: node list with online/offline status dot, last seen time, event count, and a compact heartbeat line (uptime · free heap · AP signal).
A node is considered **online** if a heartbeat was received within the last 30 seconds. Falls back to beacon timestamp (60s threshold) if no heartbeat has been received yet.
### Live updates
SSE stream pushes new beacon events as they arrive. The feed updates immediately. All other views (sidebar, networks, chart, cross-node, presence) refresh on a 2-second debounce so a single scan burst doesn't flood the server with requests.
2026-04-03 14:38:38 +03:00
---
## Observations and notes
### Hidden networks
Networks with a blank SSID but valid BSSID, signal, and channel are **hidden networks** — APs configured not to broadcast their name. They are not hidden from passive scanners. The encryption showing as `OPEN` is a firmware parsing artifact; the network itself may be encrypted. The BSSID OUI still identifies the manufacturer.
### Probe requests and MAC randomization
Modern phones and laptops (iOS since ~2020, Android since ~2021) randomize their MAC address for probe requests. A randomized MAC has bit 1 of the first byte set — the dashboard detects this and labels it "Randomized MAC" instead of doing a meaningless OUI lookup.
What probe data tells you:
- **Directed probes** (with SSID): the device has connected to that network in the past
- **Wildcard probes** (empty SSID): the device is searching for any available network
- Probe data maps **client devices**, not infrastructure — complementary to beacon scan data
### OUI lookup
The `oui.txt` file is the IEEE public OUI database (39,171 entries as of download). It maps the first 3 bytes of a real MAC to a manufacturer name. Used in the Clients tab. Refresh it occasionally by re-running `deploy.sh` after downloading a fresh copy from `https://standards-oui.ieee.org/oui/oui.txt`.
---
## Current status
- [x] Arduino CLI installed, ESP32 core configured
- [x] Node firmware: beacon scan + probe sniffing
- [x] Three ESP32-S3 nodes flashed and running
- [x] UDP ingestor (`udp_ingest.py`) — beacon + probe routing with field validation
- [x] Web dashboard: Feed, Networks, Cross-node, Clients, Node detail views
- [x] OUI vendor lookup in Clients view
- [x] Dashboard split into HTML / CSS / JS (no monolithic file)
- [x] Deployed to Orange Pi as systemd services (runs 24/7)
- [x] All three nodes sending to Orange Pi, confirmed live
- [x] Node heartbeat every 10s — uptime, free heap, AP signal, drives online/offline status
- [x] RSSI history chart per network (canvas, per-node coloured lines, 1h/2h/6h/24h range)
- [x] Presence tab — Present Now, New Arrivals, Regulars
2026-04-03 14:38:38 +03:00
- [ ] Scan interval control from dashboard
- [ ] DB pruning / retention policy (events.db grows indefinitely)
2026-04-03 14:38:38 +03:00
---
## Python dependencies
```
fastapi
uvicorn
```
```bash
pip install fastapi uvicorn
```