294 lines
11 KiB
Markdown
294 lines
11 KiB
Markdown
# ESP32 WiFi Recon Cluster
|
||
|
||
## What this is
|
||
|
||
A distributed WiFi reconnaissance system using ESP32-S3 nodes and an Orange Pi as coordinator.
|
||
|
||
Each ESP32 node scans the WiFi environment, builds structured events (not raw data), and ships them over UDP to a coordinator. The coordinator stores events to SQLite. A separate web dashboard reads the database and displays everything in real time via SSE.
|
||
|
||
This is a learning/research project covering distributed systems, event-driven architecture, sensor fusion, and detection engineering concepts.
|
||
|
||
---
|
||
|
||
## Hardware
|
||
|
||
- **Nodes:** ESP32-S3 N16R8 (2 active, scaling later)
|
||
- **Coordinator:** Orange Pi at `192.168.1.133`, runs 24/7 as systemd services
|
||
- **Dev machine:** This PC at `192.168.1.101` — coding and flashing only
|
||
- **Flashing:** Arduino CLI on this PC
|
||
|
||
### Active nodes
|
||
|
||
| node_id | MAC | Port |
|
||
|----------|-------------------|--------------|
|
||
| F68D6E30 | 44:1b:f6:8d:6e:30 | /dev/ttyACM0 |
|
||
| A1D6F190 | e0:72:a1:d6:f1:90 | /dev/ttyACM1 |
|
||
| A1D658D4 | e0:72:a1:d6:58:d4 | /dev/ttyACM2 |
|
||
|
||
---
|
||
|
||
## Network
|
||
|
||
- **ESP32 nodes connect to:** WiFi SSID `sandbox`
|
||
- **Coordinator IP:** `192.168.1.133` (Orange Pi, production)
|
||
- **UDP port:** `5005`
|
||
- **Dashboard port:** `8080`
|
||
|
||
---
|
||
|
||
## Architecture
|
||
|
||
```
|
||
[ESP32 nodes] --UDP 5005--> [udp_ingest.py] --write--> [events.db (SQLite)]
|
||
|
|
||
[Browser] <--HTTP/SSE 8080-- [dashboard.py] ------read--------+
|
||
```
|
||
|
||
The two coordinator processes are intentionally separate. The ingestor has one job: receive UDP, write to DB. The dashboard has one job: read DB, serve UI. They communicate only through the database. Each is independently restartable without affecting the other.
|
||
|
||
---
|
||
|
||
## Development workflow
|
||
|
||
**All coding is done on this PC.** The Orange Pi only runs the deployed files — never edit directly on it.
|
||
|
||
```
|
||
[This PC] → edit code → rsync → [Orange Pi] → restart services
|
||
```
|
||
|
||
### Sync to Orange Pi
|
||
|
||
```bash
|
||
bash /home/sandbox/Documents/projects/esp32_cluster/deploy.sh
|
||
```
|
||
|
||
This rsyncs `coordinator/` to `/opt/esp32_cluster/coordinator/` on the Orange Pi, excluding `events.db` (the Pi keeps its own database).
|
||
|
||
### Restart services after deploy
|
||
|
||
```bash
|
||
ssh root@192.168.1.133 'systemctl restart udp_ingest dashboard'
|
||
```
|
||
|
||
### View live logs on Orange Pi
|
||
|
||
```bash
|
||
# Ingestor (UDP events coming in)
|
||
ssh root@192.168.1.133 'journalctl -u udp_ingest -f'
|
||
|
||
# Dashboard
|
||
ssh root@192.168.1.133 'journalctl -u dashboard -f'
|
||
```
|
||
|
||
---
|
||
|
||
## Project structure
|
||
|
||
```
|
||
esp32_cluster/
|
||
├── firmware/
|
||
│ └── node/
|
||
│ ├── node.ino # ESP32 node firmware
|
||
│ └── config.h # WiFi creds, coordinator IP, scan/probe config
|
||
├── coordinator/
|
||
│ ├── udp_ingest.py # UDP listener — receives packets, validates, writes to SQLite
|
||
│ ├── dashboard.py # FastAPI app — reads SQLite, serves UI + SSE stream
|
||
│ ├── templates/
|
||
│ │ └── dashboard.html # Dashboard skeleton HTML
|
||
│ ├── static/
|
||
│ │ ├── dashboard.css # All dashboard styles
|
||
│ │ └── dashboard.js # All dashboard logic (vanilla JS)
|
||
│ ├── oui.txt # IEEE OUI database for vendor lookup (39k entries)
|
||
│ ├── udp_ingest.service # systemd service file for ingestor
|
||
│ ├── dashboard.service # systemd service file for dashboard
|
||
│ └── events.db # SQLite database — on Orange Pi only, not synced
|
||
├── deploy.sh # rsync script: this PC → Orange Pi
|
||
├── starting_plan.md # Original architecture design doc
|
||
└── README.md # This file
|
||
```
|
||
|
||
---
|
||
|
||
## Firmware
|
||
|
||
Each node:
|
||
- Connects to WiFi `sandbox`
|
||
- Scans for nearby APs every 15 seconds (active scan, includes hidden SSIDs)
|
||
- Captures probe request frames passively (promiscuous mode) between scans
|
||
- Sends a heartbeat packet every 10 seconds (uptime, free heap, WiFi RSSI to router)
|
||
- Sends beacon, probe, and heartbeat events as UDP JSON packets to the coordinator
|
||
- `node_id` is derived from bytes 2–5 of the ESP32's base MAC address
|
||
|
||
### Event formats
|
||
|
||
**Beacon event** (AP discovered):
|
||
```json
|
||
{
|
||
"node_id": "F68D6E30",
|
||
"ts": 12345,
|
||
"type": "beacon",
|
||
"conf": "high",
|
||
"imp": "normal",
|
||
"ssid": "NetworkName",
|
||
"bssid": "AA:BB:CC:DD:EE:FF",
|
||
"rssi": -65,
|
||
"ch": 6,
|
||
"enc": "WPA2"
|
||
}
|
||
```
|
||
|
||
**Probe event** (client device searching for a network):
|
||
```json
|
||
{
|
||
"node_id": "F68D6E30",
|
||
"ts": 12345,
|
||
"type": "probe",
|
||
"conf": "high",
|
||
"imp": "normal",
|
||
"src_mac": "AA:BB:CC:DD:EE:FF",
|
||
"ssid": "HomeNetwork",
|
||
"rssi": -55
|
||
}
|
||
```
|
||
|
||
**Heartbeat event** (node health, sent every 10 seconds):
|
||
```json
|
||
{
|
||
"node_id": "F68D6E30",
|
||
"ts": 12345,
|
||
"type": "heartbeat",
|
||
"uptime_ms": 123456,
|
||
"free_heap": 245000,
|
||
"wifi_rssi": -62
|
||
}
|
||
```
|
||
|
||
- `imp`: `high` if RSSI >= -50, `low` if <= -80, else `normal`
|
||
- `ssid` in probe events is the network the device is searching for — empty means wildcard (any network)
|
||
- Probe dedup: same MAC+SSID suppressed for 30 seconds to avoid flooding
|
||
- `wifi_rssi` in heartbeat is the node's own signal to the `sandbox` AP (not a scanned network)
|
||
- A node is considered **online** if a heartbeat arrived within the last 30 seconds; falls back to beacon within 60 seconds if no heartbeat has been received yet
|
||
|
||
### Flashing a node
|
||
|
||
Requirements: Arduino CLI v1.4.1, ESP32 core v3.3.7, FQBN `esp32:esp32:esp32s3`, user in `dialout` group.
|
||
|
||
```bash
|
||
# Compile
|
||
arduino-cli compile --fqbn esp32:esp32:esp32s3 /home/sandbox/Documents/projects/esp32_cluster/firmware/node/
|
||
|
||
# Flash — adjust port as needed (check with: ls /dev/ttyACM*)
|
||
arduino-cli upload --fqbn esp32:esp32:esp32s3 --port /dev/ttyACM0 \
|
||
/home/sandbox/Documents/projects/esp32_cluster/firmware/node/
|
||
```
|
||
|
||
The same binary works on every node — `node_id` is auto-derived from the MAC, no per-node config needed.
|
||
|
||
> Check group membership with `groups`. You should see `dialout`.
|
||
|
||
### config.h reference
|
||
|
||
| Constant | Default | Purpose |
|
||
|-------------------|------------------|-------------------------------------------|
|
||
| `WIFI_SSID` | `sandbox` | WiFi network to connect to |
|
||
| `COORDINATOR_IP` | `192.168.1.133` | Orange Pi address |
|
||
| `COORDINATOR_PORT`| `5005` | UDP port |
|
||
| `SCAN_INTERVAL_MS`| `15000` | How often to run a beacon scan (ms) |
|
||
| `PROBE_SNIFF` | `1` | Enable/disable probe sniffing (1/0) |
|
||
| `PROBE_DEDUP_SECS`| `30` | Suppress duplicate probe MAC+SSID (secs) |
|
||
| `PROBE_QUEUE_SIZE`| `32` | Max probe events buffered between scans |
|
||
|
||
---
|
||
|
||
## Orange Pi — running services
|
||
|
||
Services are managed by systemd and start automatically on boot.
|
||
|
||
```bash
|
||
# Status
|
||
ssh root@192.168.1.133 'systemctl status udp_ingest dashboard'
|
||
|
||
# Restart both
|
||
ssh root@192.168.1.133 'systemctl restart udp_ingest dashboard'
|
||
|
||
# Stop
|
||
ssh root@192.168.1.133 'systemctl stop udp_ingest dashboard'
|
||
```
|
||
|
||
Service files are in `coordinator/` and deployed to `/etc/systemd/system/` on the Orange Pi.
|
||
The coordinator files live at `/opt/esp32_cluster/coordinator/` on the Orange Pi.
|
||
|
||
---
|
||
|
||
## Dashboard
|
||
|
||
Open `http://192.168.1.133:8080` in your browser.
|
||
|
||
### Views
|
||
|
||
- **Feed** — live beacon event stream across all nodes, capped at 100 rows, newest on top
|
||
- **Networks** — deduplicated AP table (by BSSID): times seen, best/avg RSSI, channel, encryption, node count. Sortable by any column. Each row has a ▶ button to open the RSSI history chart for that network.
|
||
- **RSSI history chart** — opens from any Networks row. Shows signal strength over time per node (1h / 2h / 6h / 24h selectable). Each node gets its own coloured line. Useful for spotting interference patterns, seeing how signal fluctuates by time of day, and comparing which node consistently hears a given AP better. Updates live as new scans arrive.
|
||
- **Cross-node** — side-by-side per-node RSSI for every AP. Shows which node is physically closer to each network. Filter to multi-node only to focus on confirmed cross-node observations.
|
||
- **Clients** — probe request data: unique MACs, vendor (OUI lookup), what SSIDs they're searching for, signal, which nodes saw them.
|
||
- **Node detail** — click a node in the sidebar for stats: total events, unique SSIDs/BSSIDs, RSSI range, first/last seen, uptime, free heap, AP signal, last 100 events.
|
||
|
||
### Sidebar
|
||
|
||
Always visible: node list with online/offline status dot, last seen time, event count, and a compact heartbeat line (uptime · free heap · AP signal).
|
||
A node is considered **online** if a heartbeat was received within the last 30 seconds. Falls back to beacon timestamp (60s threshold) if no heartbeat has been received yet.
|
||
|
||
### Live updates
|
||
|
||
SSE stream pushes new beacon events as they arrive. The feed updates immediately. All other views (sidebar, networks, chart, cross-node) refresh on a 2-second debounce so a single scan burst doesn't flood the server with requests.
|
||
|
||
---
|
||
|
||
## Observations and notes
|
||
|
||
### Hidden networks
|
||
|
||
Networks with a blank SSID but valid BSSID, signal, and channel are **hidden networks** — APs configured not to broadcast their name. They are not hidden from passive scanners. The encryption showing as `OPEN` is a firmware parsing artifact; the network itself may be encrypted. The BSSID OUI still identifies the manufacturer.
|
||
|
||
### Probe requests and MAC randomization
|
||
|
||
Modern phones and laptops (iOS since ~2020, Android since ~2021) randomize their MAC address for probe requests. A randomized MAC has bit 1 of the first byte set — the dashboard detects this and labels it "Randomized MAC" instead of doing a meaningless OUI lookup.
|
||
|
||
What probe data tells you:
|
||
- **Directed probes** (with SSID): the device has connected to that network in the past
|
||
- **Wildcard probes** (empty SSID): the device is searching for any available network
|
||
- Probe data maps **client devices**, not infrastructure — complementary to beacon scan data
|
||
|
||
### OUI lookup
|
||
|
||
The `oui.txt` file is the IEEE public OUI database (39,171 entries as of download). It maps the first 3 bytes of a real MAC to a manufacturer name. Used in the Clients tab. Refresh it occasionally by re-running `deploy.sh` after downloading a fresh copy from `https://standards-oui.ieee.org/oui/oui.txt`.
|
||
|
||
---
|
||
|
||
## Current status
|
||
|
||
- [x] Arduino CLI installed, ESP32 core configured
|
||
- [x] Node firmware: beacon scan + probe sniffing
|
||
- [x] Three ESP32-S3 nodes flashed and running
|
||
- [x] UDP ingestor (`udp_ingest.py`) — beacon + probe routing with field validation
|
||
- [x] Web dashboard: Feed, Networks, Cross-node, Clients, Node detail views
|
||
- [x] OUI vendor lookup in Clients view
|
||
- [x] Dashboard split into HTML / CSS / JS (no monolithic file)
|
||
- [x] Deployed to Orange Pi as systemd services (runs 24/7)
|
||
- [x] All three nodes sending to Orange Pi, confirmed live
|
||
- [x] Node heartbeat every 10s — uptime, free heap, AP signal, drives online/offline status
|
||
- [ ] Scan interval control from dashboard
|
||
|
||
---
|
||
|
||
## Python dependencies
|
||
|
||
```
|
||
fastapi
|
||
uvicorn
|
||
```
|
||
|
||
```bash
|
||
pip install fastapi uvicorn
|
||
```
|