# ESP32 WiFi Recon Cluster ## What this is A distributed WiFi reconnaissance system using ESP32-S3 nodes and an Orange Pi as coordinator. Each ESP32 node scans the WiFi environment, builds structured events (not raw data), and ships them over UDP to a coordinator. The coordinator stores events to SQLite. A separate web dashboard reads the database and displays everything in real time via SSE. This is a learning/research project covering distributed systems, event-driven architecture, sensor fusion, and detection engineering concepts. --- ## Hardware - **Nodes:** ESP32-S3 N16R8 (2 active, scaling later) - **Coordinator:** Orange Pi at `192.168.1.133`, runs 24/7 as systemd services - **Dev machine:** This PC at `192.168.1.101` — coding and flashing only - **Flashing:** Arduino CLI on this PC ### Active nodes | node_id | MAC | Port | |----------|-------------------|--------------| | F68D6E30 | 44:1b:f6:8d:6e:30 | /dev/ttyACM0 | | A1D6F190 | e0:72:a1:d6:f1:90 | /dev/ttyACM1 | | A1D658D4 | e0:72:a1:d6:58:d4 | /dev/ttyACM2 | --- ## Network - **ESP32 nodes connect to:** WiFi SSID `sandbox` - **Coordinator IP:** `192.168.1.133` (Orange Pi, production) - **UDP port:** `5005` - **Dashboard port:** `8080` --- ## Architecture ``` [ESP32 nodes] --UDP 5005--> [udp_ingest.py] --write--> [events.db (SQLite)] | [Browser] <--HTTP/SSE 8080-- [dashboard.py] ------read--------+ ``` The two coordinator processes are intentionally separate. The ingestor has one job: receive UDP, write to DB. The dashboard has one job: read DB, serve UI. They communicate only through the database. Each is independently restartable without affecting the other. --- ## Development workflow **All coding is done on this PC.** The Orange Pi only runs the deployed files — never edit directly on it. ``` [This PC] → edit code → rsync → [Orange Pi] → restart services ``` ### Sync to Orange Pi ```bash bash /home/sandbox/Documents/projects/esp32_cluster/deploy.sh ``` This rsyncs `coordinator/` to `/opt/esp32_cluster/coordinator/` on the Orange Pi, excluding `events.db` (the Pi keeps its own database). ### Restart services after deploy ```bash ssh root@192.168.1.133 'systemctl restart udp_ingest dashboard' ``` ### View live logs on Orange Pi ```bash # Ingestor (UDP events coming in) ssh root@192.168.1.133 'journalctl -u udp_ingest -f' # Dashboard ssh root@192.168.1.133 'journalctl -u dashboard -f' ``` --- ## Project structure ``` esp32_cluster/ ├── firmware/ │ └── node/ │ ├── node.ino # ESP32 node firmware │ └── config.h # WiFi creds, coordinator IP, scan/probe config ├── coordinator/ │ ├── udp_ingest.py # UDP listener — receives packets, validates, writes to SQLite │ ├── dashboard.py # FastAPI app — reads SQLite, serves UI + SSE stream │ ├── templates/ │ │ └── dashboard.html # Dashboard skeleton HTML │ ├── static/ │ │ ├── dashboard.css # All dashboard styles │ │ └── dashboard.js # All dashboard logic (vanilla JS) │ ├── oui.txt # IEEE OUI database for vendor lookup (39k entries) │ ├── udp_ingest.service # systemd service file for ingestor │ ├── dashboard.service # systemd service file for dashboard │ └── events.db # SQLite database — on Orange Pi only, not synced ├── deploy.sh # rsync script: this PC → Orange Pi ├── starting_plan.md # Original architecture design doc └── README.md # This file ``` --- ## Firmware Each node: - Connects to WiFi `sandbox` - Scans for nearby APs every 15 seconds (active scan, includes hidden SSIDs) - Captures probe request frames passively (promiscuous mode) between scans - Sends a heartbeat packet every 10 seconds (uptime, free heap, WiFi RSSI to router) - Sends beacon, probe, and heartbeat events as UDP JSON packets to the coordinator - `node_id` is derived from bytes 2–5 of the ESP32's base MAC address ### Event formats **Beacon event** (AP discovered): ```json { "node_id": "F68D6E30", "ts": 12345, "type": "beacon", "conf": "high", "imp": "normal", "ssid": "NetworkName", "bssid": "AA:BB:CC:DD:EE:FF", "rssi": -65, "ch": 6, "enc": "WPA2" } ``` **Probe event** (client device searching for a network): ```json { "node_id": "F68D6E30", "ts": 12345, "type": "probe", "conf": "high", "imp": "normal", "src_mac": "AA:BB:CC:DD:EE:FF", "ssid": "HomeNetwork", "rssi": -55 } ``` **Heartbeat event** (node health, sent every 10 seconds): ```json { "node_id": "F68D6E30", "ts": 12345, "type": "heartbeat", "uptime_ms": 123456, "free_heap": 245000, "wifi_rssi": -62 } ``` - `imp`: `high` if RSSI >= -50, `low` if <= -80, else `normal` - `ssid` in probe events is the network the device is searching for — empty means wildcard (any network) - Probe dedup: same MAC+SSID suppressed for 30 seconds to avoid flooding - `wifi_rssi` in heartbeat is the node's own signal to the `sandbox` AP (not a scanned network) - A node is considered **online** if a heartbeat arrived within the last 30 seconds; falls back to beacon within 60 seconds if no heartbeat has been received yet ### Flashing a node Requirements: Arduino CLI v1.4.1, ESP32 core v3.3.7, FQBN `esp32:esp32:esp32s3`, user in `dialout` group. ```bash # Compile arduino-cli compile --fqbn esp32:esp32:esp32s3 /home/sandbox/Documents/projects/esp32_cluster/firmware/node/ # Flash — adjust port as needed (check with: ls /dev/ttyACM*) arduino-cli upload --fqbn esp32:esp32:esp32s3 --port /dev/ttyACM0 \ /home/sandbox/Documents/projects/esp32_cluster/firmware/node/ ``` The same binary works on every node — `node_id` is auto-derived from the MAC, no per-node config needed. > Check group membership with `groups`. You should see `dialout`. ### config.h reference | Constant | Default | Purpose | |-------------------|------------------|-------------------------------------------| | `WIFI_SSID` | `sandbox` | WiFi network to connect to | | `COORDINATOR_IP` | `192.168.1.133` | Orange Pi address | | `COORDINATOR_PORT`| `5005` | UDP port | | `SCAN_INTERVAL_MS`| `15000` | How often to run a beacon scan (ms) | | `PROBE_SNIFF` | `1` | Enable/disable probe sniffing (1/0) | | `PROBE_DEDUP_SECS`| `30` | Suppress duplicate probe MAC+SSID (secs) | | `PROBE_QUEUE_SIZE`| `32` | Max probe events buffered between scans | --- ## Orange Pi — running services Services are managed by systemd and start automatically on boot. ```bash # Status ssh root@192.168.1.133 'systemctl status udp_ingest dashboard' # Restart both ssh root@192.168.1.133 'systemctl restart udp_ingest dashboard' # Stop ssh root@192.168.1.133 'systemctl stop udp_ingest dashboard' ``` Service files are in `coordinator/` and deployed to `/etc/systemd/system/` on the Orange Pi. The coordinator files live at `/opt/esp32_cluster/coordinator/` on the Orange Pi. --- ## Dashboard Open `http://192.168.1.133:8080` in your browser. ### Views - **Feed** — live beacon event stream across all nodes, capped at 100 rows, newest on top - **Networks** — deduplicated AP table (by BSSID): times seen, best/avg RSSI, channel, encryption, node count. Sortable by any column. Each row has a ▶ button to open the RSSI history chart for that network. - **RSSI history chart** — opens from any Networks row. Shows signal strength over time per node (1h / 2h / 6h / 24h selectable). Each node gets its own coloured line. Useful for spotting interference patterns, seeing how signal fluctuates by time of day, and comparing which node consistently hears a given AP better. Updates live as new scans arrive. - **Cross-node** — side-by-side per-node RSSI for every AP. Shows which node is physically closer to each network. Filter to multi-node only to focus on confirmed cross-node observations. - **Clients** — probe request data: unique MACs, vendor (OUI lookup), what SSIDs they're searching for, signal, which nodes saw them. - **Node detail** — click a node in the sidebar for stats: total events, unique SSIDs/BSSIDs, RSSI range, first/last seen, uptime, free heap, AP signal, last 100 events. ### Sidebar Always visible: node list with online/offline status dot, last seen time, event count, and a compact heartbeat line (uptime · free heap · AP signal). A node is considered **online** if a heartbeat was received within the last 30 seconds. Falls back to beacon timestamp (60s threshold) if no heartbeat has been received yet. ### Live updates SSE stream pushes new beacon events as they arrive. The feed updates immediately. All other views (sidebar, networks, chart, cross-node) refresh on a 2-second debounce so a single scan burst doesn't flood the server with requests. --- ## Observations and notes ### Hidden networks Networks with a blank SSID but valid BSSID, signal, and channel are **hidden networks** — APs configured not to broadcast their name. They are not hidden from passive scanners. The encryption showing as `OPEN` is a firmware parsing artifact; the network itself may be encrypted. The BSSID OUI still identifies the manufacturer. ### Probe requests and MAC randomization Modern phones and laptops (iOS since ~2020, Android since ~2021) randomize their MAC address for probe requests. A randomized MAC has bit 1 of the first byte set — the dashboard detects this and labels it "Randomized MAC" instead of doing a meaningless OUI lookup. What probe data tells you: - **Directed probes** (with SSID): the device has connected to that network in the past - **Wildcard probes** (empty SSID): the device is searching for any available network - Probe data maps **client devices**, not infrastructure — complementary to beacon scan data ### OUI lookup The `oui.txt` file is the IEEE public OUI database (39,171 entries as of download). It maps the first 3 bytes of a real MAC to a manufacturer name. Used in the Clients tab. Refresh it occasionally by re-running `deploy.sh` after downloading a fresh copy from `https://standards-oui.ieee.org/oui/oui.txt`. --- ## Current status - [x] Arduino CLI installed, ESP32 core configured - [x] Node firmware: beacon scan + probe sniffing - [x] Three ESP32-S3 nodes flashed and running - [x] UDP ingestor (`udp_ingest.py`) — beacon + probe routing with field validation - [x] Web dashboard: Feed, Networks, Cross-node, Clients, Node detail views - [x] OUI vendor lookup in Clients view - [x] Dashboard split into HTML / CSS / JS (no monolithic file) - [x] Deployed to Orange Pi as systemd services (runs 24/7) - [x] All three nodes sending to Orange Pi, confirmed live - [x] Node heartbeat every 10s — uptime, free heap, AP signal, drives online/offline status - [ ] Scan interval control from dashboard --- ## Python dependencies ``` fastapi uvicorn ``` ```bash pip install fastapi uvicorn ```