Add BLE layer: firmware, coordinator support, dashboard tab

2026-04-27 14:21:02 +03:00
parent 2ebafc5584
commit 1ca2dc21f5
8 changed files with 807 additions and 13 deletions
@@ -31,12 +31,14 @@ This is a learning/research project covering distributed systems, event-driven a
 ### Active nodes
-| node_id  | MAC               | Location                     |
+| node_id  | MAC               | Firmware | Location                     |
-|----------|-------------------|------------------------------|
+|----------|-------------------|----------|------------------------------|
-| F68D6E30 | 44:1b:f6:8d:6e:30 | room 1 (permanent)           |
+| F68D6E30 | 44:1b:f6:8d:6e:30 | WiFi     | room 1 (permanent)           |
-| A1D658D4 | e0:72:a1:d6:58:d4 | room 2 (permanent)           |
+| A1D658D4 | e0:72:a1:d6:58:d4 | WiFi     | room 2 (permanent)           |
-| A1D700C4 | e0:72:a1:d7:00:c4 | room 3 (permanent)           |
+| A1D700C4 | e0:72:a1:d7:00:c4 | BLE      | room 3 (permanent)           |
-| A1D6F190 | e0:72:a1:d6:f1:90 | dev machine (desk, USB)      |
+| A1D6F190 | e0:72:a1:d6:f1:90 | BLE      | dev machine (desk, USB)      |
 WiFi nodes stay on their existing firmware. BLE nodes are reflashed with `firmware/ble_node/`.
 ---
@@ -53,11 +55,14 @@ This is a learning/research project covering distributed systems, event-driven a
 ## Architecture
 ```
-[ESP32 nodes] --UDP 5005--> [udp_ingest.py] --write--> [events.db (SQLite)]
+[WiFi nodes]  --UDP 5005--> [udp_ingest.py] --write--> [events.db (SQLite)]
 [BLE nodes]   --UDP 5005-+                                     |
                                                               |
 [Browser] <--HTTP/SSE 8080-- [dashboard.py] ------read--------+
 ```
 Both node types share the same UDP ingestor and database. They are distinguished by event `type` field. BLE nodes send `ble_adv` and `heartbeat` events; WiFi nodes send `beacon`, `probe`, `deauth`, `assoc`, and `heartbeat` events.
 The two coordinator processes are intentionally separate. The ingestor has one job: receive UDP, write to DB. The dashboard has one job: read DB, serve UI. They communicate only through the database. Each is independently restartable without affecting the other.
 ---
@@ -263,6 +268,95 @@ The same binary works on every node — `node_id` is auto-derived from the MAC,
 ---
 ## BLE Firmware
 BLE nodes run `firmware/ble_node/`. Same hardware (ESP32-S3), same transport (UDP JSON to coordinator), different job.
 ### What BLE nodes do
 - Connect to WiFi `sandbox` for UDP transport only (no WiFi scanning)
 - Run a passive BLE scan continuously (5-second windows, auto-restart)
 - Parse each advertisement: MAC, address type, device name, manufacturer specific data (company ID + payload)
 - Dedup by MAC within a 30-second window to avoid flooding the coordinator
 - Buffer events in a FreeRTOS queue, flush every 5 seconds
 - Send a heartbeat every 10 seconds (same format as WiFi nodes, adds `ble_drops` field)
 The ESP32 radio is shared between WiFi and BLE via the hardware coexistence controller. WiFi only transmits briefly during UDP flushes, so BLE scan coverage is near-continuous.
 ### BLE event format
 ```json
 {
  "node_id":  "A1D700C4",
  "ts":       12345,
  "type":     "ble_adv",
  "mac":      "AA:BB:CC:DD:EE:FF",
  "addr_type": 1,
  "name":     "DeviceName",
  "rssi":     -65,
  "mfr_id":   76,
  "mfr_data": "1201..."
 }
 ```
 - `addr_type`: `0` = public (real, stable MAC), `1` = random (rotates periodically)
 - `mfr_id`: 16-bit little-endian Bluetooth company ID, `-1` if the advertisement has no manufacturer data
 - `mfr_data`: hex string of the manufacturer payload **after** the 2-byte company ID, up to 16 bytes
 ### Known company IDs
 | ID (hex) | Vendor |
 |----------|--------|
 | 0x004C | Apple (AirTags, AirPods, iBeacon, FindMy, HomeKit) |
 | 0x0075 | Samsung (SmartTag, Galaxy devices) |
 | 0x0006 | Microsoft (Swift Pair) |
 | 0x00E0 | Google |
 | 0x0059 | Nordic Semiconductor (common in IoT devices) |
 | 0x0138 | Garmin |
 | 0x0499 | Ruuvi Innovations |
 | 0x0157 | Polar Electro |
 | 0x00BD | Fitbit |
 | 0x0171 | Amazon |
 Apple type byte (first byte of `mfr_data` when `mfr_id` = 76):
 | Byte | Device |
 |------|--------|
 | `02` | iBeacon |
 | `05` | AirDrop |
 | `07` | HomeKit |
 | `09` | AirPods |
 | `0a` | AirPods Pro |
 | `0f` | AirPods 3rd gen |
 | `12` | FindMy / AirTag |
 | `15` | Proximity Pair |
 ### Flashing a BLE node
 Same requirements as WiFi nodes. Flash `firmware/ble_node/` instead of `firmware/node/`:
 ```bash
 # Compile
 arduino-cli compile --fqbn esp32:esp32:esp32s3 /home/sandbox/Documents/projects/esp32_cluster/firmware/ble_node/
 # Flash
 arduino-cli upload --fqbn esp32:esp32:esp32s3 --port /dev/ttyACM0 \
  /home/sandbox/Documents/projects/esp32_cluster/firmware/ble_node/
 ```
 ### config.h reference (BLE node)
 | Constant | Default | Purpose |
 |----------|---------|---------|
 | `BLE_SCAN_DURATION_SECS` | `5` | Passive scan window length |
 | `BLE_FLUSH_INTERVAL_MS` | `5000` | How often to send queued events to coordinator |
 | `HEARTBEAT_INTERVAL_MS` | `10000` | Heartbeat cadence |
 | `BLE_DEDUP_SECS` | `30` | Suppress same MAC within this window |
 | `BLE_DEDUP_CACHE_SIZE` | `300` | Dedup ring-buffer entries |
 | `BLE_QUEUE_SIZE` | `256` | Max events buffered between flushes |
 ---
 ## Orange Pi — running services
 Services are managed by systemd and start automatically on boot.
@@ -381,7 +475,9 @@ The `oui.txt` file is the IEEE public OUI database (39,171 entries as of downloa
 ## Active investigations
 ### Sustained deauth attack on Tuya device
-A persistent automated deauth flood has been running since **2026-04-03**, targeting `38:2C:E5:7E:77:1D` (Tuya Smart Inc. device). Four spoofed source MACs fire simultaneously roughly every hour, all using reason code 2, impersonating real AP BSSIDs in the building. All 4 nodes confirm it. Consistent with an automated WPA2 handshake capture tool. No action taken yet — being passively monitored.
+~~A persistent automated deauth flood has been running since **2026-04-03**~~
 **Status as of 2026-04-25: attack has wound down.** Down from hundreds of frames/day to 1–7 frames/day. The direct target `38:2C:E5:7E:77:1D` is no longer being hit. Two of the four attacker MACs (`82:4E:66:47:09:C1`, `62:87:CB:38:2C:20`) have gone silent. `42:8C:46:6E:12:9C` and `62:D9:AA:E8:B4:09` still appear occasionally at very low volume. Conclusion: whoever was running the tool either captured the WPA2 handshake they needed or moved away. Keep in watch list for re-activation.
 Attacker src MACs: `82:4E:66:47:09:C1`, `42:8C:46:6E:12:9C`, `62:D9:AA:E8:B4:09`, `62:87:CB:38:2C:20`
@@ -392,6 +488,64 @@ Broadcasting an open unauthenticated hotspot. Next step: connect via Parrot lapt
 ---
 ## BLE observations and notes
 ### Heap baseline
 BLE nodes floor out at **109–127K free heap** vs 222K+ for WiFi nodes. The ~100K difference is the BLE stack overhead (BLEDevice + BLEScan + coexistence controller). This is expected and stable — no downward drift observed after 21 hours. The WiFi-only queue flush model keeps heap use predictable.
 ### MAC randomization and device counting
 BLE unique MAC counts are misleading without context:
 - **Apple FindMy / AirTag** rotates MAC on every advertisement interval (seconds). One physical AirTag generates hundreds of apparent "unique" MACs per day. The `mfr_data` payload is more useful for identity than the MAC.
 - **Sony WH-1000XM5** rotates MAC every ~15 minutes. One device appears as ~28 different MACs over a day. Identify by device name (`LE_WH-1000XM5`) rather than MAC.
 - **Public MACs** (addr_type = public) are stable and reliably trackable (B&W PX5, ELK-BLEDOM, ATH-M50xBT2).
 - **Apple Proximity Pair / Nearby Info** are iPhones advertising presence — high volume, all randomized.
 ### Apple type byte reference (first byte of mfr_data when mfr_id = 76)
 Most common types seen in practice:
 | Byte | Type | Notes |
 |------|------|-------|
 | `10` | Proximity Pair | iPhone/Watch advertising nearby — very common |
 | `12` | FindMy / AirTag | MAC rotates every advertisement; best RSSI seen: −38 dBm |
 | `16` | Nearby Info | iPhone battery/status broadcast |
 | `0c` | AirPods case / Watch | Seen when case lid is open or Watch is unlocked |
 | `07` | HomeKit | Smart home accessory (lights, locks, sensors) |
 | `09`/`0a`/`0f` | AirPods (various gen) | Seen when case is open or buds are in use |
 | `02` | iBeacon | Retail/venue tracking beacons |
 ### Identified permanent BLE neighbours (as of 2026-04-26)
 | Device | MAC type | Pattern | Notes |
 |--------|----------|---------|-------|
 | Sony WH-1000XM5 | Random (rotates ~15min) | All day + late night | Likely 1 device; user wears them most of the day, leaves connected overnight |
 | B&W PX5 headphones | Public — stable | Weekday afternoons ~12:00–18:00 | Consistent with WFH office hours |
 | ELK-BLEDOM LED strip | Public — stable | Evenings | BLE-controlled RGB LED strip, cheap Chinese controller |
 | Audio-Technica ATH-M50xBT2 | Public — stable | Afternoon sessions | Professional wireless headphones |
 | `0x5148` device (c2:fe:68:e8:01:a6) | Public — stable | Persistent all day | Unknown company ID, payload ASCII "364656", best RSSI −56 dBm — very close, identity unknown |
 | "net" (80:3e:4f:1b:4f:e3) | Public — stable | 07:00–20:00 | Short device name, likely a smart home hub or IoT device |
 ### Parking structure BLE
 The same parking structure that generates hundreds of 70mai dashcam WiFi beacons also contains AirTags. The FindMy/AirTag type (`0x12`) consistently hits −38 dBm best RSSI — some are close enough to be in vehicles parked right outside. At least one is persistent across multiple scans, suggesting a parked vehicle with an AirTag rather than a passing pedestrian.
 ### WiFi-derived context: notable open networks
 For reference during future BLE correlation work, the known open networks in range:
 | SSID | Device | Notes |
 |------|--------|-------|
 | `Living Room speaker.n078` | Unknown speaker | Unauthenticated provisioning hotspot, RSSI −38 dBm, same building. **Pending active investigation.** |
 | `yeelink-light-strip2_miap7AB7` | Xiaomi Yeelight LED strip | Stuck in setup mode for 8+ days, open |
 | `xiaomi-fryer-maf65_mibt962C` | Xiaomi smart air fryer | Open setup AP |
 | `zhimi-airp-mb5_mibt5FF0` | Xiaomi Mi Air Purifier MB5 | Open setup AP |
 | `REOLINK-HHRYzJCESD-2.4G` | Reolink IP camera | Permanent neighbour's camera, 8 days uptime |
 ---
 ## Current status
 - [x] Arduino CLI installed, ESP32 core configured
@@ -423,6 +577,9 @@ Broadcasting an open unauthenticated hotspot. Next step: connect via Parrot lapt
 - [x] Sessions tab — dominant reason code + description shown per session row and in expanded detail panel
 - [x] SSE reconnect bug fix — multiple stale EventSource instances were accumulating on reconnect, flooding /api/nodes on tab re-open; fixed with proper close-before-reconnect and in-flight guard
 - [x] WAL corruption fix — recurring DB corruption under sustained write load; fixed with RESTART checkpointing every 30 minutes and reduced retention to 7 days (see below)
 - [x] BLE firmware (`firmware/ble_node/`) — passive scan, dedup, mfr data parsing, heartbeat
 - [x] Coordinator BLE support — `ble_events` table, ingest handler, `ble_drops` in heartbeat
 - [x] Dashboard BLE tab — devices table, manufacturer breakdown, live feed
 - [ ] Surface assoc events in dashboard (Search results or dedicated view)
 - [ ] Scan interval control from dashboard
 - [ ] Dedicated AP for nodes to isolate reconnect churn from regular network traffic
@@ -131,6 +131,49 @@ def _node_status(last_seen: str | None, threshold: int = ONLINE_SECS) -> str:
 # ─── API data builders ────────────────────────────────────────────────────────
 # ─── BLE classification ──────────────────────────────────────────────────────
 BLE_COMPANIES: dict[int, str] = {
    0x004C: "Apple",
    0x0075: "Samsung",
    0x0006: "Microsoft",
    0x00E0: "Google",
    0x0059: "Nordic Semiconductor",
    0x0138: "Garmin",
    0x0499: "Ruuvi Innovations",
    0x0157: "Polar Electro",
    0x00BD: "Fitbit",
    0x0171: "Amazon",
    0x004F: "MediaTek",
    0x001D: "Qualcomm",
 }
 # Apple manufacturer data: after the 2-byte company ID, byte 0 of mfr_data is the type.
 APPLE_TYPES: dict[str, str] = {
    "02": "iBeacon",
    "05": "AirDrop",
    "07": "HomeKit",
    "09": "AirPods",
    "0a": "AirPods Pro",
    "0b": "AirPods 2nd gen",
    "0c": "AirPods case / Watch",
    "0f": "AirPods 3rd gen",
    "12": "FindMy / AirTag",
    "13": "AirPods 4th gen",
    "15": "Proximity Pair",
    "1e": "AirPods Pro 2nd gen",
 }
 def ble_device_type(mfr_id: int | None, mfr_data: str | None) -> str:
    if mfr_id is None:
        return "Unknown"
    if mfr_id == 0x004C and mfr_data and len(mfr_data) >= 2:
        return APPLE_TYPES.get(mfr_data[:2].lower(), "Apple device")
    vendor = BLE_COMPANIES.get(mfr_id)
    return f"{vendor} device" if vendor else f"Unknown (0x{mfr_id:04X})"
 _nodes_cache: tuple[float, list] | None = None
 _networks_cache: tuple[float, list] | None = None
 _clients_cache: tuple[float, list] | None = None
@@ -138,6 +181,7 @@ _cross_node_cache: tuple[float, dict] | None = None
 _presence_cache: tuple[float, dict] | None = None
 _alerts_cache: tuple[float, dict] | None = None
 _sessions_cache: tuple[float, list] | None = None
 _ble_cache: tuple[float, dict] | None = None
 _NODES_CACHE_TTL      = 10   # seconds
 _NETWORKS_CACHE_TTL   = 30   # seconds
 _CLIENTS_CACHE_TTL    = 30   # seconds
@@ -145,6 +189,7 @@ _CROSS_NODE_CACHE_TTL = 60   # seconds
 _PRESENCE_CACHE_TTL   = 60   # seconds
 _ALERTS_CACHE_TTL     = 60   # seconds
 _SESSIONS_CACHE_TTL   = 60   # seconds
 _BLE_CACHE_TTL        = 30   # seconds
 def build_nodes() -> list[dict]:
    global _nodes_cache
@@ -1190,6 +1235,12 @@ async def api_node_detail(node_id: str):
    return detail
@app.get("/api/ble")
 async def api_ble():
    data = await asyncio.to_thread(build_ble)
    return data
@app.get("/api/stream")
 async def api_stream():
    """SSE stream — polls DB every 2 s, pushes new events to the browser."""
@@ -1225,6 +1276,65 @@ async def api_stream():
    )
 def build_ble() -> dict:
    global _ble_cache
    now = datetime.datetime.utcnow().timestamp()
    if _ble_cache and (now - _ble_cache[0]) < _BLE_CACHE_TTL:
        return _ble_cache[1]
    cutoff = _hot_cutoff()
    devices = query("""
        SELECT
            mac, addr_type,
            MAX(name)        AS name,
            mfr_id,
            MAX(mfr_data)    AS mfr_data,
            MAX(rssi)        AS best_rssi,
            COUNT(*)         AS times_seen,
            MIN(received_at) AS first_seen,
            MAX(received_at) AS last_seen,
            COUNT(DISTINCT node_id) AS node_count
        FROM ble_events
        WHERE received_at >= ?
        GROUP BY mac
        ORDER BY times_seen DESC
        LIMIT 300
    """, (cutoff,))
    for d in devices:
        d["type"]   = ble_device_type(d["mfr_id"], d["mfr_data"])
        d["vendor"] = BLE_COMPANIES.get(d["mfr_id"], "Unknown") if d["mfr_id"] is not None else "Unknown"
    feed = query("""
        SELECT received_at, node_id, mac, addr_type, name, rssi, mfr_id, mfr_data
        FROM ble_events
        WHERE received_at >= ?
        ORDER BY id DESC
        LIMIT 100
    """, (cutoff,))
    for f in feed:
        f["type"] = ble_device_type(f["mfr_id"], f["mfr_data"])
    # Manufacturer breakdown
    breakdown = query("""
        SELECT mfr_id, COUNT(DISTINCT mac) AS unique_devices
        FROM ble_events
        WHERE received_at >= ?
        GROUP BY mfr_id
        ORDER BY unique_devices DESC
        LIMIT 20
    """, (cutoff,))
    for b in breakdown:
        b["vendor"] = BLE_COMPANIES.get(b["mfr_id"], "Unknown") if b["mfr_id"] is not None else "No mfr data"
    result = {"devices": devices, "feed": feed, "breakdown": breakdown}
    _ble_cache = (now, result)
    return result
 # ─── Entry point ─────────────────────────────────────────────────────────────
 if __name__ == "__main__":
@@ -607,6 +607,33 @@ td.disassoc-type { color: var(--orange); }
  border-bottom: 1px solid var(--border);
 }
 /* ── BLE ── */
 .ble-type-apple    { color: #a8c8ff; }
 .ble-type-samsung  { color: #7eb3ff; }
 .ble-type-other    { color: var(--text-muted); }
 .ble-addr-public   { color: var(--green); font-size: 10px; }
 .ble-addr-random   { color: var(--text-muted); font-size: 10px; }
 .ble-breakdown-row {
  display: flex;
  flex-wrap: wrap;
  gap: 8px;
  margin-bottom: 4px;
 }
 .ble-breakdown-chip {
  background: var(--surface);
  border: 1px solid var(--border);
  border-radius: 4px;
  padding: 4px 10px;
  font-size: 11px;
  color: var(--text-muted);
 }
 .ble-breakdown-chip strong {
  color: var(--text);
  font-size: 13px;
  margin-right: 4px;
 }
 /* ── Scrollbar ── */
 ::-webkit-scrollbar { width: 5px; height: 5px; }
 ::-webkit-scrollbar-track { background: var(--bg); }
@@ -1190,6 +1190,9 @@ function setView(view) {
  document.getElementById('tab-sessions').classList.toggle('active',   view === 'sessions');
  document.getElementById('tab-analysis').classList.toggle('active',   view === 'analysis');
  document.getElementById('tab-search').classList.toggle('active',     view === 'search');
  document.getElementById('view-ble').classList.toggle('active',       view === 'ble');
  document.getElementById('tab-ble').classList.toggle('active',        view === 'ble');
  if (view !== 'ble' && _blePoll) { clearInterval(_blePoll); _blePoll = null; }
 }
 function showFeed() {
@@ -1280,6 +1283,7 @@ function startSSE() {
      else if (currentView === 'presence')      await fetchPresence();
      else if (currentView === 'alerts')        await fetchAlerts();
      else if (currentView === 'analysis')      await showAnalysis();
      else if (currentView === 'ble')           await fetchBle();
      if (!_nodesFetching) {
        _nodesFetching = true;
@@ -1306,6 +1310,96 @@ function startSSE() {
  };
 }
 // ── BLE ───────────────────────────────────────────────────────────────────────
 let bleData    = { devices: [], feed: [], breakdown: [] };
 let _blePoll   = null;
 function showBle() {
  setView('ble');
  fetchBle();
  if (_blePoll) clearInterval(_blePoll);
  _blePoll = setInterval(fetchBle, 5000);
 }
 async function fetchBle() {
  try {
    bleData = await fetch('/api/ble').then(r => r.json());
    renderBle();
  } catch (e) { /* network hiccup */ }
 }
 function bleTypeClass(type) {
  if (!type) return 'ble-type-other';
  if (type.startsWith('Apple') || type === 'iBeacon' || type.includes('AirPods') ||
      type.includes('FindMy') || type.includes('AirTag') || type.includes('HomeKit') ||
      type.includes('AirDrop') || type.includes('Proximity'))
    return 'ble-type-apple';
  if (type.startsWith('Samsung')) return 'ble-type-samsung';
  return 'ble-type-other';
 }
 function renderBle() {
  // Breakdown chips
  const bd = bleData.breakdown || [];
  const bdEl = document.getElementById('ble-breakdown');
  document.getElementById('ble-breakdown-count').textContent = bd.length ? `${bd.length} manufacturers` : '';
  bdEl.innerHTML = bd.map(b => {
    const label = b.vendor && b.vendor !== 'Unknown' && b.vendor !== 'No mfr data'
      ? b.vendor
      : b.mfr_id != null ? `0x${b.mfr_id.toString(16).padStart(4,'0').toUpperCase()}` : 'No mfr data';
    return `<div class="ble-breakdown-chip"><strong>${b.unique_devices}</strong>${label}</div>`;
  }).join('');
  // Devices table
  const devs = bleData.devices || [];
  const devsEl = document.getElementById('ble-devices-body');
  const emptyEl = document.getElementById('ble-devices-empty');
  const tableEl = document.getElementById('ble-devices-table');
  document.getElementById('ble-devices-count').textContent = devs.length ? `${devs.length} unique` : '';
  if (!devs.length) {
    emptyEl.style.display = '';
    tableEl.style.display = 'none';
  } else {
    emptyEl.style.display = 'none';
    tableEl.style.display = '';
    devsEl.innerHTML = devs.map(d => {
      const typeClass = bleTypeClass(d.type);
      const addrClass = d.addr_type === 'public' ? 'ble-addr-public' : 'ble-addr-random';
      const name = d.name || '<span class="muted">—</span>';
      const vendor = d.vendor && d.vendor !== 'Unknown' ? d.vendor : '<span class="muted">—</span>';
      return `<tr>
        <td class="${typeClass}">${fmt(d.type)}</td>
        <td class="mono">${fmt(d.mac)}</td>
        <td>${name}</td>
        <td>${vendor}</td>
        <td class="${addrClass}">${d.addr_type || '—'}</td>
        <td class="${rssiClass(d.best_rssi)}">${fmt(d.best_rssi)} dBm</td>
        <td>${fmt(d.times_seen)}</td>
        <td>${fmt(d.node_count)}</td>
        <td class="muted">${shortTime(d.last_seen)}</td>
      </tr>`;
    }).join('');
  }
  // Feed
  const feed = bleData.feed || [];
  document.getElementById('ble-feed-count').textContent = feed.length ? `last ${feed.length}` : '';
  document.getElementById('ble-feed-body').innerHTML = feed.map(f => {
    const typeClass = bleTypeClass(f.type);
    const addrClass = f.addr_type === 'public' ? 'ble-addr-public' : 'ble-addr-random';
    return `<tr>
      <td class="muted" style="font-size:11px">${shortTime(f.received_at)}</td>
      <td class="muted">${fmt(f.node_id)}</td>
      <td class="${typeClass}">${fmt(f.type)}</td>
      <td class="mono">${fmt(f.mac)}</td>
      <td>${f.name || '<span class="muted">—</span>'}</td>
      <td class="${rssiClass(f.rssi)}">${fmt(f.rssi)} dBm</td>
      <td class="${addrClass}">${f.addr_type || '—'}</td>
    </tr>`;
  }).join('');
 }
 // ── Boot ──────────────────────────────────────────────────────────────────────
 initNetworkSort();
 init().then(startSSE);
@@ -33,6 +33,7 @@
    <button class="tab-btn"        id="tab-sessions"   onclick="showSessions()">Sessions</button>
    <button class="tab-btn"        id="tab-analysis"   onclick="showAnalysis()">Analysis</button>
    <button class="tab-btn"        id="tab-search"     onclick="showSearch()">Search</button>
    <button class="tab-btn"        id="tab-ble"        onclick="showBle()">BLE</button>
  </div>
  <!-- Sidebar -->
@@ -42,6 +43,7 @@
      <div class="stat-row"><span>nodes</span><span id="sum-nodes">—</span></div>
      <div class="stat-row"><span>online</span><span id="sum-online">—</span></div>
      <div class="stat-row"><span>events (session)</span><span id="sum-events">—</span></div>
      <div class="stat-row muted"><span>data window</span><span>30 days</span></div>
    </div>
    <div class="sidebar-section">
      <div class="sidebar-label">Nodes</div>
@@ -327,7 +329,7 @@
            <table>
              <thead><tr>
                <th>BSSID</th><th>SSID</th><th>Frames</th>
-                <th>Unique Srcs</th><th>Nodes</th><th>Last Seen</th>
+                <th>Dom. Reason</th><th>Unique Srcs</th><th>Nodes</th><th>Last Seen</th>
              </tr></thead>
              <tbody id="targets-tbody"></tbody>
            </table>
@@ -344,7 +346,7 @@
            <table>
              <thead><tr>
                <th>Dst MAC</th><th>Vendor</th><th>Frames</th>
-                <th>Networks Used</th><th>Nodes</th><th>Last Seen</th>
+                <th>Dom. Reason</th><th>Networks Used</th><th>Nodes</th><th>Last Seen</th>
              </tr></thead>
              <tbody id="attackers-tbody"></tbody>
            </table>
@@ -426,6 +428,7 @@
              <th class="sortable" data-col="targets"       data-arrow="">Targets</th>
              <th class="sortable" data-col="node_count"    data-arrow="">Nodes</th>
              <th class="sortable" data-col="peak_rssi"     data-arrow="">Peak RSSI</th>
              <th>Dominant Reason</th>
              <th class="sortable" data-col="severity"      data-arrow="">Severity</th>
            </tr>
          </thead>
@@ -462,6 +465,50 @@
      </div>
    </div>
    <!-- BLE view -->
    <div id="view-ble" class="view">
      <!-- Manufacturer breakdown -->
      <div class="section-header">
        <span class="section-title">Manufacturers</span>
        <span class="section-count" id="ble-breakdown-count"></span>
      </div>
      <div id="ble-breakdown" class="ble-breakdown-row"></div>
      <!-- Devices table -->
      <div class="section-header" style="margin-top:18px">
        <span class="section-title">Devices</span>
        <span class="section-count" id="ble-devices-count"></span>
        <span class="presence-sub">unique MACs — last 7 days</span>
      </div>
      <div class="empty" id="ble-devices-empty" style="display:none">No BLE data yet. Flash a node with BLE firmware first.</div>
      <table class="data-table" id="ble-devices-table">
        <thead>
          <tr>
            <th>Type</th><th>MAC</th><th>Name</th><th>Vendor</th>
            <th>Addr</th><th>Best RSSI</th><th>Seen</th><th>Nodes</th><th>Last seen</th>
          </tr>
        </thead>
        <tbody id="ble-devices-body"></tbody>
      </table>
      <!-- Live feed -->
      <div class="section-header" style="margin-top:24px">
        <span class="section-title">Recent Advertisements</span>
        <span class="section-count" id="ble-feed-count"></span>
      </div>
      <table class="data-table" id="ble-feed-table">
        <thead>
          <tr>
            <th>Time</th><th>Node</th><th>Type</th><th>MAC</th>
            <th>Name</th><th>RSSI</th><th>Addr</th>
          </tr>
        </thead>
        <tbody id="ble-feed-body"></tbody>
      </table>
    </div><!-- /#view-ble -->
  </div><!-- /#main -->
 </div><!-- /#app -->
@@ -89,6 +89,26 @@ def init_db(path: str) -> sqlite3.Connection:
            rssi        INTEGER
        )
    """)
    conn.execute("""
        CREATE TABLE IF NOT EXISTS ble_events (
            id          INTEGER PRIMARY KEY AUTOINCREMENT,
            received_at TEXT NOT NULL,
            node_id     TEXT,
            node_ts     INTEGER,
            mac         TEXT,
            addr_type   TEXT,
            name        TEXT,
            rssi        INTEGER,
            mfr_id      INTEGER,
            mfr_data    TEXT
        )
    """)
    # Safe migration: add ble_drops to heartbeat table if not present yet
    try:
        conn.execute("ALTER TABLE heartbeat_events ADD COLUMN ble_drops INTEGER")
        conn.commit()
    except sqlite3.OperationalError:
        pass
    # Indexes for common query patterns
    conn.executescript("""
        CREATE INDEX IF NOT EXISTS idx_beacon_received_at  ON beacon_events  (received_at);
@@ -107,6 +127,10 @@ def init_db(path: str) -> sqlite3.Connection:
        CREATE INDEX IF NOT EXISTS idx_assoc_node_id       ON assoc_events   (node_id);
        CREATE INDEX IF NOT EXISTS idx_assoc_src           ON assoc_events   (src);
        CREATE INDEX IF NOT EXISTS idx_assoc_bssid         ON assoc_events   (bssid);
        CREATE INDEX IF NOT EXISTS idx_ble_received_at      ON ble_events      (received_at);
        CREATE INDEX IF NOT EXISTS idx_ble_node_id          ON ble_events      (node_id);
        CREATE INDEX IF NOT EXISTS idx_ble_mac              ON ble_events      (mac);
        CREATE INDEX IF NOT EXISTS idx_ble_mfr_id           ON ble_events      (mfr_id);
    """)
    conn.commit()
    return conn
@@ -136,8 +160,8 @@ def store_heartbeat(conn: sqlite3.Connection, ev: dict, received_at: str):
    conn.execute("""
        INSERT INTO heartbeat_events
            (received_at, node_id, node_ts, uptime_ms, free_heap, wifi_rssi,
-             probe_drops, deauth_drops, assoc_drops)
+             probe_drops, deauth_drops, assoc_drops, ble_drops)
-        VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
+        VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
    """, (
        received_at,
        ev.get("node_id"),
@@ -148,6 +172,7 @@ def store_heartbeat(conn: sqlite3.Connection, ev: dict, received_at: str):
        ev.get("probe_drops"),
        ev.get("deauth_drops"),
        ev.get("assoc_drops"),
        ev.get("ble_drops"),
    ))
    conn.commit()
@@ -270,6 +295,37 @@ def validate_assoc(ev: dict) -> str | None:
    return None
 def store_ble(conn: sqlite3.Connection, ev: dict, received_at: str):
    mfr_id = ev.get("mfr_id")
    conn.execute("""
        INSERT INTO ble_events
            (received_at, node_id, node_ts, mac, addr_type, name, rssi, mfr_id, mfr_data)
        VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)
    """, (
        received_at,
        ev.get("node_id"),
        ev.get("ts"),
        ev.get("mac"),
        "public" if ev.get("addr_type") == 0 else "random",
        ev.get("name", ""),
        ev.get("rssi"),
        mfr_id if mfr_id is not None and mfr_id >= 0 else None,
        ev.get("mfr_data", ""),
    ))
    conn.commit()
 def validate_ble(ev: dict) -> str | None:
    for field in ("node_id", "mac", "rssi"):
        if ev.get(field) is None:
            return f"missing field '{field}'"
    if not _check_mac(ev["mac"]):
        return f"bad mac format: {ev['mac']}"
    if not _check_rssi(ev["rssi"]):
        return f"rssi out of range: {ev['rssi']}"
    return None
 def validate_probe(ev: dict) -> str | None:
    """Returns an error string if invalid, else None."""
    for field in ("node_id", "src_mac", "rssi"):
@@ -283,6 +339,8 @@ def validate_probe(ev: dict) -> str | None:
 # ─── Packet handling ─────────────────────────────────────────────────────────
 MAGENTA = "\033[95m"
 IMP_COLOR = {
    "high":   "\033[92m",
    "normal": "\033[0m",
@@ -303,7 +361,21 @@ def handle_packet(data: bytes, addr: tuple, conn: sqlite3.Connection):
    received_at = datetime.datetime.utcnow().isoformat(timespec="seconds")
    pkt_type    = ev.get("type", "beacon")
-    if pkt_type in ("assoc", "reassoc"):
+    if pkt_type == "ble_adv":
        err = validate_ble(ev)
        if err:
            print(f"[DROP] {addr[0]} ble_adv — {err}")
            return
        store_ble(conn, ev, received_at)
        node    = ev.get("node_id", "?")
        mac     = ev.get("mac", "?")
        name    = ev.get("name") or "<anon>"
        rssi    = ev.get("rssi", 0)
        mfr_id  = ev.get("mfr_id", -1)
        at      = "pub" if ev.get("addr_type") == 0 else "rnd"
        mfr_str = f"mfr=0x{mfr_id:04X}" if mfr_id is not None and mfr_id >= 0 else "mfr=—"
        print(f"{MAGENTA}[{received_at}] {node}  BLE    {mac} ({at})  \"{name}\"  {rssi:>4}dBm  {mfr_str}{RESET}")
    elif pkt_type in ("assoc", "reassoc"):
        err = validate_assoc(ev)
        if err:
            print(f"[DROP] {addr[0]} {pkt_type} — {err}")
@@ -0,0 +1,264 @@
 #include <WiFi.h>
 #include <WiFiUdp.h>
 #include <BLEDevice.h>
 #include <BLEScan.h>
 #include <BLEAdvertisedDevice.h>
 #include "esp_efuse.h"
 #include "esp_mac.h"
 #include "freertos/queue.h"
 #include "config.h"
 WiFiUDP udp;
 String  nodeId;
 unsigned long lastHeartbeat = 0;
 unsigned long lastFlush     = 0;
 static volatile bool scanDone  = false;
 static volatile uint32_t bleDrops = 0;
 // ─── BLE event ───────────────────────────────────────────────────────────────
 struct BleEvent {
    char    mac[18];
    uint8_t addr_type;    // 0 = public, 1 = random
    char    name[32];
    int8_t  rssi;
    int32_t mfr_id;       // company ID (little-endian), -1 if absent
    char    mfr_data[33]; // hex string of bytes after company ID, up to 16 bytes
 };
 static QueueHandle_t bleQueue;
 // ─── Dedup ───────────────────────────────────────────────────────────────────
 struct BleDedup { char mac[18]; uint32_t last_ms; };
 static BleDedup dedupCache[BLE_DEDUP_CACHE_SIZE];
 static int      dedupNext = 0;
 static bool isDuplicate(const char* mac) {
    uint32_t now = millis();
    for (int i = 0; i < BLE_DEDUP_CACHE_SIZE; i++) {
        if (dedupCache[i].last_ms == 0) continue;
        if (strcmp(dedupCache[i].mac, mac) == 0) {
            if ((now - dedupCache[i].last_ms) < (uint32_t)(BLE_DEDUP_SECS * 1000))
                return true;
            dedupCache[i].last_ms = now;
            return false;
        }
    }
    strncpy(dedupCache[dedupNext].mac, mac, 17);
    dedupCache[dedupNext].mac[17] = '\0';
    dedupCache[dedupNext].last_ms  = now;
    dedupNext = (dedupNext + 1) % BLE_DEDUP_CACHE_SIZE;
    return false;
 }
 // ─── Advertisement callback ──────────────────────────────────────────────────
 // Runs in the BLE task — must be fast and must not call UDP.
 class AdvCallback : public BLEAdvertisedDeviceCallbacks {
    void onResult(BLEAdvertisedDevice dev) override {
        char mac[18];
        strncpy(mac, dev.getAddress().toString().c_str(), 17);
        mac[17] = '\0';
        if (isDuplicate(mac)) return;
        BleEvent ev;
        memcpy(ev.mac, mac, 18);
        // BLE_ADDR_TYPE_PUBLIC = 0, everything else treated as random
        ev.addr_type = (dev.getAddressType() == BLE_ADDR_PUBLIC) ? 0 : 1;
        ev.rssi      = (int8_t)dev.getRSSI();
        ev.mfr_id    = -1;
        ev.mfr_data[0] = '\0';
        ev.name[0]     = '\0';
        // Device name — strip non-printable bytes
        if (dev.haveName()) {
            String raw = dev.getName();
            int j = 0;
            for (int i = 0; i < (int)raw.length() && j < 31; i++) {
                char c = raw[i];
                ev.name[j++] = (c >= 32 && c <= 126) ? c : '?';
            }
            ev.name[j] = '\0';
        }
        // Manufacturer specific data (AD type 0xFF)
        // First 2 bytes (little-endian) are the company ID; rest is payload.
        if (dev.haveManufacturerData()) {
            String md = dev.getManufacturerData();
            if (md.length() >= 2) {
                ev.mfr_id = (int32_t)((uint8_t)md[0] | ((uint8_t)md[1] << 8));
                int payload_len = min((int)16, (int)md.length() - 2);
                for (int i = 0; i < payload_len; i++) {
                    snprintf(ev.mfr_data + i * 2, 3, "%02x", (uint8_t)md[2 + i]);
                }
                ev.mfr_data[payload_len * 2] = '\0';
            }
        }
        if (xQueueSend(bleQueue, &ev, 0) != pdTRUE) bleDrops++;
    }
 };
 static AdvCallback advCallback;
 static BLEScan*    pBLEScan = nullptr;
 // Called from BLE task when a scan window completes.
 static void onScanComplete(BLEScanResults) {
    pBLEScan->clearResults();
    scanDone = true;
 }
 // ─── Setup ───────────────────────────────────────────────────────────────────
 void setup() {
    Serial.begin(115200);
    delay(500);
    uint8_t mac[6];
    esp_efuse_mac_get_default(mac);
    char macBuf[9];
    snprintf(macBuf, sizeof(macBuf), "%02X%02X%02X%02X", mac[2], mac[3], mac[4], mac[5]);
    nodeId = String(macBuf);
    Serial.printf("\n[BLE NODE] ID: %s\n", nodeId.c_str());
    memset(dedupCache, 0, sizeof(dedupCache));
    bleQueue = xQueueCreate(BLE_QUEUE_SIZE, sizeof(BleEvent));
    connectWiFi();
    BLEDevice::init("");
    pBLEScan = BLEDevice::getScan();
    pBLEScan->setAdvertisedDeviceCallbacks(&advCallback, true); // true = keep duplicates in scan window
    pBLEScan->setActiveScan(false);   // passive — do not send scan requests
    pBLEScan->setInterval(100);       // scan interval (ms × 0.625 = 62.5ms)
    pBLEScan->setWindow(99);          // scan window — near-continuous coverage
    pBLEScan->start(BLE_SCAN_DURATION_SECS, onScanComplete, false);
    Serial.println("[BLE] Passive scan started");
 }
 // ─── Loop ────────────────────────────────────────────────────────────────────
 void loop() {
    if (WiFi.status() != WL_CONNECTED) {
        Serial.println("[WIFI] Lost connection, reconnecting...");
        connectWiFi();
        return;
    }
    unsigned long now = millis();
    if (now - lastHeartbeat >= HEARTBEAT_INTERVAL_MS) {
        lastHeartbeat = now;
        sendHeartbeat();
    }
    if (now - lastFlush >= BLE_FLUSH_INTERVAL_MS) {
        lastFlush = now;
        flushBleQueue();
    }
    // Restart scan after each window completes
    if (scanDone) {
        scanDone = false;
        pBLEScan->start(BLE_SCAN_DURATION_SECS, onScanComplete, false);
    }
    delay(10);
 }
 // ─── WiFi ────────────────────────────────────────────────────────────────────
 void connectWiFi() {
    Serial.printf("[WIFI] Connecting to %s\n", WIFI_SSID);
    WiFi.mode(WIFI_STA);
    WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
    int attempts = 0;
    while (WiFi.status() != WL_CONNECTED && attempts < 20) {
        delay(500);
        Serial.print(".");
        attempts++;
    }
    if (WiFi.status() == WL_CONNECTED) {
        Serial.printf("\n[WIFI] Connected — IP: %s\n", WiFi.localIP().toString().c_str());
    } else {
        Serial.println("\n[WIFI] Failed to connect, will retry in loop");
    }
 }
 // ─── Send ────────────────────────────────────────────────────────────────────
 void sendBleEvent(const BleEvent& ev) {
    // Escape name for JSON
    char nameEsc[66];
    int j = 0;
    for (int i = 0; ev.name[i] && j < 64; i++) {
        if (ev.name[i] == '"' || ev.name[i] == '\\') nameEsc[j++] = '\\';
        nameEsc[j++] = ev.name[i];
    }
    nameEsc[j] = '\0';
    char buf[320];
    snprintf(buf, sizeof(buf),
        "{\"node_id\":\"%s\",\"ts\":%lu,\"type\":\"ble_adv\","
        "\"mac\":\"%s\",\"addr_type\":%u,"
        "\"name\":\"%s\",\"rssi\":%d,"
        "\"mfr_id\":%ld,\"mfr_data\":\"%s\"}",
        nodeId.c_str(), millis(),
        ev.mac, (unsigned)ev.addr_type,
        nameEsc, (int)ev.rssi,
        (long)ev.mfr_id, ev.mfr_data
    );
    udp.beginPacket(COORDINATOR_IP, COORDINATOR_PORT);
    udp.print(buf);
    udp.endPacket();
    const char* at = ev.addr_type == 0 ? "pub" : "rnd";
    Serial.printf("  [BLE] %s (%s)  \"%s\"  %ddBm  mfr=0x%04lX  data=%s\n",
        ev.mac, at, ev.name[0] ? ev.name : "<anon>",
        (int)ev.rssi, (long)(ev.mfr_id >= 0 ? ev.mfr_id : 0), ev.mfr_data);
 }
 void flushBleQueue() {
    if (WiFi.status() != WL_CONNECTED) return;
    BleEvent ev;
    while (xQueueReceive(bleQueue, &ev, 0) == pdTRUE) {
        sendBleEvent(ev);
    }
 }
 // ─── Heartbeat ───────────────────────────────────────────────────────────────
 void sendHeartbeat() {
    unsigned long now = millis();
    char buf[256];
    snprintf(buf, sizeof(buf),
        "{\"node_id\":\"%s\",\"ts\":%lu,\"type\":\"heartbeat\","
        "\"uptime_ms\":%lu,\"free_heap\":%lu,\"wifi_rssi\":%d,"
        "\"probe_drops\":0,\"deauth_drops\":0,\"assoc_drops\":0,"
        "\"ble_drops\":%lu}",
        nodeId.c_str(), now,
        now,
        (unsigned long)ESP.getFreeHeap(),
        WiFi.RSSI(),
        (unsigned long)bleDrops
    );
    udp.beginPacket(COORDINATOR_IP, COORDINATOR_PORT);
    udp.print(buf);
    udp.endPacket();
    Serial.printf("[HB] uptime=%lus  heap=%luK  ap=%ddBm  ble_drops=%lu\n",
        now / 1000,
        (unsigned long)ESP.getFreeHeap() / 1024,
        WiFi.RSSI(),
        (unsigned long)bleDrops
    );
 }
@@ -0,0 +1,23 @@
 #pragma once
 // ─── WiFi (for UDP transport only) ───────────────────────────────────────────
 #define WIFI_SSID        "sandbox"
 #define WIFI_PASSWORD    "Jaunsgads11!!"
 // ─── Coordinator ─────────────────────────────────────────────────────────────
 #define COORDINATOR_IP   "192.168.1.133"
 #define COORDINATOR_PORT 5005
 // ─── BLE scan ────────────────────────────────────────────────────────────────
 #define BLE_SCAN_DURATION_SECS  5     // each passive scan window (seconds)
 #define BLE_FLUSH_INTERVAL_MS   5000  // how often to flush queue to coordinator (ms)
 #define HEARTBEAT_INTERVAL_MS   10000 // heartbeat cadence (ms) — same as WiFi nodes
 // ─── Dedup ───────────────────────────────────────────────────────────────────
 // Suppress the same MAC within this window. Randomised-MAC devices will
 // reappear when their MAC rotates (typically every 10–15 min), which is fine.
 #define BLE_DEDUP_SECS       30   // seconds before the same MAC is allowed again
 #define BLE_DEDUP_CACHE_SIZE 300  // ring-buffer size; BLE sees far more MACs than WiFi
 // ─── Queue ───────────────────────────────────────────────────────────────────
 #define BLE_QUEUE_SIZE 256        // events buffered between flushes