Add Presence tab, RSSI chart, heartbeat; fix SSE flood, chart timezone, N+1 clients query

This commit is contained in:
bot
2026-04-03 21:50:41 +03:00
parent a73aae21ad
commit cbef4143ee
5 changed files with 376 additions and 9 deletions
+8 -1
View File
@@ -231,6 +231,10 @@ Open `http://192.168.1.133:8080` in your browser.
- **RSSI history chart** — opens from any Networks row. Shows signal strength over time per node (1h / 2h / 6h / 24h selectable). Each node gets its own coloured line. Useful for spotting interference patterns, seeing how signal fluctuates by time of day, and comparing which node consistently hears a given AP better. Updates live as new scans arrive.
- **Cross-node** — side-by-side per-node RSSI for every AP. Shows which node is physically closer to each network. Filter to multi-node only to focus on confirmed cross-node observations.
- **Clients** — probe request data: unique MACs, vendor (OUI lookup), what SSIDs they're searching for, signal, which nodes saw them.
- **Presence** — three sections driven entirely by probe data:
- *Present Now* — real (non-randomized) MACs seen 2+ times in the last hour. Filters out the city-center noise of transient devices.
- *New Arrivals* — devices and networks first seen in the last 24 hours, split into two side-by-side tables.
- *Regulars* — devices and networks seen on 2 or more distinct calendar days. Takes at least two days of data to populate.
- **Node detail** — click a node in the sidebar for stats: total events, unique SSIDs/BSSIDs, RSSI range, first/last seen, uptime, free heap, AP signal, last 100 events.
### Sidebar
@@ -240,7 +244,7 @@ A node is considered **online** if a heartbeat was received within the last 30 s
### Live updates
SSE stream pushes new beacon events as they arrive. The feed updates immediately. All other views (sidebar, networks, chart, cross-node) refresh on a 2-second debounce so a single scan burst doesn't flood the server with requests.
SSE stream pushes new beacon events as they arrive. The feed updates immediately. All other views (sidebar, networks, chart, cross-node, presence) refresh on a 2-second debounce so a single scan burst doesn't flood the server with requests.
---
@@ -277,7 +281,10 @@ The `oui.txt` file is the IEEE public OUI database (39,171 entries as of downloa
- [x] Deployed to Orange Pi as systemd services (runs 24/7)
- [x] All three nodes sending to Orange Pi, confirmed live
- [x] Node heartbeat every 10s — uptime, free heap, AP signal, drives online/offline status
- [x] RSSI history chart per network (canvas, per-node coloured lines, 1h/2h/6h/24h range)
- [x] Presence tab — Present Now, New Arrivals, Regulars
- [ ] Scan interval control from dashboard
- [ ] DB pruning / retention policy (events.db grows indefinitely)
---
+129 -7
View File
@@ -170,14 +170,18 @@ def build_clients() -> list[dict]:
GROUP BY src_mac
ORDER BY times_seen DESC
""")
# Attach probed SSIDs and OUI vendor info to each row
for r in rows:
# Fetch all probed SSIDs in one query, group in Python — avoids N+1
ssid_rows = query("""
SELECT DISTINCT ssid FROM probe_events
WHERE src_mac = ? AND ssid IS NOT NULL AND ssid != ''
ORDER BY ssid
""", (r["src_mac"],))
r["probed_ssids"] = [s["ssid"] for s in ssid_rows]
SELECT DISTINCT src_mac, ssid FROM probe_events
WHERE src_mac IS NOT NULL AND ssid IS NOT NULL AND ssid != ''
ORDER BY src_mac, ssid
""")
ssids_by_mac: dict = {}
for s in ssid_rows:
ssids_by_mac.setdefault(s["src_mac"], []).append(s["ssid"])
for r in rows:
r["probed_ssids"] = ssids_by_mac.get(r["src_mac"], [])
vendor, randomized = oui_lookup(r["src_mac"])
r["vendor"] = vendor
r["randomized"] = randomized
@@ -237,6 +241,119 @@ def build_cross_node() -> dict:
return {"node_ids": node_ids, "networks": result}
def build_presence() -> dict:
now = datetime.datetime.utcnow()
cutoff_1h = (now - datetime.timedelta(hours=1)).isoformat(timespec="seconds")
cutoff_24h = (now - datetime.timedelta(hours=24)).isoformat(timespec="seconds")
# ── Present now: real MACs seen 2+ times in the last hour ────────────────
present_rows = query("""
SELECT src_mac,
COUNT(*) AS times_seen,
MAX(rssi) AS best_rssi,
ROUND(AVG(rssi), 1) AS avg_rssi,
MAX(received_at) AS last_seen,
COUNT(DISTINCT node_id) AS node_count
FROM probe_events
WHERE src_mac IS NOT NULL AND received_at > ?
GROUP BY src_mac
HAVING COUNT(*) > 1
ORDER BY last_seen DESC
""", (cutoff_1h,))
present = []
for r in present_rows:
vendor, randomized = oui_lookup(r["src_mac"])
if not randomized:
r["vendor"] = vendor
present.append(r)
# ── New arrivals: first seen in last 24h ─────────────────────────────────
new_mac_rows = query("""
SELECT src_mac,
MIN(received_at) AS first_seen,
MAX(rssi) AS best_rssi,
COUNT(*) AS times_seen,
COUNT(DISTINCT node_id) AS node_count
FROM probe_events
WHERE src_mac IS NOT NULL
GROUP BY src_mac
HAVING MIN(received_at) > ?
ORDER BY first_seen DESC
LIMIT 100
""", (cutoff_24h,))
new_macs = []
for r in new_mac_rows:
vendor, randomized = oui_lookup(r["src_mac"])
r["vendor"] = vendor
r["randomized"] = randomized
new_macs.append(r)
new_networks = query("""
SELECT bssid,
MAX(ssid) AS ssid,
MIN(received_at) AS first_seen,
MAX(rssi) AS best_rssi,
MAX(channel) AS channel,
MAX(encryption) AS encryption,
COUNT(DISTINCT node_id) AS node_count
FROM beacon_events
WHERE bssid IS NOT NULL
GROUP BY bssid
HAVING MIN(received_at) > ?
ORDER BY first_seen DESC
LIMIT 100
""", (cutoff_24h,))
# ── Regulars: seen on 2+ distinct calendar days ───────────────────────────
reg_mac_rows = query("""
SELECT src_mac,
COUNT(DISTINCT date(received_at)) AS days_seen,
MIN(received_at) AS first_seen,
MAX(received_at) AS last_seen,
MAX(rssi) AS best_rssi,
COUNT(*) AS times_seen,
COUNT(DISTINCT node_id) AS node_count
FROM probe_events
WHERE src_mac IS NOT NULL
GROUP BY src_mac
HAVING COUNT(DISTINCT date(received_at)) >= 2
ORDER BY days_seen DESC, times_seen DESC
LIMIT 50
""")
reg_macs = []
for r in reg_mac_rows:
vendor, randomized = oui_lookup(r["src_mac"])
r["vendor"] = vendor
r["randomized"] = randomized
reg_macs.append(r)
reg_networks = query("""
SELECT bssid,
MAX(ssid) AS ssid,
COUNT(DISTINCT date(received_at)) AS days_seen,
MIN(received_at) AS first_seen,
MAX(received_at) AS last_seen,
MAX(rssi) AS best_rssi,
MAX(channel) AS channel,
MAX(encryption) AS encryption,
COUNT(DISTINCT node_id) AS node_count
FROM beacon_events
WHERE bssid IS NOT NULL
GROUP BY bssid
HAVING COUNT(DISTINCT date(received_at)) >= 2
ORDER BY days_seen DESC, best_rssi DESC
LIMIT 50
""")
return {
"present": present,
"new_macs": new_macs,
"new_networks": new_networks,
"reg_macs": reg_macs,
"reg_networks": reg_networks,
}
def build_node_detail(node_id: str) -> dict | None:
rows = query("""
SELECT
@@ -349,6 +466,11 @@ async def api_cross_node():
return build_cross_node()
@app.get("/api/presence")
async def api_presence():
return build_presence()
@app.get("/api/rssi-history")
async def api_rssi_history(bssid: str, hours: int = 2):
hours = min(max(hours, 1), 48)
+19
View File
@@ -360,6 +360,25 @@ thead th.sort-active::after { content: ' ' attr(data-arrow); }
letter-spacing: 0.06em;
}
/* ── Presence ── */
.presence-split {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 16px;
}
.presence-sub {
font-size: 10px;
color: var(--text-muted);
margin-left: auto;
}
.presence-sub-label {
font-size: 9px;
letter-spacing: 0.1em;
text-transform: uppercase;
color: var(--text-muted);
margin-bottom: 6px;
}
/* ── Scrollbar ── */
::-webkit-scrollbar { width: 5px; height: 5px; }
::-webkit-scrollbar-track { background: var(--bg); }
+127 -1
View File
@@ -8,6 +8,7 @@ let networksData = [];
let netSort = { col: 'times_seen', dir: 'desc' };
let crossNodeData = { node_ids: [], networks: [] };
let clientsData = [];
let presenceData = {};
let chartBssid = null;
let chartHours = 2;
@@ -316,6 +317,126 @@ function renderClientsTable() {
}).join('');
}
// ── Presence view ─────────────────────────────────────────────────────────────
async function fetchPresence() {
presenceData = await fetch('/api/presence').then(r => r.json());
renderPresenceView();
}
function renderPresenceView() {
const p = presenceData;
// Present Now
const presentTbody = document.getElementById('present-tbody');
const presentEmpty = document.getElementById('present-empty');
const presentCount = document.getElementById('present-count');
const present = p.present || [];
presentCount.textContent = `${present.length} devices`;
if (!present.length) {
presentTbody.innerHTML = '';
presentEmpty.style.display = 'block';
} else {
presentEmpty.style.display = 'none';
presentTbody.innerHTML = present.map(r => {
const nodeCls = r.node_count > 1 ? 'nodes-multi' : 'muted';
return `<tr>
<td>${fmt(r.src_mac)}</td>
<td class="dim">${r.vendor || '—'}</td>
<td class="${rssiClass(r.best_rssi)}">${fmt(r.best_rssi)} dBm</td>
<td class="${rssiClass(r.avg_rssi)}">${fmt(r.avg_rssi)} dBm</td>
<td>${fmt(r.times_seen)}</td>
<td class="${nodeCls}">${fmt(r.node_count)}</td>
<td class="dim">${shortTime(r.last_seen)}</td>
</tr>`;
}).join('');
}
// New Arrivals — devices
const arrMacsTbody = document.getElementById('arrivals-macs-tbody');
const arrMacsEmpty = document.getElementById('arrivals-macs-empty');
const arrCount = document.getElementById('arrivals-count');
const newMacs = p.new_macs || [];
const newNets = p.new_networks || [];
arrCount.textContent = `${newMacs.length} devices · ${newNets.length} networks`;
if (!newMacs.length) {
arrMacsTbody.innerHTML = '';
arrMacsEmpty.style.display = 'block';
} else {
arrMacsEmpty.style.display = 'none';
arrMacsTbody.innerHTML = newMacs.map(r => `<tr>
<td>${fmt(r.src_mac)}</td>
<td class="dim">${r.vendor || '—'}</td>
<td class="dim">${shortTime(r.first_seen)}</td>
<td class="${rssiClass(r.best_rssi)}">${fmt(r.best_rssi)} dBm</td>
<td>${fmt(r.times_seen)}</td>
</tr>`).join('');
}
// New Arrivals — networks
const arrNetsTbody = document.getElementById('arrivals-nets-tbody');
const arrNetsEmpty = document.getElementById('arrivals-nets-empty');
if (!newNets.length) {
arrNetsTbody.innerHTML = '';
arrNetsEmpty.style.display = 'block';
} else {
arrNetsEmpty.style.display = 'none';
arrNetsTbody.innerHTML = newNets.map(r => {
const encCls = (!r.encryption || r.encryption === 'OPEN' || r.encryption === 'None') ? 'enc-open' : 'muted';
return `<tr>
<td>${fmt(r.ssid, '<hidden>')}</td>
<td class="dim">${fmt(r.bssid)}</td>
<td class="dim">${shortTime(r.first_seen)}</td>
<td class="${rssiClass(r.best_rssi)}">${fmt(r.best_rssi)} dBm</td>
<td class="muted">${fmt(r.channel)}</td>
<td class="${encCls}">${r.encryption || 'OPEN'}</td>
</tr>`;
}).join('');
}
// Regulars — devices
const regMacsTbody = document.getElementById('regulars-macs-tbody');
const regMacsEmpty = document.getElementById('regulars-macs-empty');
const regCount = document.getElementById('regulars-count');
const regMacs = p.regular_macs || [];
const regNets = p.regular_networks || [];
regCount.textContent = `${regMacs.length} devices · ${regNets.length} networks`;
if (!regMacs.length) {
regMacsTbody.innerHTML = '';
regMacsEmpty.style.display = 'block';
} else {
regMacsEmpty.style.display = 'none';
regMacsTbody.innerHTML = regMacs.map(r => `<tr>
<td>${fmt(r.src_mac)}</td>
<td class="dim">${r.vendor || '—'}</td>
<td>${fmt(r.days_seen)}</td>
<td class="dim">${shortTime(r.first_seen)}</td>
<td class="dim">${shortTime(r.last_seen)}</td>
<td class="${rssiClass(r.best_rssi)}">${fmt(r.best_rssi)} dBm</td>
</tr>`).join('');
}
// Regulars — networks
const regNetsTbody = document.getElementById('regulars-nets-tbody');
const regNetsEmpty = document.getElementById('regulars-nets-empty');
if (!regNets.length) {
regNetsTbody.innerHTML = '';
regNetsEmpty.style.display = 'block';
} else {
regNetsEmpty.style.display = 'none';
regNetsTbody.innerHTML = regNets.map(r => {
const encCls = (!r.encryption || r.encryption === 'OPEN' || r.encryption === 'None') ? 'enc-open' : 'muted';
return `<tr>
<td>${fmt(r.ssid, '<hidden>')}</td>
<td class="dim">${fmt(r.bssid)}</td>
<td>${fmt(r.days_seen)}</td>
<td class="dim">${shortTime(r.first_seen)}</td>
<td class="${rssiClass(r.best_rssi)}">${fmt(r.best_rssi)} dBm</td>
<td class="muted">${fmt(r.channel)}</td>
</tr>`;
}).join('');
}
}
// ── RSSI chart ────────────────────────────────────────────────────────────────
async function showNetworkChart(bssid) {
chartBssid = bssid;
@@ -344,7 +465,7 @@ function renderRssiChart(series) {
const canvas = document.getElementById('rssi-chart');
const empty = document.getElementById('chart-empty');
const legend = document.getElementById('chart-legend');
const nodeIds = Object.keys(series);
const nodeIds = Object.keys(series).sort();
const allPts = nodeIds.flatMap(id => series[id]);
if (!allPts.length) {
@@ -446,10 +567,12 @@ function setView(view) {
document.getElementById('view-crossnode').classList.toggle('active', view === 'crossnode');
document.getElementById('view-clients').classList.toggle('active', view === 'clients');
document.getElementById('view-detail').classList.toggle('active', view === 'detail');
document.getElementById('view-presence').classList.toggle('active', view === 'presence');
document.getElementById('tab-feed').classList.toggle('active', view === 'feed' || view === 'detail');
document.getElementById('tab-networks').classList.toggle('active', view === 'networks' || view === 'network-chart');
document.getElementById('tab-crossnode').classList.toggle('active', view === 'crossnode');
document.getElementById('tab-clients').classList.toggle('active', view === 'clients');
document.getElementById('tab-presence').classList.toggle('active', view === 'presence');
}
function showFeed() {
@@ -466,6 +589,8 @@ async function showCrossNode() { setView('crossnode'); await fetchCrossNode(); }
async function showClients() { setView('clients'); await fetchClients(); }
async function showPresence() { setView('presence'); await fetchPresence(); }
async function showDetail(node_id) {
currentNode = node_id;
setView('detail');
@@ -517,6 +642,7 @@ function startSSE() {
if (currentView === 'networks') await fetchNetworks();
else if (currentView === 'network-chart') await loadAndRenderChart();
else if (currentView === 'crossnode') await fetchCrossNode();
else if (currentView === 'presence') await fetchPresence();
const nodes = await fetch('/api/nodes').then(r => r.json());
nodeData = Object.fromEntries(nodes.map(n => [n.node_id, n]));
+93
View File
@@ -27,6 +27,7 @@
<button class="tab-btn" id="tab-networks" onclick="showNetworks()">Networks</button>
<button class="tab-btn" id="tab-crossnode" onclick="showCrossNode()">Cross-node</button>
<button class="tab-btn" id="tab-clients" onclick="showClients()">Clients</button>
<button class="tab-btn" id="tab-presence" onclick="showPresence()">Presence</button>
</div>
<!-- Sidebar -->
@@ -211,6 +212,98 @@
</div>
</div>
<!-- Presence view -->
<div id="view-presence" class="view">
<!-- Present now -->
<div class="section-header">
<span class="section-title">Present Now</span>
<span class="section-count" id="present-count"></span>
<span class="presence-sub">real MACs · seen 2+ times in last hour</span>
</div>
<div class="tbl-wrap">
<table>
<thead><tr>
<th>MAC</th><th>Vendor</th><th>Best RSSI</th><th>Avg RSSI</th>
<th>Times Seen</th><th>Nodes</th><th>Last Seen</th>
</tr></thead>
<tbody id="present-tbody"></tbody>
</table>
<div class="empty" id="present-empty" style="display:none">No devices currently present.</div>
</div>
<!-- New arrivals -->
<div class="section-header" style="margin-top:8px">
<span class="section-title">New Arrivals</span>
<span class="section-count" id="arrivals-count"></span>
<span class="presence-sub">first seen in last 24h</span>
</div>
<div class="presence-split">
<div>
<div class="presence-sub-label">Devices</div>
<div class="tbl-wrap">
<table>
<thead><tr>
<th>MAC</th><th>Vendor</th><th>First Seen</th>
<th>Best RSSI</th><th>Times Seen</th>
</tr></thead>
<tbody id="arrivals-macs-tbody"></tbody>
</table>
<div class="empty" id="arrivals-macs-empty" style="display:none">No new devices.</div>
</div>
</div>
<div>
<div class="presence-sub-label">Networks</div>
<div class="tbl-wrap">
<table>
<thead><tr>
<th>SSID</th><th>BSSID</th><th>First Seen</th>
<th>Best RSSI</th><th>Ch</th><th>Enc</th>
</tr></thead>
<tbody id="arrivals-nets-tbody"></tbody>
</table>
<div class="empty" id="arrivals-nets-empty" style="display:none">No new networks.</div>
</div>
</div>
</div>
<!-- Regulars -->
<div class="section-header" style="margin-top:8px">
<span class="section-title">Regulars</span>
<span class="section-count" id="regulars-count"></span>
<span class="presence-sub">seen on 2+ distinct days</span>
</div>
<div class="presence-split">
<div>
<div class="presence-sub-label">Devices</div>
<div class="tbl-wrap">
<table>
<thead><tr>
<th>MAC</th><th>Vendor</th><th>Days Seen</th>
<th>First Seen</th><th>Last Seen</th><th>Best RSSI</th>
</tr></thead>
<tbody id="regulars-macs-tbody"></tbody>
</table>
<div class="empty" id="regulars-macs-empty" style="display:none">Regulars appear after 2+ days of data.</div>
</div>
</div>
<div>
<div class="presence-sub-label">Networks</div>
<div class="tbl-wrap">
<table>
<thead><tr>
<th>SSID</th><th>BSSID</th><th>Days Seen</th>
<th>First Seen</th><th>Best RSSI</th><th>Ch</th>
</tr></thead>
<tbody id="regulars-nets-tbody"></tbody>
</table>
<div class="empty" id="regulars-nets-empty" style="display:none">Regulars appear after 2+ days of data.</div>
</div>
</div>
</div>
</div><!-- /#view-presence -->
</div><!-- /#main -->
</div><!-- /#app -->