A modern and elegant dashboard for network traffic visualization and analysis.
feat: Implement hourly fact tables and dynamic time-range resolution for improved stats performance and granularity.
luoleibef85c019969
diff --git a/CHANGELOG.en.md b/CHANGELOG.en.mdindex cb77e1f..89de0cb 100644--- a/CHANGELOG.en.md+++ b/CHANGELOG.en.md@@ -7,6 +7,48 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0## [Unreleased]+## [1.2.8] - 2026-02-15++### Performance++- **Up to 60x query performance improvement** 🚀+ - Added `hourly_dim_stats` / `hourly_country_stats` pre-aggregation tables, maintained in real-time on write+ - All dimension queries (domain/ip/proxy/rule/device/country) automatically route to hourly pre-aggregated tables for ranges > 6h+ - Timeseries query optimization: `getHourlyStats`, `getTrafficInRange`, `getTrafficTrend`, `getTrafficTrendAggregated` now read directly from `hourly_stats` for long-range queries instead of scanning and re-aggregating `minute_stats`+ - 7-day range queries reduced from ~10,080 rows scanned to ~168 rows+ - Per WebSocket broadcast total row scans reduced from ~20,160 to ~336+- **`resolveFactTableSplit` hybrid query strategy**: Long-range queries split into hourly (completed hours) + minute (current hour tail) for both performance and precision++### Added++- **Testing Infrastructure** 🧪+ - Introduced Vitest framework with unit tests for `traffic-writer`, `auth.service`, and `stats.service`+ - Added test helpers (`helpers.ts`)+ - Added ESLint configuration and `.env.example`+- **Time Range Picker Enhancements**+ - Added "1 hour" quick preset, replacing the default 30-minute view+ - Added "Today" quick option for trend charts (midnight to current time)+ - Moved 30-minute preset to debug short presets list+- **`BatchBuffer` module**: Standalone batch buffering module, decoupled from collector++### Fixed++- **Cookie authentication security**: Changed `secure` flag from `process.env.NODE_ENV === 'production'` to `request.protocol === 'https'`, fixing login loop on HTTP intranet environments where cookies could not be set+- **Windows emoji flag display**: Added `emoji-flag-font` CSS class to proxy-related components (list, chart, grid, interactive stats, rule stats) to fix broken flag emoji rendering on Windows++### Refactored++- **Global AuthGuard refactoring**: Extracted authentication logic from dashboard layout into a standalone `AuthGuard` component, simplified `auth.tsx` and `auth-queries.ts`+- **Collector service decomposition**: Significantly slimmed down `collector.ts` and `surge-collector.ts` by extracting `BatchBuffer` and `RealtimeStore` modules+- Removed legacy `api.ts` entry file, unified to modular controllers++### Technical Details++- `hourly_dim_stats` schema: `(backend_id, hour, dimension, dim_key, upload, download, connections)`, updated in real-time via `INSERT ... ON CONFLICT DO UPDATE`+- `resolveFactTable` / `resolveFactTableSplit` methods implemented in `BaseRepository`, shared across all repositories+- Timeseries query thresholds: `getTrafficInRange`/`getTrafficTrend` switch to `hourly_stats` at > 6h; `getTrafficTrendAggregated` switches when `bucketMinutes >= 60`+- Seamless upgrade: new tables auto-created via `CREATE TABLE IF NOT EXISTS`, existing `minute_dim_stats` data backfilled to hourly tables on first startup+## [1.2.7] - 2026-02-14### Addeddiff --git a/CHANGELOG.md b/CHANGELOG.mdindex 1c59ce7..a1b232f 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,6 +7,47 @@## [未发布]+## [1.2.8] - 2026-02-15++### 性能优化++- **查询性能大幅提升(最高 60x)** 🚀+ - 新增 `hourly_dim_stats` / `hourly_country_stats` 预聚合表,写入时实时维护+ - 所有维度表查询(domain/ip/proxy/rule/device/country)在 > 6h 范围时自动路由到小时级预聚合表+ - 时序查询优化:`getHourlyStats`、`getTrafficInRange`、`getTrafficTrend`、`getTrafficTrendAggregated` 在长范围查询时直接读取 `hourly_stats`,避免扫描 `minute_stats` 并重新聚合+ - 7 天范围查询扫描行数从 ~10,080 行降至 ~168 行+ - 每次 WebSocket broadcast 总扫描行数从 ~20,160 行降至 ~336 行+- **`resolveFactTableSplit` 混合查询策略**:长范围查询拆分为 hourly(已完成小时)+ minute(当前小时尾部),兼顾性能与精度++### 新增++- **测试基础设施** 🧪+ - 引入 Vitest 测试框架,新增 `traffic-writer`、`auth.service`、`stats.service` 单元测试+ - 新增测试辅助工具 `helpers.ts`+ - 新增 ESLint 配置和 `.env.example`+- **时间范围选择器增强**+ - 新增「1 小时」快捷预设,替代默认的 30 分钟视图+ - 趋势图新增「今天」快捷选项,从午夜到当前时间+ - 30 分钟预设移至调试模式的短预设列表+- **`BatchBuffer` 模块**:独立的批量缓冲处理模块,从 collector 中解耦++### 修复++- **Cookie 认证安全性**:将 `secure` 标志从 `process.env.NODE_ENV === 'production'` 改为 `request.protocol === 'https'`,修复 HTTP 内网环境下无法设置 Cookie 导致登录循环的问题+- **Windows 平台 emoji 国旗显示**:为 proxy 相关组件(列表、图表、Grid、交互式统计、规则统计)添加 `emoji-flag-font` 样式类,修复 Windows 下国旗 emoji 显示异常++### 重构++- **全局 AuthGuard 重构**:将认证逻辑从 dashboard layout 提取为独立的 `AuthGuard` 组件,简化 `auth.tsx` 和 `auth-queries.ts`+- **Collector 服务拆分**:`collector.ts` 和 `surge-collector.ts` 大幅瘦身,提取 `BatchBuffer` 和 `RealtimeStore` 模块+- 移除旧的 `api.ts` 入口文件,统一使用模块化控制器++### 技术细节++- `hourly_dim_stats` 表结构:`(backend_id, hour, dimension, dim_key, upload, download, connections)`,写入时通过 `INSERT ... ON CONFLICT DO UPDATE` 实时更新+- `resolveFactTable` / `resolveFactTableSplit` 方法在 `BaseRepository` 中实现,所有 Repository 共享+- 时序查询阈值:`getTrafficInRange`/`getTrafficTrend` 在 > 6h 时切换到 `hourly_stats`;`getTrafficTrendAggregated` 在 `bucketMinutes >= 60` 时切换+## [1.2.7] - 2026-02-14### 新增diff --git a/apps/collector/src/database/repositories/backend.repository.ts b/apps/collector/src/database/repositories/backend.repository.tsindex 4f55a96..4d807c5 100644--- a/apps/collector/src/database/repositories/backend.repository.ts+++ b/apps/collector/src/database/repositories/backend.repository.ts@@ -195,7 +195,9 @@ export class BackendRepository {this.db.prepare(`DELETE FROM connection_logs WHERE backend_id = ?`).run(id);this.db.prepare(`DELETE FROM minute_stats WHERE backend_id = ?`).run(id);this.db.prepare(`DELETE FROM minute_dim_stats WHERE backend_id = ?`).run(id);+ this.db.prepare(`DELETE FROM hourly_dim_stats WHERE backend_id = ?`).run(id);this.db.prepare(`DELETE FROM minute_country_stats WHERE backend_id = ?`).run(id);+ this.db.prepare(`DELETE FROM hourly_country_stats WHERE backend_id = ?`).run(id);this.db.prepare(`DELETE FROM domain_proxy_stats WHERE backend_id = ?`).run(id);this.db.prepare(`DELETE FROM ip_proxy_stats WHERE backend_id = ?`).run(id);this.db.prepare(`DELETE FROM device_stats WHERE backend_id = ?`).run(id);diff --git a/apps/collector/src/database/repositories/base.repository.ts b/apps/collector/src/database/repositories/base.repository.tsindex 32cb95f..7915e3c 100644--- a/apps/collector/src/database/repositories/base.repository.ts+++ b/apps/collector/src/database/repositories/base.repository.ts@@ -28,6 +28,136 @@ export abstract class BaseRepository {return `${date.toISOString().slice(0, 13)}:00:00`;}+ /**+ * Resolve which fact table to use based on the time range span.+ * For ranges > 6 hours, uses hourly_dim_stats (which is written to in real-time).+ * The start boundary is rounded down to the hour, introducing up to ~59 min of extra+ * data — negligible for long ranges (< 0.3% for 12h+, < 1% for 6h+).+ */+ protected resolveFactTable(+ start: string,+ end: string,+ ): { table: 'hourly_dim_stats' | 'minute_dim_stats'; startKey: string; endKey: string; timeCol: 'hour' | 'minute' } {+ const startDate = new Date(start);+ const endDate = new Date(end);++ if (Number.isNaN(startDate.getTime()) || Number.isNaN(endDate.getTime())) {+ return { table: 'minute_dim_stats', startKey: this.toMinuteKey(startDate), endKey: this.toMinuteKey(endDate), timeCol: 'minute' };+ }++ const rangeMs = endDate.getTime() - startDate.getTime();+ const SIX_HOURS_MS = 6 * 60 * 60 * 1000;++ if (rangeMs > SIX_HOURS_MS) {+ return {+ table: 'hourly_dim_stats',+ startKey: this.toHourKey(startDate),+ endKey: this.toHourKey(endDate),+ timeCol: 'hour',+ };+ }++ return {+ table: 'minute_dim_stats',+ startKey: this.toMinuteKey(startDate),+ endKey: this.toMinuteKey(endDate),+ timeCol: 'minute',+ };+ }++ /**+ * Split a time range into hourly + minute segments for precise long-range queries.+ * For ranges > 2 hours where end is in the current hour, returns two segments:+ * 1. hourly_dim_stats for all completed hours (bulk, ~60x fewer rows)+ * 2. minute_dim_stats for the current hour tail (~0-59 minutes)+ * No overlap, no gap, no data loss.+ */+ protected resolveFactTableSplit(+ start: string,+ end: string,+ ): Array<{ table: 'hourly_dim_stats' | 'minute_dim_stats'; startKey: string; endKey: string; timeCol: 'hour' | 'minute' }> {+ const startDate = new Date(start);+ const endDate = new Date(end);++ if (Number.isNaN(startDate.getTime()) || Number.isNaN(endDate.getTime())) {+ return [{ table: 'minute_dim_stats', startKey: this.toMinuteKey(startDate), endKey: this.toMinuteKey(endDate), timeCol: 'minute' }];+ }++ const rangeMs = endDate.getTime() - startDate.getTime();+ const TWO_HOURS_MS = 2 * 60 * 60 * 1000;++ // Short ranges: always use minute table (precise, fast enough)+ if (rangeMs <= TWO_HOURS_MS) {+ return [{ table: 'minute_dim_stats', startKey: this.toMinuteKey(startDate), endKey: this.toMinuteKey(endDate), timeCol: 'minute' }];+ }++ const currentHourStart = new Date();+ currentHourStart.setMinutes(0, 0, 0);++ // Purely historical: hourly only+ if (endDate.getTime() < currentHourStart.getTime()) {+ return [{ table: 'hourly_dim_stats', startKey: this.toHourKey(startDate), endKey: this.toHourKey(endDate), timeCol: 'hour' }];+ }++ // Split: hourly for completed hours + minute for current hour tail+ const segments: Array<{ table: 'hourly_dim_stats' | 'minute_dim_stats'; startKey: string; endKey: string; timeCol: 'hour' | 'minute' }> = [];++ // Hourly segment: from start up to last completed hour+ if (startDate.getTime() < currentHourStart.getTime()) {+ const lastCompletedHour = new Date(currentHourStart.getTime() - 1);+ segments.push({+ table: 'hourly_dim_stats',+ startKey: this.toHourKey(startDate),+ endKey: this.toHourKey(lastCompletedHour),+ timeCol: 'hour',+ });+ }++ // Minute segment: from current hour start (or range start if later) to end+ const minuteSegStart = startDate.getTime() >= currentHourStart.getTime() ? startDate : currentHourStart;+ segments.push({+ table: 'minute_dim_stats',+ startKey: this.toMinuteKey(minuteSegStart),+ endKey: this.toMinuteKey(endDate),+ timeCol: 'minute',+ });++ return segments;+ }++ /**+ * Split version for country fact tables.+ */+ protected resolveCountryFactTableSplit(+ start: string,+ end: string,+ ): Array<{ table: 'hourly_country_stats' | 'minute_country_stats'; startKey: string; endKey: string; timeCol: 'hour' | 'minute' }> {+ const segments = this.resolveFactTableSplit(start, end);+ return segments.map(s => ({+ table: (s.table === 'hourly_dim_stats' ? 'hourly_country_stats' : 'minute_country_stats') as 'hourly_country_stats' | 'minute_country_stats',+ startKey: s.startKey,+ endKey: s.endKey,+ timeCol: s.timeCol,+ }));+ }++ /**+ * Resolve fact table for country queries.+ * Same logic as resolveFactTable but returns country table names.+ */+ protected resolveCountryFactTable(+ start: string,+ end: string,+ ): { table: 'hourly_country_stats' | 'minute_country_stats'; startKey: string; endKey: string; timeCol: 'hour' | 'minute' } {+ const resolved = this.resolveFactTable(start, end);+ return {+ table: resolved.table === 'hourly_dim_stats' ? 'hourly_country_stats' : 'minute_country_stats',+ startKey: resolved.startKey,+ endKey: resolved.endKey,+ timeCol: resolved.timeCol,+ };+ }+/*** Parse minute range from start and end ISO strings*/diff --git a/apps/collector/src/database/repositories/config.repository.ts b/apps/collector/src/database/repositories/config.repository.tsindex 07e651c..332e6b7 100644--- a/apps/collector/src/database/repositories/config.repository.ts+++ b/apps/collector/src/database/repositories/config.repository.ts@@ -88,6 +88,8 @@ export class ConfigRepository extends BaseRepository {deletedLogs = minuteResult.changes;this.db.prepare(`DELETE FROM minute_dim_stats WHERE backend_id = ?`).run(backendId);this.db.prepare(`DELETE FROM minute_country_stats WHERE backend_id = ?`).run(backendId);+ this.db.prepare(`DELETE FROM hourly_dim_stats WHERE backend_id = ?`).run(backendId);+ this.db.prepare(`DELETE FROM hourly_country_stats WHERE backend_id = ?`).run(backendId);this.db.prepare(`DELETE FROM connection_logs WHERE backend_id = ?`).run(backendId);deletedDomains = this.db.prepare(`DELETE FROM domain_stats WHERE backend_id = ?`).run(backendId).changes;@@ -109,11 +111,14 @@ export class ConfigRepository extends BaseRepository {} else {const cutoff = new Date(Date.now() - days * 24 * 60 * 60 * 1000);const minuteCutoff = cutoff.toISOString().slice(0, 16) + ':00';+ const hourCutoff = cutoff.toISOString().slice(0, 13) + ':00:00';const minuteResult = this.db.prepare(`DELETE FROM minute_stats WHERE backend_id = ? AND minute < ?`).run(backendId, minuteCutoff);deletedConnections = minuteResult.changes;deletedLogs = minuteResult.changes;this.db.prepare(`DELETE FROM minute_dim_stats WHERE backend_id = ? AND minute < ?`).run(backendId, minuteCutoff);this.db.prepare(`DELETE FROM minute_country_stats WHERE backend_id = ? AND minute < ?`).run(backendId, minuteCutoff);+ this.db.prepare(`DELETE FROM hourly_dim_stats WHERE backend_id = ? AND hour < ?`).run(backendId, hourCutoff);+ this.db.prepare(`DELETE FROM hourly_country_stats WHERE backend_id = ? AND hour < ?`).run(backendId, hourCutoff);this.db.prepare(`DELETE FROM connection_logs WHERE backend_id = ? AND timestamp < ?`).run(backendId, cutoff.toISOString());}} else {@@ -123,6 +128,8 @@ export class ConfigRepository extends BaseRepository {deletedLogs = minuteResult.changes;this.db.prepare(`DELETE FROM minute_dim_stats`).run();this.db.prepare(`DELETE FROM minute_country_stats`).run();+ this.db.prepare(`DELETE FROM hourly_dim_stats`).run();+ this.db.prepare(`DELETE FROM hourly_country_stats`).run();this.db.prepare(`DELETE FROM connection_logs`).run();deletedDomains = this.db.prepare(`DELETE FROM domain_stats`).run().changes;@@ -144,11 +151,14 @@ export class ConfigRepository extends BaseRepository {} else {const cutoff = new Date(Date.now() - days * 24 * 60 * 60 * 1000);const minuteCutoff = cutoff.toISOString().slice(0, 16) + ':00';+ const hourCutoff = cutoff.toISOString().slice(0, 13) + ':00:00';const minuteResult = this.db.prepare(`DELETE FROM minute_stats WHERE minute < ?`).run(minuteCutoff);deletedConnections = minuteResult.changes;deletedLogs = minuteResult.changes;this.db.prepare(`DELETE FROM minute_dim_stats WHERE minute < ?`).run(minuteCutoff);this.db.prepare(`DELETE FROM minute_country_stats WHERE minute < ?`).run(minuteCutoff);+ this.db.prepare(`DELETE FROM hourly_dim_stats WHERE hour < ?`).run(hourCutoff);+ this.db.prepare(`DELETE FROM hourly_country_stats WHERE hour < ?`).run(hourCutoff);this.db.prepare(`DELETE FROM connection_logs WHERE timestamp < ?`).run(cutoff.toISOString());}}@@ -190,6 +200,10 @@ export class ConfigRepository extends BaseRepository {this.db.prepare(`DELETE FROM minute_dim_stats WHERE minute < ?`).run(minuteCutoff);this.db.prepare(`DELETE FROM minute_country_stats WHERE minute < ?`).run(minuteCutoff);this.db.prepare(`DELETE FROM connection_logs WHERE timestamp < ?`).run(cutoff);+ // Also clean hourly_dim_stats and hourly_country_stats using the same cutoff (hour granularity)+ const hourCutoff = cutoff.slice(0, 13) + ':00:00';+ this.db.prepare(`DELETE FROM hourly_dim_stats WHERE hour < ?`).run(hourCutoff);+ this.db.prepare(`DELETE FROM hourly_country_stats WHERE hour < ?`).run(hourCutoff);return this.db.prepare(`DELETE FROM minute_stats WHERE minute < ?`).run(minuteCutoff).changes;}diff --git a/apps/collector/src/database/repositories/country.repository.ts b/apps/collector/src/database/repositories/country.repository.tsindex 8eb88e5..8820a3e 100644--- a/apps/collector/src/database/repositories/country.repository.ts+++ b/apps/collector/src/database/repositories/country.repository.ts@@ -23,41 +23,44 @@ export class CountryRepository extends BaseRepository {getCountryStats(backendId: number, limit = 50, start?: string, end?: string): CountryStatsRow[] {const range = this.parseMinuteRange(start, end);if (range) {+ const resolved = this.resolveCountryFactTable(start!, end!);const minuteCountryStmt = this.db.prepare(`SELECT country, MAX(country_name) as countryName, MAX(continent) as continent,SUM(upload) as totalUpload, SUM(download) as totalDownload, SUM(connections) as totalConnections- FROM minute_country_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ?+ FROM ${resolved.table}+ WHERE backend_id = ? AND ${resolved.timeCol} >= ? AND ${resolved.timeCol} <= ?GROUP BY countryORDER BY (SUM(upload) + SUM(download)) DESCLIMIT ?`);- const minuteCountryRows = minuteCountryStmt.all(backendId, range.startMinute, range.endMinute, limit) as CountryStatsRow[];+ const minuteCountryRows = minuteCountryStmt.all(backendId, resolved.startKey, resolved.endKey, limit) as CountryStatsRow[];if (minuteCountryRows.length > 0) return minuteCountryRows;- // Fallback 1: derive from minute_dim_stats + geoip_cache+ // Fallback 1: derive from dim_stats + geoip_cache+ const dimResolved = this.resolveFactTable(start!, end!);const minuteDimFallbackStmt = this.db.prepare(`SELECTCOALESCE(g.country, 'UNKNOWN') as country,COALESCE(MAX(g.country_name), 'Unknown') as countryName,COALESCE(MAX(g.continent), 'Unknown') as continent,SUM(m.upload) as totalUpload, SUM(m.download) as totalDownload, SUM(m.connections) as totalConnections- FROM minute_dim_stats m+ FROM ${dimResolved.table} mLEFT JOIN geoip_cache g ON m.ip = g.ip- WHERE m.backend_id = ? AND m.minute >= ? AND m.minute <= ? AND m.ip != ''+ WHERE m.backend_id = ? AND m.${dimResolved.timeCol} >= ? AND m.${dimResolved.timeCol} <= ? AND m.ip != ''GROUP BY COALESCE(g.country, 'UNKNOWN')ORDER BY (SUM(m.upload) + SUM(m.download)) DESCLIMIT ?`);- const minuteDimFallbackRows = minuteDimFallbackStmt.all(backendId, range.startMinute, range.endMinute, limit) as CountryStatsRow[];+ const minuteDimFallbackRows = minuteDimFallbackStmt.all(backendId, dimResolved.startKey, dimResolved.endKey, limit) as CountryStatsRow[];if (minuteDimFallbackRows.length > 0) return minuteDimFallbackRows;// Fallback 2: aggregate from minute_stats as UNKNOWN+ const minuteRange = this.parseMinuteRange(start, end)!;const totalStmt = this.db.prepare(`SELECT COALESCE(SUM(upload), 0) as upload, COALESCE(SUM(download), 0) as download, COALESCE(SUM(connections), 0) as connectionsFROM minute_stats WHERE backend_id = ? AND minute >= ? AND minute <= ?`);- const total = totalStmt.get(backendId, range.startMinute, range.endMinute) as { upload: number; download: number; connections: number };+ const total = totalStmt.get(backendId, minuteRange.startMinute, minuteRange.endMinute) as { upload: number; download: number; connections: number };if (total.upload > 0 || total.download > 0 || total.connections > 0) {return [{ country: 'UNKNOWN', countryName: 'Unknown', continent: 'Unknown', totalUpload: total.upload, totalDownload: total.download, totalConnections: total.connections }];}@@ -74,7 +77,7 @@ export class CountryRepository extends BaseRepository {return stmt.all(backendId, limit) as CountryStatsRow[];}- updateCountryStats(backendId: number, country: string, countryName: string, continent: string, upload: number, download: number): void {+ updateCountryStats(backendId: number, country: string, countryName: string, continent: string, upload: number, download: number, timestampMs?: number): void {const stmt = this.db.prepare(`INSERT INTO country_stats (backend_id, country, country_name, continent, total_upload, total_download, total_connections, last_seen)VALUES (?, ?, ?, ?, ?, ?, 1, CURRENT_TIMESTAMP)@@ -83,6 +86,24 @@ export class CountryRepository extends BaseRepository {total_connections = total_connections + 1, last_seen = CURRENT_TIMESTAMP`);stmt.run(backendId, country, countryName, continent, upload, download, upload, download);++ const now = new Date(timestampMs ?? Date.now());+ const minute = this.toMinuteKey(now);+ const hour = this.toHourKey(now);++ this.db.prepare(`+ INSERT INTO minute_country_stats (backend_id, minute, country, country_name, continent, upload, download, connections)+ VALUES (?, ?, ?, ?, ?, ?, ?, 1)+ ON CONFLICT(backend_id, minute, country) DO UPDATE SET+ upload = upload + ?, download = download + ?, connections = connections + 1+ `).run(backendId, minute, country, countryName, continent, upload, download, upload, download);++ this.db.prepare(`+ INSERT INTO hourly_country_stats (backend_id, hour, country, country_name, continent, upload, download, connections)+ VALUES (?, ?, ?, ?, ?, ?, ?, 1)+ ON CONFLICT(backend_id, hour, country) DO UPDATE SET+ upload = upload + ?, download = download + ?, connections = connections + 1+ `).run(backendId, hour, country, countryName, continent, upload, download, upload, download);}batchUpdateCountryStats(backendId: number, results: Array<{@@ -106,29 +127,56 @@ export class CountryRepository extends BaseRepository {upload = upload + @upload, download = download + @download, connections = connections + @connections`);+ const hourlyStmt = this.db.prepare(`+ INSERT INTO hourly_country_stats (backend_id, hour, country, country_name, continent, upload, download, connections)+ VALUES (@backendId, @hour, @country, @countryName, @continent, @upload, @download, @connections)+ ON CONFLICT(backend_id, hour, country) DO UPDATE SET+ upload = upload + @upload, download = download + @download, connections = connections + @connections+ `);+const tx = this.db.transaction(() => {const minuteMap = new Map<string, {minute: string; country: string; countryName: string; continent: string;upload: number; download: number; connections: number;}>();+ const hourlyMap = new Map<string, {+ hour: string; country: string; countryName: string; continent: string;+ upload: number; download: number; connections: number;+ }>();for (const r of results) {cumulativeStmt.run(backendId, r.country, r.countryName, r.continent, r.upload, r.download, r.upload, r.download);- const minute = this.toMinuteKey(new Date(r.timestampMs ?? Date.now()));- const key = `${minute}:${r.country}`;- const existing = minuteMap.get(key);+ const now = new Date(r.timestampMs ?? Date.now());+ const minute = this.toMinuteKey(now);+ const hour = this.toHourKey(now);++ const minuteKey = `${minute}:${r.country}`;+ const existing = minuteMap.get(minuteKey);if (existing) {existing.upload += r.upload;existing.download += r.download;existing.connections++;} else {- minuteMap.set(key, { minute, country: r.country, countryName: r.countryName, continent: r.continent, upload: r.upload, download: r.download, connections: 1 });+ minuteMap.set(minuteKey, { minute, country: r.country, countryName: r.countryName, continent: r.continent, upload: r.upload, download: r.download, connections: 1 });+ }++ const hourlyKey = `${hour}:${r.country}`;+ const existingHourly = hourlyMap.get(hourlyKey);+ if (existingHourly) {+ existingHourly.upload += r.upload;+ existingHourly.download += r.download;+ existingHourly.connections++;+ } else {+ hourlyMap.set(hourlyKey, { hour, country: r.country, countryName: r.countryName, continent: r.continent, upload: r.upload, download: r.download, connections: 1 });}}for (const [, item] of minuteMap) {minuteStmt.run({ backendId, ...item });}+ for (const [, item] of hourlyMap) {+ hourlyStmt.run({ backendId, ...item });+ }});tx();}diff --git a/apps/collector/src/database/repositories/device.repository.ts b/apps/collector/src/database/repositories/device.repository.tsindex d0c447e..ac88520 100644--- a/apps/collector/src/database/repositories/device.repository.ts+++ b/apps/collector/src/database/repositories/device.repository.ts@@ -15,16 +15,17 @@ export class DeviceRepository extends BaseRepository {getDevices(backendId: number, limit = 50, start?: string, end?: string): DeviceStats[] {const range = this.parseMinuteRange(start, end);if (range) {+ const resolved = this.resolveFactTable(start!, end!);const stmt = this.db.prepare(`SELECT source_ip as sourceIP, SUM(upload) as totalUpload, SUM(download) as totalDownload,- SUM(connections) as totalConnections, MAX(minute) as lastSeen- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND source_ip != ''+ SUM(connections) as totalConnections, MAX(${resolved.timeCol}) as lastSeen+ FROM ${resolved.table}+ WHERE backend_id = ? AND ${resolved.timeCol} >= ? AND ${resolved.timeCol} <= ? AND source_ip != ''GROUP BY source_ipORDER BY (SUM(upload) + SUM(download)) DESCLIMIT ?`);- return stmt.all(backendId, range.startMinute, range.endMinute, limit) as DeviceStats[];+ return stmt.all(backendId, resolved.startKey, resolved.endKey, limit) as DeviceStats[];}const stmt = this.db.prepare(`@@ -40,15 +41,16 @@ export class DeviceRepository extends BaseRepository {getDeviceDomains(backendId: number, sourceIP: string, limit = 5000, start?: string, end?: string): DomainStats[] {const range = this.parseMinuteRange(start, end);if (range) {+ const resolved = this.resolveFactTable(start!, end!);const stmt = this.db.prepare(`SELECT domain, SUM(upload) as totalUpload, SUM(download) as totalDownload,- SUM(connections) as totalConnections, MAX(minute) as lastSeen,+ SUM(connections) as totalConnections, MAX(${resolved.timeCol}) as lastSeen,GROUP_CONCAT(DISTINCT ip) as ips, GROUP_CONCAT(DISTINCT rule) as rules, GROUP_CONCAT(DISTINCT chain) as chains- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND source_ip = ? AND domain != ''+ FROM ${resolved.table}+ WHERE backend_id = ? AND ${resolved.timeCol} >= ? AND ${resolved.timeCol} <= ? AND source_ip = ? AND domain != ''GROUP BY domain ORDER BY (SUM(upload) + SUM(download)) DESC LIMIT ?`);- const rows = stmt.all(backendId, range.startMinute, range.endMinute, sourceIP, limit) as Array<{+ const rows = stmt.all(backendId, resolved.startKey, resolved.endKey, sourceIP, limit) as Array<{domain: string; totalUpload: number; totalDownload: number; totalConnections: number; lastSeen: string;ips: string | null; rules: string | null; chains: string | null;}>;@@ -74,21 +76,22 @@ export class DeviceRepository extends BaseRepository {getDeviceIPs(backendId: number, sourceIP: string, limit = 5000, start?: string, end?: string): IPStats[] {const range = this.parseMinuteRange(start, end);if (range) {+ const resolved = this.resolveFactTable(start!, end!);const stmt = this.db.prepare(`SELECT m.ip, SUM(m.upload) as totalUpload, SUM(m.download) as totalDownload,- SUM(m.connections) as totalConnections, MAX(m.minute) as lastSeen,+ SUM(m.connections) as totalConnections, MAX(m.${resolved.timeCol}) as lastSeen,GROUP_CONCAT(DISTINCT CASE WHEN m.domain != '' THEN m.domain END) as domains,COALESCE(i.asn, g.asn) as asn,CASE WHEN g.country IS NOT NULL THEN json_array(g.country, COALESCE(g.country_name, g.country), COALESCE(g.city, ''), COALESCE(g.as_name, ''))WHEN i.geoip IS NOT NULL THEN json(i.geoip) ELSE NULL END as geoIP,GROUP_CONCAT(DISTINCT m.chain) as chains, GROUP_CONCAT(DISTINCT m.rule) as rules- FROM minute_dim_stats m+ FROM ${resolved.table} mLEFT JOIN ip_stats i ON m.backend_id = i.backend_id AND m.ip = i.ipLEFT JOIN geoip_cache g ON m.ip = g.ip- WHERE m.backend_id = ? AND m.minute >= ? AND m.minute <= ? AND m.source_ip = ? AND m.ip != ''+ WHERE m.backend_id = ? AND m.${resolved.timeCol} >= ? AND m.${resolved.timeCol} <= ? AND m.source_ip = ? AND m.ip != ''GROUP BY m.ip ORDER BY (SUM(m.upload) + SUM(m.download)) DESC LIMIT ?`);- const rows = stmt.all(backendId, range.startMinute, range.endMinute, sourceIP, limit) as Array<{+ const rows = stmt.all(backendId, resolved.startKey, resolved.endKey, sourceIP, limit) as Array<{ip: string; totalUpload: number; totalDownload: number; totalConnections: number; lastSeen: string;domains: string | null; asn: string | null; geoIP: string | null; chains: string | null; rules: string | null;}>;diff --git a/apps/collector/src/database/repositories/domain.repository.ts b/apps/collector/src/database/repositories/domain.repository.tsindex 1e486aa..e257f38 100644--- a/apps/collector/src/database/repositories/domain.repository.ts+++ b/apps/collector/src/database/repositories/domain.repository.ts@@ -53,26 +53,27 @@ export class DomainRepository extends BaseRepository {getDomainStats(backendId: number, limit = 100, start?: string, end?: string): DomainStats[] {const range = this.parseMinuteRange(start, end);if (range) {+ const resolved = this.resolveFactTable(start!, end!);const stmt = this.db.prepare(`SELECTdomain,SUM(upload) as totalUpload,SUM(download) as totalDownload,SUM(connections) as totalConnections,- MAX(minute) as lastSeen,+ MAX(${resolved.timeCol}) as lastSeen,GROUP_CONCAT(DISTINCT ip) as ips,GROUP_CONCAT(DISTINCT rule) as rules,GROUP_CONCAT(DISTINCT chain) as chains- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND domain != ''+ FROM ${resolved.table}+ WHERE backend_id = ? AND ${resolved.timeCol} >= ? AND ${resolved.timeCol} <= ? AND domain != ''GROUP BY domainORDER BY (SUM(upload) + SUM(download)) DESCLIMIT ?`);const rows = stmt.all(backendId,- range.startMinute,- range.endMinute,+ resolved.startKey,+ resolved.endKey,limit,) as Array<{domain: string;@@ -135,6 +136,48 @@ export class DomainRepository extends BaseRepository {return this.getDomainStats(backendId, limit, start, end);}+ /**+ * Get top domains (light version) - only name + traffic totals, no GROUP_CONCAT+ * Used by overview/summary endpoints where IPs/rules/chains are not needed+ */+ getTopDomainsLight(backendId: number, limit = 10, start?: string, end?: string): DomainStats[] {+ const range = this.parseMinuteRange(start, end);+ if (range) {+ const resolved = this.resolveFactTable(start!, end!);+ const timeCol = resolved.table === 'hourly_dim_stats' ? 'hour' : 'minute';+ const stmt = this.db.prepare(`+ SELECT+ domain,+ SUM(upload) as totalUpload,+ SUM(download) as totalDownload,+ SUM(connections) as totalConnections,+ MAX(${timeCol}) as lastSeen+ FROM ${resolved.table}+ WHERE backend_id = ? AND ${timeCol} >= ? AND ${timeCol} <= ? AND domain != ''+ GROUP BY domain+ ORDER BY (SUM(upload) + SUM(download)) DESC+ LIMIT ?+ `);+ const rows = stmt.all(backendId, resolved.startKey, resolved.endKey, limit) as Array<{+ domain: string; totalUpload: number; totalDownload: number; totalConnections: number; lastSeen: string;+ }>;+ return rows.map(row => ({ ...row, ips: [], rules: [], chains: [] })) as DomainStats[];+ }++ const stmt = this.db.prepare(`+ SELECT domain, total_upload as totalUpload, total_download as totalDownload,+ total_connections as totalConnections, last_seen as lastSeen+ FROM domain_stats+ WHERE backend_id = ?+ ORDER BY (total_upload + total_download) DESC+ LIMIT ?+ `);+ const rows = stmt.all(backendId, limit) as Array<{+ domain: string; totalUpload: number; totalDownload: number; totalConnections: number; lastSeen: string;+ }>;+ return rows.map(row => ({ ...row, ips: [], rules: [], chains: [] })) as DomainStats[];+ }+/*** Get domain stats with server-side pagination, sorting and search*/@@ -164,6 +207,7 @@ export class DomainRepository extends BaseRepository {const sortColumn = sortColumnMap[opts.sortBy || 'totalDownload'] || 'total_download';if (range) {+ const resolved = this.resolveFactTable(opts.start!, opts.end!);const rangeSortColumnMap: Record<string, string> = {domain: 'domain',totalDownload: 'totalDownload',@@ -176,15 +220,15 @@ export class DomainRepository extends BaseRepository {rangeSortColumnMap[opts.sortBy || 'totalDownload'] || 'totalDownload';const whereSearch = search ? 'AND domain LIKE ?' : '';- const baseParams: any[] = [backendId, range.startMinute, range.endMinute];+ const baseParams: any[] = [backendId, resolved.startKey, resolved.endKey];const searchParams: any[] = search ? [`%${search}%`] : [];const countStmt = this.db.prepare(`SELECT COUNT(*) as totalFROM (SELECT domain- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND domain != '' ${whereSearch}+ FROM ${resolved.table}+ WHERE backend_id = ? AND ${resolved.timeCol} >= ? AND ${resolved.timeCol} <= ? AND domain != '' ${whereSearch}GROUP BY domain)`);@@ -199,12 +243,12 @@ export class DomainRepository extends BaseRepository {SUM(upload) as totalUpload,SUM(download) as totalDownload,SUM(connections) as totalConnections,- MAX(minute) as lastSeen,+ MAX(${resolved.timeCol}) as lastSeen,GROUP_CONCAT(DISTINCT ip) as ips,GROUP_CONCAT(DISTINCT rule) as rules,GROUP_CONCAT(DISTINCT chain) as chains- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND domain != '' ${whereSearch}+ FROM ${resolved.table}+ WHERE backend_id = ? AND ${resolved.timeCol} >= ? AND ${resolved.timeCol} <= ? AND domain != '' ${whereSearch}GROUP BY domainORDER BY ${rangeSortColumn} ${sortOrder}LIMIT ? OFFSET ?@@ -298,11 +342,14 @@ export class DomainRepository extends BaseRepository {): IPStats[] {const range = this.parseMinuteRange(start, end);if (range || sourceIP || sourceChain) {+ const resolved = range ? this.resolveFactTable(start!, end!) : null;+ const factTable = resolved?.table ?? 'minute_dim_stats';+ const timeCol = resolved?.timeCol ?? 'minute';const conditions = ["m.backend_id = ?", "m.domain = ?", "m.ip != ''"];const params: Array<string | number> = [backendId, domain];- if (range) {- conditions.push("m.minute >= ?", "m.minute <= ?");- params.push(range.startMinute, range.endMinute);+ if (resolved) {+ conditions.push(`m.${timeCol} >= ?`, `m.${timeCol} <= ?`);+ params.push(resolved.startKey, resolved.endKey);}if (sourceIP) {conditions.push("m.source_ip = ?");@@ -320,7 +367,7 @@ export class DomainRepository extends BaseRepository {SUM(m.upload) as totalUpload,SUM(m.download) as totalDownload,SUM(m.connections) as totalConnections,- MAX(m.minute) as lastSeen,+ MAX(m.${timeCol}) as lastSeen,COALESCE(i.asn, g.asn) as asn,CASEWHEN g.country IS NOT NULL THEN@@ -337,7 +384,7 @@ export class DomainRepository extends BaseRepository {END as geoIP,GROUP_CONCAT(DISTINCT m.chain) as chains,GROUP_CONCAT(DISTINCT m.rule) as rules- FROM minute_dim_stats m+ FROM ${factTable} mLEFT JOIN ip_stats i ON m.backend_id = i.backend_id AND m.ip = i.ipLEFT JOIN geoip_cache g ON m.ip = g.ipWHERE ${conditions.join(" AND ")}@@ -391,11 +438,14 @@ export class DomainRepository extends BaseRepository {): Array<{ chain: string; totalUpload: number; totalDownload: number; totalConnections: number }> {const range = this.parseMinuteRange(start, end);if (range || sourceIP || sourceChain) {+ const resolved = range ? this.resolveFactTable(start!, end!) : null;+ const factTable = resolved?.table ?? 'minute_dim_stats';+ const timeCol = resolved?.timeCol ?? 'minute';const conditions = ["backend_id = ?", "domain = ?"];const params: Array<string | number> = [backendId, domain];- if (range) {- conditions.push("minute >= ?", "minute <= ?");- params.push(range.startMinute, range.endMinute);+ if (resolved) {+ conditions.push(`${timeCol} >= ?`, `${timeCol} <= ?`);+ params.push(resolved.startKey, resolved.endKey);}if (sourceIP) {conditions.push("source_ip = ?");@@ -412,7 +462,7 @@ export class DomainRepository extends BaseRepository {SUM(upload) as totalUpload,SUM(download) as totalDownload,SUM(connections) as totalConnections- FROM minute_dim_stats+ FROM ${factTable}WHERE ${conditions.join(" AND ")}GROUP BY chainORDER BY (SUM(upload) + SUM(download)) DESC@@ -444,24 +494,25 @@ export class DomainRepository extends BaseRepository {): DomainStats[] {const range = this.parseMinuteRange(start, end);if (range) {+ const resolved = this.resolveFactTable(start!, end!);const stmt = this.db.prepare(`SELECTdomain,SUM(upload) as totalUpload,SUM(download) as totalDownload,SUM(connections) as totalConnections,- MAX(minute) as lastSeen,+ MAX(${resolved.timeCol}) as lastSeen,GROUP_CONCAT(DISTINCT ip) as ips- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND (chain = ? OR chain LIKE ?) AND domain != ''+ FROM ${resolved.table}+ WHERE backend_id = ? AND ${resolved.timeCol} >= ? AND ${resolved.timeCol} <= ? AND (chain = ? OR chain LIKE ?) AND domain != ''GROUP BY domainORDER BY (SUM(upload) + SUM(download)) DESCLIMIT ?`);const rows = stmt.all(backendId,- range.startMinute,- range.endMinute,+ resolved.startKey,+ resolved.endKey,chain,`${chain} > %`,limit,@@ -525,25 +576,26 @@ export class DomainRepository extends BaseRepository {): DomainStats[] {const range = this.parseMinuteRange(start, end);if (range) {+ const resolved = this.resolveFactTable(start!, end!);const stmt = this.db.prepare(`SELECTdomain,SUM(upload) as totalUpload,SUM(download) as totalDownload,SUM(connections) as totalConnections,- MAX(minute) as lastSeen,+ MAX(${resolved.timeCol}) as lastSeen,GROUP_CONCAT(DISTINCT ip) as ips,GROUP_CONCAT(DISTINCT chain) as chains- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND rule = ? AND domain != ''+ FROM ${resolved.table}+ WHERE backend_id = ? AND ${resolved.timeCol} >= ? AND ${resolved.timeCol} <= ? AND rule = ? AND domain != ''GROUP BY domainORDER BY (SUM(upload) + SUM(download)) DESCLIMIT ?`);const rows = stmt.all(backendId,- range.startMinute,- range.endMinute,+ resolved.startKey,+ resolved.endKey,rule,limit,) as Array<{diff --git a/apps/collector/src/database/repositories/ip.repository.ts b/apps/collector/src/database/repositories/ip.repository.tsindex 482d0e5..b55f0f0 100644--- a/apps/collector/src/database/repositories/ip.repository.ts+++ b/apps/collector/src/database/repositories/ip.repository.ts@@ -34,21 +34,22 @@ export class IPRepository extends BaseRepository {getIPStats(backendId: number, limit = 100, start?: string, end?: string): IPStats[] {const range = this.parseMinuteRange(start, end);if (range) {+ const resolved = this.resolveFactTable(start!, end!);const stmt = this.db.prepare(`SELECT m.ip, GROUP_CONCAT(DISTINCT CASE WHEN m.domain != '' THEN m.domain END) as domains,SUM(m.upload) as totalUpload, SUM(m.download) as totalDownload,- SUM(m.connections) as totalConnections, MAX(m.minute) as lastSeen,+ SUM(m.connections) as totalConnections, MAX(m.${resolved.timeCol}) as lastSeen,COALESCE(i.asn, g.asn) as asn,CASE WHEN g.country IS NOT NULL THEN json_array(g.country, COALESCE(g.country_name, g.country), COALESCE(g.city, ''), COALESCE(g.as_name, ''))WHEN i.geoip IS NOT NULL THEN json(i.geoip) ELSE NULL END as geoIP,GROUP_CONCAT(DISTINCT m.chain) as chains, GROUP_CONCAT(DISTINCT m.rule) as rules- FROM minute_dim_stats m+ FROM ${resolved.table} mLEFT JOIN ip_stats i ON m.backend_id = i.backend_id AND m.ip = i.ipLEFT JOIN geoip_cache g ON m.ip = g.ip- WHERE m.backend_id = ? AND m.minute >= ? AND m.minute <= ? AND m.ip != ''+ WHERE m.backend_id = ? AND m.${resolved.timeCol} >= ? AND m.${resolved.timeCol} <= ? AND m.ip != ''GROUP BY m.ip ORDER BY (SUM(m.upload) + SUM(m.download)) DESC LIMIT ?`);- const rows = stmt.all(backendId, range.startMinute, range.endMinute, limit) as Array<{+ const rows = stmt.all(backendId, resolved.startKey, resolved.endKey, limit) as Array<{ip: string; domains: string; totalUpload: number; totalDownload: number; totalConnections: number;lastSeen: string; asn: string | null; geoIP: string | null; chains: string | null; rules: string | null;}>;@@ -88,6 +89,48 @@ export class IPRepository extends BaseRepository {})) as IPStats[];}+ /**+ * Get top IPs (light version) - only ip + traffic totals, no GROUP_CONCAT or JOINs+ * Used by overview/summary endpoints where domains/geo/chains are not needed+ */+ getTopIPsLight(backendId: number, limit = 10, start?: string, end?: string): IPStats[] {+ const range = this.parseMinuteRange(start, end);+ if (range) {+ const resolved = this.resolveFactTable(start!, end!);+ const timeCol = resolved.table === 'hourly_dim_stats' ? 'hour' : 'minute';+ const stmt = this.db.prepare(`+ SELECT+ ip,+ SUM(upload) as totalUpload,+ SUM(download) as totalDownload,+ SUM(connections) as totalConnections,+ MAX(${timeCol}) as lastSeen+ FROM ${resolved.table}+ WHERE backend_id = ? AND ${timeCol} >= ? AND ${timeCol} <= ? AND ip != ''+ GROUP BY ip+ ORDER BY (SUM(upload) + SUM(download)) DESC+ LIMIT ?+ `);+ const rows = stmt.all(backendId, resolved.startKey, resolved.endKey, limit) as Array<{+ ip: string; totalUpload: number; totalDownload: number; totalConnections: number; lastSeen: string;+ }>;+ return rows.map(row => ({ ...row, domains: [], chains: [] })) as IPStats[];+ }++ const stmt = this.db.prepare(`+ SELECT ip, total_upload as totalUpload, total_download as totalDownload,+ total_connections as totalConnections, last_seen as lastSeen+ FROM ip_stats+ WHERE backend_id = ? AND ip != ''+ ORDER BY (total_upload + total_download) DESC+ LIMIT ?+ `);+ const rows = stmt.all(backendId, limit) as Array<{+ ip: string; totalUpload: number; totalDownload: number; totalConnections: number; lastSeen: string;+ }>;+ return rows.map(row => ({ ...row, domains: [], chains: [] })) as IPStats[];+ }+getIPStatsByIPs(backendId: number, ips: string[]): IPStats[] {const filteredIps = ips.filter(ip => ip && ip.trim() !== '');if (filteredIps.length === 0) return [];@@ -134,6 +177,7 @@ export class IPRepository extends BaseRepository {const sortColumn = sortColumnMap[opts.sortBy || 'totalDownload'] || 'i.total_download';if (range) {+ const resolved = this.resolveFactTable(opts.start!, opts.end!);const rangeSortColumnMap: Record<string, string> = {ip: 'agg.ip', totalDownload: 'agg.totalDownload', totalUpload: 'agg.totalUpload',totalTraffic: '(agg.totalUpload + agg.totalDownload)',@@ -142,13 +186,13 @@ export class IPRepository extends BaseRepository {const rangeSortColumn = rangeSortColumnMap[opts.sortBy || 'totalDownload'] || 'agg.totalDownload';const whereSearch = search ? "AND (ip LIKE ? OR domain LIKE ?)" : "";- const baseParams: any[] = [backendId, range.startMinute, range.endMinute];+ const baseParams: any[] = [backendId, resolved.startKey, resolved.endKey];const searchParams: any[] = search ? [`%${search}%`, `%${search}%`] : [];const countStmt = this.db.prepare(`SELECT COUNT(*) as total FROM (- SELECT ip FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND ip != '' ${whereSearch}+ SELECT ip FROM ${resolved.table}+ WHERE backend_id = ? AND ${resolved.timeCol} >= ? AND ${resolved.timeCol} <= ? AND ip != '' ${whereSearch}GROUP BY ip)`);@@ -159,10 +203,10 @@ export class IPRepository extends BaseRepository {SELECT ip, GROUP_CONCAT(DISTINCT CASE WHEN domain != '' THEN domain END) as domains,GROUP_CONCAT(DISTINCT rule) as rules,SUM(upload) as totalUpload, SUM(download) as totalDownload,- SUM(connections) as totalConnections, MAX(minute) as lastSeen,+ SUM(connections) as totalConnections, MAX(${resolved.timeCol}) as lastSeen,GROUP_CONCAT(DISTINCT chain) as chains- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND ip != '' ${whereSearch}+ FROM ${resolved.table}+ WHERE backend_id = ? AND ${resolved.timeCol} >= ? AND ${resolved.timeCol} <= ? AND ip != '' ${whereSearch}GROUP BY ip)SELECT agg.ip, agg.domains, agg.rules, agg.totalUpload, agg.totalDownload,@@ -243,11 +287,14 @@ export class IPRepository extends BaseRepository {): IPStats[] {const range = this.parseMinuteRange(start, end);if (range || sourceIP || sourceChain) {+ const resolved = range ? this.resolveFactTable(start!, end!) : null;+ const factTable = resolved?.table ?? 'minute_dim_stats';+ const timeCol = resolved?.timeCol ?? 'minute';const conditions = ["m.backend_id = ?", "m.domain = ?", "m.ip != ''"];const params: Array<string | number> = [backendId, domain];- if (range) {- conditions.push("m.minute >= ?", "m.minute <= ?");- params.push(range.startMinute, range.endMinute);+ if (resolved) {+ conditions.push(`m.${timeCol} >= ?`, `m.${timeCol} <= ?`);+ params.push(resolved.startKey, resolved.endKey);}if (sourceIP) {conditions.push("m.source_ip = ?");@@ -261,12 +308,12 @@ export class IPRepository extends BaseRepository {const stmt = this.db.prepare(`SELECT m.ip, GROUP_CONCAT(DISTINCT CASE WHEN m.domain != '' THEN m.domain END) as domains,SUM(m.upload) as totalUpload, SUM(m.download) as totalDownload,- SUM(m.connections) as totalConnections, MAX(m.minute) as lastSeen,+ SUM(m.connections) as totalConnections, MAX(m.${timeCol}) as lastSeen,COALESCE(i.asn, g.asn) as asn,CASE WHEN g.country IS NOT NULL THEN json_array(g.country, COALESCE(g.country_name, g.country), COALESCE(g.city, ''), COALESCE(g.as_name, ''))WHEN i.geoip IS NOT NULL THEN json(i.geoip) ELSE NULL END as geoIP,GROUP_CONCAT(DISTINCT m.chain) as chains, GROUP_CONCAT(DISTINCT m.rule) as rules- FROM minute_dim_stats m+ FROM ${factTable} mLEFT JOIN ip_stats i ON m.backend_id = i.backend_id AND m.ip = i.ipLEFT JOIN geoip_cache g ON m.ip = g.ipWHERE ${conditions.join(" AND ")}@@ -307,11 +354,14 @@ export class IPRepository extends BaseRepository {limit = 100, sourceIP?: string, sourceChain?: string,): DomainStats[] {const range = this.parseMinuteRange(start, end);+ const resolved = range ? this.resolveFactTable(start!, end!) : null;+ const factTable = resolved?.table ?? 'minute_dim_stats';+ const timeCol = resolved?.timeCol ?? 'minute';const conditions = ["backend_id = ?", "ip = ?", "domain != ''"];const params: Array<string | number> = [backendId, ip];- if (range) {- conditions.push("minute >= ?", "minute <= ?");- params.push(range.startMinute, range.endMinute);+ if (resolved) {+ conditions.push(`${timeCol} >= ?`, `${timeCol} <= ?`);+ params.push(resolved.startKey, resolved.endKey);}if (sourceIP) {conditions.push("source_ip = ?");@@ -325,10 +375,10 @@ export class IPRepository extends BaseRepository {const stmt = this.db.prepare(`SELECT domain, GROUP_CONCAT(DISTINCT ip) as ips,SUM(upload) as totalUpload, SUM(download) as totalDownload,- SUM(connections) as totalConnections, MAX(minute) as lastSeen,+ SUM(connections) as totalConnections, MAX(${timeCol}) as lastSeen,GROUP_CONCAT(DISTINCT CASE WHEN rule != '' THEN rule END) as rules,GROUP_CONCAT(DISTINCT chain) as chains- FROM minute_dim_stats WHERE ${conditions.join(" AND ")}+ FROM ${factTable} WHERE ${conditions.join(" AND ")}GROUP BY domain ORDER BY (SUM(upload) + SUM(download)) DESC LIMIT ?`);@@ -355,11 +405,14 @@ export class IPRepository extends BaseRepository {): ProxyTrafficStats[] {const range = this.parseMinuteRange(start, end);if (range || sourceIP || sourceChain) {+ const resolved = range ? this.resolveFactTable(start!, end!) : null;+ const factTable = resolved?.table ?? 'minute_dim_stats';+ const timeCol = resolved?.timeCol ?? 'minute';const conditions = ["backend_id = ?", "ip = ?"];const params: Array<string | number> = [backendId, ip];- if (range) {- conditions.push("minute >= ?", "minute <= ?");- params.push(range.startMinute, range.endMinute);+ if (resolved) {+ conditions.push(`${timeCol} >= ?`, `${timeCol} <= ?`);+ params.push(resolved.startKey, resolved.endKey);}if (sourceIP) {conditions.push("source_ip = ?");@@ -372,7 +425,7 @@ export class IPRepository extends BaseRepository {const stmt = this.db.prepare(`SELECT chain, SUM(upload) as totalUpload, SUM(download) as totalDownload, SUM(connections) as totalConnections- FROM minute_dim_stats WHERE ${conditions.join(" AND ")}+ FROM ${factTable} WHERE ${conditions.join(" AND ")}GROUP BY chain ORDER BY (SUM(upload) + SUM(download)) DESC`);return stmt.all(...params) as ProxyTrafficStats[];diff --git a/apps/collector/src/database/repositories/proxy.repository.ts b/apps/collector/src/database/repositories/proxy.repository.tsindex 758f719..c55b6f0 100644--- a/apps/collector/src/database/repositories/proxy.repository.ts+++ b/apps/collector/src/database/repositories/proxy.repository.ts@@ -15,13 +15,14 @@ export class ProxyRepository extends BaseRepository {getProxyStats(backendId: number, start?: string, end?: string): ProxyStats[] {const range = this.parseMinuteRange(start, end);if (range) {+ const resolved = this.resolveFactTable(start!, end!);const stmt = this.db.prepare(`SELECT chain, SUM(upload) as totalUpload, SUM(download) as totalDownload,- SUM(connections) as totalConnections, MAX(minute) as lastSeen- FROM minute_dim_stats WHERE backend_id = ? AND minute >= ? AND minute <= ?+ SUM(connections) as totalConnections, MAX(${resolved.timeCol}) as lastSeen+ FROM ${resolved.table} WHERE backend_id = ? AND ${resolved.timeCol} >= ? AND ${resolved.timeCol} <= ?GROUP BY chain ORDER BY (SUM(upload) + SUM(download)) DESC`);- return this.aggregateProxyStatsByFirstHop(stmt.all(backendId, range.startMinute, range.endMinute) as ProxyStats[]);+ return this.aggregateProxyStatsByFirstHop(stmt.all(backendId, resolved.startKey, resolved.endKey) as ProxyStats[]);}const stmt = this.db.prepare(`@@ -35,14 +36,15 @@ export class ProxyRepository extends BaseRepository {getProxyDomains(backendId: number, chain: string, limit = 50, start?: string, end?: string): DomainStats[] {const range = this.parseMinuteRange(start, end);if (range) {+ const resolved = this.resolveFactTable(start!, end!);const stmt = this.db.prepare(`SELECT domain, SUM(upload) as totalUpload, SUM(download) as totalDownload,- SUM(connections) as totalConnections, MAX(minute) as lastSeen, GROUP_CONCAT(DISTINCT ip) as ips- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND (chain = ? OR chain LIKE ?) AND domain != ''+ SUM(connections) as totalConnections, MAX(${resolved.timeCol}) as lastSeen, GROUP_CONCAT(DISTINCT ip) as ips+ FROM ${resolved.table}+ WHERE backend_id = ? AND ${resolved.timeCol} >= ? AND ${resolved.timeCol} <= ? AND (chain = ? OR chain LIKE ?) AND domain != ''GROUP BY domain ORDER BY (SUM(upload) + SUM(download)) DESC LIMIT ?`);- const rows = stmt.all(backendId, range.startMinute, range.endMinute, chain, `${chain} > %`, limit) as Array<{+ const rows = stmt.all(backendId, resolved.startKey, resolved.endKey, chain, `${chain} > %`, limit) as Array<{domain: string; totalUpload: number; totalDownload: number; totalConnections: number; lastSeen: string; ips: string | null;}>;return rows.map(row => ({ ...row, ips: row.ips ? row.ips.split(',').filter(Boolean) : [], rules: [], chains: [chain] })) as DomainStats[];@@ -65,20 +67,21 @@ export class ProxyRepository extends BaseRepository {getProxyIPs(backendId: number, chain: string, limit = 50, start?: string, end?: string): IPStats[] {const range = this.parseMinuteRange(start, end);if (range) {+ const resolved = this.resolveFactTable(start!, end!);const stmt = this.db.prepare(`SELECT m.ip, SUM(m.upload) as totalUpload, SUM(m.download) as totalDownload,- SUM(m.connections) as totalConnections, MAX(m.minute) as lastSeen,+ SUM(m.connections) as totalConnections, MAX(m.${resolved.timeCol}) as lastSeen,GROUP_CONCAT(DISTINCT CASE WHEN m.domain != '' THEN m.domain END) as domains,COALESCE(i.asn, g.asn) as asn,CASE WHEN g.country IS NOT NULL THEN json_array(g.country, COALESCE(g.country_name, g.country), COALESCE(g.city, ''), COALESCE(g.as_name, ''))WHEN i.geoip IS NOT NULL THEN json(i.geoip) ELSE NULL END as geoIP- FROM minute_dim_stats m+ FROM ${resolved.table} mLEFT JOIN ip_stats i ON m.backend_id = i.backend_id AND m.ip = i.ipLEFT JOIN geoip_cache g ON m.ip = g.ip- WHERE m.backend_id = ? AND m.minute >= ? AND m.minute <= ? AND (m.chain = ? OR m.chain LIKE ?) AND m.ip != ''+ WHERE m.backend_id = ? AND m.${resolved.timeCol} >= ? AND m.${resolved.timeCol} <= ? AND (m.chain = ? OR m.chain LIKE ?) AND m.ip != ''GROUP BY m.ip ORDER BY (SUM(m.upload) + SUM(m.download)) DESC LIMIT ?`);- const rows = stmt.all(backendId, range.startMinute, range.endMinute, chain, `${chain} > %`, limit) as Array<{+ const rows = stmt.all(backendId, resolved.startKey, resolved.endKey, chain, `${chain} > %`, limit) as Array<{ip: string; totalUpload: number; totalDownload: number; totalConnections: number; lastSeen: string;domains: string | null; asn: string | null; geoIP: string | null;}>;diff --git a/apps/collector/src/database/repositories/rule.repository.ts b/apps/collector/src/database/repositories/rule.repository.tsindex 9efefce..1c22f00 100644--- a/apps/collector/src/database/repositories/rule.repository.ts+++ b/apps/collector/src/database/repositories/rule.repository.ts@@ -23,15 +23,16 @@ export class RuleRepository extends BaseRepository {getRuleStats(backendId: number, start?: string, end?: string): RuleStats[] {const range = this.parseMinuteRange(start, end);if (range) {+ const resolved = this.resolveFactTable(start!, end!);const stmt = this.db.prepare(`SELECT rule, MAX(chain) as finalProxy,SUM(upload) as totalUpload, SUM(download) as totalDownload,- SUM(connections) as totalConnections, MAX(minute) as lastSeen- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ?+ SUM(connections) as totalConnections, MAX(${resolved.timeCol}) as lastSeen+ FROM ${resolved.table}+ WHERE backend_id = ? AND ${resolved.timeCol} >= ? AND ${resolved.timeCol} <= ?GROUP BY rule ORDER BY (SUM(upload) + SUM(download)) DESC`);- return stmt.all(backendId, range.startMinute, range.endMinute) as RuleStats[];+ return stmt.all(backendId, resolved.startKey, resolved.endKey) as RuleStats[];}const stmt = this.db.prepare(`@@ -59,15 +60,16 @@ export class RuleRepository extends BaseRepository {getRuleDomains(backendId: number, rule: string, limit = 50, start?: string, end?: string): DomainStats[] {const range = this.parseMinuteRange(start, end);if (range) {+ const resolved = this.resolveFactTable(start!, end!);const stmt = this.db.prepare(`SELECT domain, SUM(upload) as totalUpload, SUM(download) as totalDownload,- SUM(connections) as totalConnections, MAX(minute) as lastSeen,+ SUM(connections) as totalConnections, MAX(${resolved.timeCol}) as lastSeen,GROUP_CONCAT(DISTINCT ip) as ips, GROUP_CONCAT(DISTINCT chain) as chains- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND rule = ? AND domain != ''+ FROM ${resolved.table}+ WHERE backend_id = ? AND ${resolved.timeCol} >= ? AND ${resolved.timeCol} <= ? AND rule = ? AND domain != ''GROUP BY domain ORDER BY (SUM(upload) + SUM(download)) DESC LIMIT ?`);- const rows = stmt.all(backendId, range.startMinute, range.endMinute, rule, limit) as Array<{+ const rows = stmt.all(backendId, resolved.startKey, resolved.endKey, rule, limit) as Array<{domain: string; totalUpload: number; totalDownload: number; totalConnections: number;lastSeen: string; ips: string | null; chains: string | null;}>;@@ -108,21 +110,22 @@ export class RuleRepository extends BaseRepository {getRuleIPs(backendId: number, rule: string, limit = 50, start?: string, end?: string): IPStats[] {const range = this.parseMinuteRange(start, end);if (range) {+ const resolved = this.resolveFactTable(start!, end!);const stmt = this.db.prepare(`SELECT m.ip, SUM(m.upload) as totalUpload, SUM(m.download) as totalDownload,- SUM(m.connections) as totalConnections, MAX(m.minute) as lastSeen,+ SUM(m.connections) as totalConnections, MAX(m.${resolved.timeCol}) as lastSeen,GROUP_CONCAT(DISTINCT CASE WHEN m.domain != '' THEN m.domain END) as domains,GROUP_CONCAT(DISTINCT m.chain) as chains,COALESCE(i.asn, g.asn) as asn,CASE WHEN g.country IS NOT NULL THEN json_array(g.country, COALESCE(g.country_name, g.country), COALESCE(g.city, ''), COALESCE(g.as_name, ''))WHEN i.geoip IS NOT NULL THEN json(i.geoip) ELSE NULL END as geoIPData- FROM minute_dim_stats m+ FROM ${resolved.table} mLEFT JOIN ip_stats i ON m.backend_id = i.backend_id AND m.ip = i.ipLEFT JOIN geoip_cache g ON m.ip = g.ip- WHERE m.backend_id = ? AND m.minute >= ? AND m.minute <= ? AND m.rule = ? AND m.ip != ''+ WHERE m.backend_id = ? AND m.${resolved.timeCol} >= ? AND m.${resolved.timeCol} <= ? AND m.rule = ? AND m.ip != ''GROUP BY m.ip ORDER BY (SUM(m.upload) + SUM(m.download)) DESC LIMIT ?`);- const rows = stmt.all(backendId, range.startMinute, range.endMinute, rule, limit) as Array<{+ const rows = stmt.all(backendId, resolved.startKey, resolved.endKey, rule, limit) as Array<{ip: string; totalUpload: number; totalDownload: number; totalConnections: number;lastSeen: string; domains: string | null; chains: string | null; asn: string | null; geoIPData: string | null;}>;@@ -170,35 +173,39 @@ export class RuleRepository extends BaseRepository {getRuleDomainProxyStats(backendId: number, rule: string, domain: string, start?: string, end?: string): ProxyTrafficStats[] {const range = this.parseMinuteRange(start, end);if (range) {+ const resolved = this.resolveFactTable(start!, end!);const stmt = this.db.prepare(`SELECT chain, SUM(upload) as totalUpload, SUM(download) as totalDownload, SUM(connections) as totalConnections- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND rule = ? AND domain = ?+ FROM ${resolved.table}+ WHERE backend_id = ? AND ${resolved.timeCol} >= ? AND ${resolved.timeCol} <= ? AND rule = ? AND domain = ?GROUP BY chain ORDER BY (SUM(upload) + SUM(download)) DESC`);- return stmt.all(backendId, range.startMinute, range.endMinute, rule, domain) as ProxyTrafficStats[];+ return stmt.all(backendId, resolved.startKey, resolved.endKey, rule, domain) as ProxyTrafficStats[];}return [];}getRuleDomainIPDetails(backendId: number, rule: string, domain: string, start?: string, end?: string, limit = 100): IPStats[] {const range = this.parseMinuteRange(start, end);+ const resolved = range ? this.resolveFactTable(start!, end!) : null;+ const factTable = resolved?.table ?? 'minute_dim_stats';+ const timeCol = resolved?.timeCol ?? 'minute';const conditions = ["m.backend_id = ?", "m.rule = ?", "m.domain = ?", "m.ip != ''"];const params: Array<string | number> = [backendId, rule, domain];- if (range) {- conditions.push("m.minute >= ?", "m.minute <= ?");- params.push(range.startMinute, range.endMinute);+ if (resolved) {+ conditions.push(`m.${timeCol} >= ?`, `m.${timeCol} <= ?`);+ params.push(resolved.startKey, resolved.endKey);}const stmt = this.db.prepare(`SELECT m.ip, GROUP_CONCAT(DISTINCT CASE WHEN m.domain != '' THEN m.domain END) as domains,SUM(m.upload) as totalUpload, SUM(m.download) as totalDownload,- SUM(m.connections) as totalConnections, MAX(m.minute) as lastSeen,+ SUM(m.connections) as totalConnections, MAX(m.${timeCol}) as lastSeen,COALESCE(i.asn, g.asn) as asn,CASE WHEN g.country IS NOT NULL THEN json_array(g.country, COALESCE(g.country_name, g.country), COALESCE(g.city, ''), COALESCE(g.as_name, ''))WHEN i.geoip IS NOT NULL THEN json(i.geoip) ELSE NULL END as geoIP,GROUP_CONCAT(DISTINCT m.chain) as chains- FROM minute_dim_stats m+ FROM ${factTable} mLEFT JOIN ip_stats i ON m.backend_id = i.backend_id AND m.ip = i.ipLEFT JOIN geoip_cache g ON m.ip = g.ipWHERE ${conditions.join(" AND ")}@@ -224,33 +231,37 @@ export class RuleRepository extends BaseRepository {getRuleIPProxyStats(backendId: number, rule: string, ip: string, start?: string, end?: string): ProxyTrafficStats[] {const range = this.parseMinuteRange(start, end);if (range) {+ const resolved = this.resolveFactTable(start!, end!);const stmt = this.db.prepare(`SELECT chain, SUM(upload) as totalUpload, SUM(download) as totalDownload, SUM(connections) as totalConnections- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND rule = ? AND ip = ?+ FROM ${resolved.table}+ WHERE backend_id = ? AND ${resolved.timeCol} >= ? AND ${resolved.timeCol} <= ? AND rule = ? AND ip = ?GROUP BY chain ORDER BY (SUM(upload) + SUM(download)) DESC`);- return stmt.all(backendId, range.startMinute, range.endMinute, rule, ip) as ProxyTrafficStats[];+ return stmt.all(backendId, resolved.startKey, resolved.endKey, rule, ip) as ProxyTrafficStats[];}return [];}getRuleIPDomainDetails(backendId: number, rule: string, ip: string, start?: string, end?: string, limit = 100): DomainStats[] {const range = this.parseMinuteRange(start, end);+ const resolved = range ? this.resolveFactTable(start!, end!) : null;+ const factTable = resolved?.table ?? 'minute_dim_stats';+ const timeCol = resolved?.timeCol ?? 'minute';const conditions = ["backend_id = ?", "rule = ?", "ip = ?", "domain != ''"];const params: Array<string | number> = [backendId, rule, ip];- if (range) {- conditions.push("minute >= ?", "minute <= ?");- params.push(range.startMinute, range.endMinute);+ if (resolved) {+ conditions.push(`${timeCol} >= ?`, `${timeCol} <= ?`);+ params.push(resolved.startKey, resolved.endKey);}const stmt = this.db.prepare(`SELECT domain, GROUP_CONCAT(DISTINCT ip) as ips,SUM(upload) as totalUpload, SUM(download) as totalDownload,- SUM(connections) as totalConnections, MAX(minute) as lastSeen,+ SUM(connections) as totalConnections, MAX(${timeCol}) as lastSeen,GROUP_CONCAT(DISTINCT CASE WHEN rule != '' THEN rule END) as rules,GROUP_CONCAT(DISTINCT chain) as chains- FROM minute_dim_stats WHERE ${conditions.join(" AND ")}+ FROM ${factTable} WHERE ${conditions.join(" AND ")}GROUP BY domain ORDER BY (SUM(upload) + SUM(download)) DESC LIMIT ?`);@@ -278,13 +289,14 @@ export class RuleRepository extends BaseRepository {let rows: Array<{ chain: string; totalUpload: number; totalDownload: number; totalConnections: number }>;if (range) {+ const resolved = this.resolveFactTable(start!, end!);const stmt = this.db.prepare(`SELECT chain, SUM(upload) as totalUpload, SUM(download) as totalDownload, SUM(connections) as totalConnections- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND rule = ? AND chain != ''+ FROM ${resolved.table}+ WHERE backend_id = ? AND ${resolved.timeCol} >= ? AND ${resolved.timeCol} <= ? AND rule = ? AND chain != ''GROUP BY chain`);- rows = stmt.all(backendId, range.startMinute, range.endMinute, rule) as typeof rows;+ rows = stmt.all(backendId, resolved.startKey, resolved.endKey, rule) as typeof rows;const baselineStmt = this.db.prepare(`SELECT rule, chain, total_upload as totalUpload, total_download as totalDownload, total_connections as totalConnections@@ -349,13 +361,14 @@ export class RuleRepository extends BaseRepository {let rows: Array<{ rule: string; chain: string; totalUpload: number; totalDownload: number; totalConnections: number }>;if (range) {+ const resolved = this.resolveFactTable(start!, end!);const stmt = this.db.prepare(`SELECT rule, chain, SUM(upload) as totalUpload, SUM(download) as totalDownload, SUM(connections) as totalConnections- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND rule != '' AND chain != ''+ FROM ${resolved.table}+ WHERE backend_id = ? AND ${resolved.timeCol} >= ? AND ${resolved.timeCol} <= ? AND rule != '' AND chain != ''GROUP BY rule, chain ORDER BY rule, chain`);- rows = stmt.all(backendId, range.startMinute, range.endMinute) as typeof rows;+ rows = stmt.all(backendId, resolved.startKey, resolved.endKey) as typeof rows;const baselineStmt = this.db.prepare(`SELECT rule, chain, total_upload as totalUpload, total_download as totalDownload, total_connections as totalConnectionsdiff --git a/apps/collector/src/database/repositories/timeseries.repository.ts b/apps/collector/src/database/repositories/timeseries.repository.tsindex 3deb545..2597248 100644--- a/apps/collector/src/database/repositories/timeseries.repository.ts+++ b/apps/collector/src/database/repositories/timeseries.repository.ts@@ -16,19 +16,17 @@ export class TimeseriesRepository extends BaseRepository {getHourlyStats(backendId: number, hours = 24, start?: string, end?: string): HourlyStats[] {const range = this.parseMinuteRange(start, end);if (range) {+ // hourly_stats is maintained in real-time — query it directly+ const startHour = this.toHourKey(new Date(start!));+ const endHour = this.toHourKey(new Date(end!));const stmt = this.db.prepare(`- SELECT- substr(minute, 1, 13) || ':00:00' as hour,- SUM(upload) as upload,- SUM(download) as download,- SUM(connections) as connections- FROM minute_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ?- GROUP BY substr(minute, 1, 13)+ SELECT hour, upload, download, connections+ FROM hourly_stats+ WHERE backend_id = ? AND hour >= ? AND hour <= ?ORDER BY hour DESCLIMIT ?`);- return stmt.all(backendId, range.startMinute, range.endMinute, hours) as HourlyStats[];+ return stmt.all(backendId, startHour, endHour, hours) as HourlyStats[];}const stmt = this.db.prepare(`@@ -57,6 +55,21 @@ export class TimeseriesRepository extends BaseRepository {return this.getTodayTraffic(backendId);}+ const startDate = new Date(start!);+ const endDate = new Date(end!);+ const SIX_HOURS_MS = 6 * 60 * 60 * 1000;++ if (endDate.getTime() - startDate.getTime() > SIX_HOURS_MS) {+ const stmt = this.db.prepare(`+ SELECT+ COALESCE(SUM(upload), 0) as upload,+ COALESCE(SUM(download), 0) as download+ FROM hourly_stats+ WHERE backend_id = ? AND hour >= ? AND hour <= ?+ `);+ return stmt.get(backendId, this.toHourKey(startDate), this.toHourKey(endDate)) as { upload: number; download: number };+ }+const stmt = this.db.prepare(`SELECTCOALESCE(SUM(upload), 0) as upload,@@ -76,17 +89,31 @@ export class TimeseriesRepository extends BaseRepository {const range = this.parseMinuteRange(start, end);const cutoff = new Date(Date.now() - minutes * 60 * 1000);const cutoffStr = cutoff.toISOString().slice(0, 16) + ':00';+ const startStr = range?.startMinute || cutoffStr;+ const endStr = range?.endMinute || this.toMinuteKey(new Date());++ const SIX_HOURS_MS = 6 * 60 * 60 * 1000;+ if (new Date(endStr).getTime() - new Date(startStr).getTime() > SIX_HOURS_MS) {+ const stmt = this.db.prepare(`+ SELECT hour as time, upload, download+ FROM hourly_stats+ WHERE backend_id = ? AND hour >= ? AND hour <= ?+ ORDER BY hour ASC+ `);+ return stmt.all(+ backendId,+ this.toHourKey(new Date(startStr)),+ this.toHourKey(new Date(endStr)),+ ) as Array<{ time: string; upload: number; download: number }>;+ }+const stmt = this.db.prepare(`SELECT minute as time, upload, downloadFROM minute_statsWHERE backend_id = ? AND minute >= ? AND minute <= ?ORDER BY minute ASC`);- return stmt.all(- backendId,- range?.startMinute || cutoffStr,- range?.endMinute || this.toMinuteKey(new Date()),- ) as Array<{ time: string; upload: number; download: number }>;+ return stmt.all(backendId, startStr, endStr) as Array<{ time: string; upload: number; download: number }>;}getTrafficTrendAggregated(@@ -99,8 +126,57 @@ export class TimeseriesRepository extends BaseRepository {const range = this.parseMinuteRange(start, end);const cutoff = new Date(Date.now() - minutes * 60 * 1000);const cutoffStr = cutoff.toISOString().slice(0, 16) + ':00';+ const startMinute = range?.startMinute || cutoffStr;const endMinute = range?.endMinute || this.toMinuteKey(new Date());+ // For bucketMinutes >= 60, use hourly_stats (60x fewer rows for 7-day queries)+ if (bucketMinutes >= 60) {+ const startHour = this.toHourKey(new Date(startMinute));+ const endHour = this.toHourKey(new Date(endMinute));++ if (bucketMinutes >= 1440) {+ // Daily bucket: group by date+ const stmt = this.db.prepare(`+ SELECT+ substr(hour, 1, 10) || 'T00:00:00' as time,+ SUM(upload) as upload,+ SUM(download) as download+ FROM hourly_stats+ WHERE backend_id = ? AND hour >= ? AND hour <= ?+ GROUP BY substr(hour, 1, 10)+ ORDER BY time ASC+ `);+ return stmt.all(backendId, startHour, endHour) as Array<{ time: string; upload: number; download: number }>;+ }++ if (bucketMinutes === 60) {+ // Hourly bucket: hourly_stats rows are already hourly+ const stmt = this.db.prepare(`+ SELECT hour as time, upload, download+ FROM hourly_stats+ WHERE backend_id = ? AND hour >= ? AND hour <= ?+ ORDER BY time ASC+ `);+ return stmt.all(backendId, startHour, endHour) as Array<{ time: string; upload: number; download: number }>;+ }++ // Other multiples of 60 (120, 180, etc.): bucket from hourly_stats+ const bucketSeconds = bucketMinutes * 60;+ const bucketExpr = `strftime('%Y-%m-%dT%H:%M:00', datetime((strftime('%s', datetime(hour)) / ${bucketSeconds}) * ${bucketSeconds}, 'unixepoch'))`;+ const stmt = this.db.prepare(`+ SELECT+ ${bucketExpr} as time,+ SUM(upload) as upload,+ SUM(download) as download+ FROM hourly_stats+ WHERE backend_id = ? AND hour >= ? AND hour <= ?+ GROUP BY ${bucketExpr}+ ORDER BY time ASC+ `);+ return stmt.all(backendId, startHour, endHour) as Array<{ time: string; upload: number; download: number }>;+ }++ // bucketMinutes < 60: use minute_statsif (bucketMinutes <= 1) {const stmt = this.db.prepare(`SELECT minute as time, upload, download@@ -108,7 +184,7 @@ export class TimeseriesRepository extends BaseRepository {WHERE backend_id = ? AND minute >= ? AND minute <= ?ORDER BY minute ASC`);- return stmt.all(backendId, range?.startMinute || cutoffStr, endMinute) as Array<{ time: string; upload: number; download: number }>;+ return stmt.all(backendId, startMinute, endMinute) as Array<{ time: string; upload: number; download: number }>;}const bucketExpr = `strftime('%Y-%m-%dT%H:%M:00', datetime((strftime('%s', datetime(minute)) / ${bucketMinutes * 60}) * ${bucketMinutes * 60}, 'unixepoch'))`;@@ -122,6 +198,6 @@ export class TimeseriesRepository extends BaseRepository {GROUP BY ${bucketExpr}ORDER BY time ASC`);- return stmt.all(backendId, range?.startMinute || cutoffStr, endMinute) as Array<{ time: string; upload: number; download: number }>;+ return stmt.all(backendId, startMinute, endMinute) as Array<{ time: string; upload: number; download: number }>;}}diff --git a/apps/collector/src/database/repositories/traffic-writer.repository.ts b/apps/collector/src/database/repositories/traffic-writer.repository.tsindex 1e5282a..61e0674 100644--- a/apps/collector/src/database/repositories/traffic-writer.repository.ts+++ b/apps/collector/src/database/repositories/traffic-writer.repository.ts@@ -110,6 +110,18 @@ export class TrafficWriterRepository extends BaseRepository {domains = CASE WHEN ip_proxy_stats.domains IS NULL THEN @domain WHEN @domain = 'unknown' THEN ip_proxy_stats.domainsWHEN INSTR(ip_proxy_stats.domains, @domain) > 0 THEN ip_proxy_stats.domains ELSE ip_proxy_stats.domains || ',' || @domain END`),+ minuteDimUpsert: this.db.prepare(`+ INSERT INTO minute_dim_stats (backend_id, minute, domain, ip, source_ip, chain, rule, upload, download, connections)+ VALUES (@backendId, @minute, @domain, @ip, @sourceIP, @chain, @rule, @upload, @download, 1)+ ON CONFLICT(backend_id, minute, domain, ip, source_ip, chain, rule) DO UPDATE SET+ upload = upload + @upload, download = download + @download, connections = connections + 1+ `),+ hourlyDimUpsert: this.db.prepare(`+ INSERT INTO hourly_dim_stats (backend_id, hour, domain, ip, source_ip, chain, rule, upload, download, connections)+ VALUES (@backendId, @hour, @domain, @ip, @sourceIP, @chain, @rule, @upload, @download, 1)+ ON CONFLICT(backend_id, hour, domain, ip, source_ip, chain, rule) DO UPDATE SET+ upload = upload + @upload, download = download + @download, connections = connections + 1+ `),};}@@ -163,6 +175,11 @@ export class TrafficWriterRepository extends BaseRepository {}s.ipProxyUpsert.run({ backendId, ip: update.ip, chain: fullChain, upload: update.upload, download: update.download, timestamp, domain: update.domain || 'unknown' });++ // Write to minute_dim_stats and hourly_dim_stats+ const dimParams = { backendId, domain: update.domain || '', ip: update.ip || '', sourceIP: update.sourceIP || '', chain: fullChain, rule: ruleName, upload: update.upload, download: update.download };+ s.minuteDimUpsert.run({ ...dimParams, minute });+ s.hourlyDimUpsert.run({ ...dimParams, hour });});transaction();@@ -188,6 +205,10 @@ export class TrafficWriterRepository extends BaseRepository {minute: string; domain: string; ip: string; sourceIP: string;chain: string; rule: string; upload: number; download: number; connections: number;}>();+ const hourlyDimMap = new Map<string, {+ hour: string; domain: string; ip: string; sourceIP: string;+ chain: string; rule: string; upload: number; download: number; connections: number;+ }>();const domainProxyMap = new Map<string, { domain: string; chain: string; upload: number; download: number; count: number }>();const ipProxyMap = new Map<string, { ip: string; chain: string; upload: number; download: number; count: number; domains: Set<string> }>();const deviceMap = new Map<string, { sourceIP: string; upload: number; download: number; count: number }>();@@ -277,6 +298,12 @@ export class TrafficWriterRepository extends BaseRepository {if (existingDim) { existingDim.upload += update.upload; existingDim.download += update.download; existingDim.connections++; }else { minuteDimMap.set(dimKey, { minute: minuteKey, domain: update.domain || '', ip: update.ip || '', sourceIP: update.sourceIP || '', chain: fullChain, rule: ruleName, upload: update.upload, download: update.download, connections: 1 }); }+ // Aggregate hourly_dim_stats+ const hourlyDimKey = `${hourKey}:${update.domain || ''}:${update.ip || ''}:${update.sourceIP || ''}:${fullChain}:${ruleName}`;+ const existingHourlyDim = hourlyDimMap.get(hourlyDimKey);+ if (existingHourlyDim) { existingHourlyDim.upload += update.upload; existingHourlyDim.download += update.download; existingHourlyDim.connections++; }+ else { hourlyDimMap.set(hourlyDimKey, { hour: hourKey, domain: update.domain || '', ip: update.ip || '', sourceIP: update.sourceIP || '', chain: fullChain, rule: ruleName, upload: update.upload, download: update.download, connections: 1 }); }+// Aggregate domain_proxy_statsif (update.domain) {const dpKey = `${update.domain}:${fullChain}`;@@ -425,6 +452,14 @@ export class TrafficWriterRepository extends BaseRepository {`);for (const [, data] of minuteDimMap) { minuteDimStmt.run({ backendId, minute: data.minute, domain: data.domain, ip: data.ip, sourceIP: data.sourceIP, chain: data.chain, rule: data.rule, upload: data.upload, download: data.download, connections: data.connections }); }+ const hourlyDimStmt = this.db.prepare(`+ INSERT INTO hourly_dim_stats (backend_id, hour, domain, ip, source_ip, chain, rule, upload, download, connections)+ VALUES (@backendId, @hour, @domain, @ip, @sourceIP, @chain, @rule, @upload, @download, @connections)+ ON CONFLICT(backend_id, hour, domain, ip, source_ip, chain, rule) DO UPDATE SET+ upload = upload + @upload, download = download + @download, connections = connections + @connections+ `);+ for (const [, data] of hourlyDimMap) { hourlyDimStmt.run({ backendId, hour: data.hour, domain: data.domain, ip: data.ip, sourceIP: data.sourceIP, chain: data.chain, rule: data.rule, upload: data.upload, download: data.download, connections: data.connections }); }+const domainProxyStmt = this.db.prepare(`INSERT INTO domain_proxy_stats (backend_id, domain, chain, total_upload, total_download, total_connections, last_seen)VALUES (@backendId, @domain, @chain, @upload, @download, @count, @timestamp)diff --git a/apps/collector/src/database/schema.ts b/apps/collector/src/database/schema.tsindex 25ee0d1..bef1356 100644--- a/apps/collector/src/database/schema.ts+++ b/apps/collector/src/database/schema.ts@@ -243,6 +243,40 @@ export const SCHEMA = {);`,+ // Hourly-level fact table for efficient long-range queries (>2h)+ HOURLY_DIM_STATS: `+ CREATE TABLE IF NOT EXISTS hourly_dim_stats (+ backend_id INTEGER NOT NULL,+ hour TEXT NOT NULL,+ domain TEXT NOT NULL DEFAULT '',+ ip TEXT NOT NULL DEFAULT '',+ source_ip TEXT NOT NULL DEFAULT '',+ chain TEXT NOT NULL,+ rule TEXT NOT NULL,+ upload INTEGER DEFAULT 0,+ download INTEGER DEFAULT 0,+ connections INTEGER DEFAULT 0,+ PRIMARY KEY (backend_id, hour, domain, ip, source_ip, chain, rule),+ FOREIGN KEY (backend_id) REFERENCES backend_configs(id) ON DELETE CASCADE+ );+ `,++ // Hourly-level country facts for efficient long-range queries+ HOURLY_COUNTRY_STATS: `+ CREATE TABLE IF NOT EXISTS hourly_country_stats (+ backend_id INTEGER NOT NULL,+ hour TEXT NOT NULL,+ country TEXT NOT NULL,+ country_name TEXT,+ continent TEXT,+ upload INTEGER DEFAULT 0,+ download INTEGER DEFAULT 0,+ connections INTEGER DEFAULT 0,+ PRIMARY KEY (backend_id, hour, country),+ FOREIGN KEY (backend_id) REFERENCES backend_configs(id) ON DELETE CASCADE+ );+ `,+// Domain×proxy traffic aggregationDOMAIN_PROXY_STATS: `CREATE TABLE IF NOT EXISTS domain_proxy_stats (@@ -406,6 +440,15 @@ export const INDEXES = [`CREATE INDEX IF NOT EXISTS idx_minute_dim_backend_minute_source ON minute_dim_stats(backend_id, minute, source_ip);`,`CREATE INDEX IF NOT EXISTS idx_minute_country_backend_minute ON minute_country_stats(backend_id, minute);`,+ // Hourly dim stats indexes+ `CREATE INDEX IF NOT EXISTS idx_hourly_dim_backend_hour ON hourly_dim_stats(backend_id, hour);`,+ `CREATE INDEX IF NOT EXISTS idx_hourly_dim_backend_hour_domain ON hourly_dim_stats(backend_id, hour, domain);`,+ `CREATE INDEX IF NOT EXISTS idx_hourly_dim_backend_hour_ip ON hourly_dim_stats(backend_id, hour, ip);`,+ `CREATE INDEX IF NOT EXISTS idx_hourly_dim_backend_hour_chain ON hourly_dim_stats(backend_id, hour, chain);`,+ `CREATE INDEX IF NOT EXISTS idx_hourly_dim_backend_hour_rule ON hourly_dim_stats(backend_id, hour, rule);`,+ `CREATE INDEX IF NOT EXISTS idx_hourly_dim_backend_hour_source ON hourly_dim_stats(backend_id, hour, source_ip);`,+ `CREATE INDEX IF NOT EXISTS idx_hourly_country_backend_hour ON hourly_country_stats(backend_id, hour);`,+// Connection logs indexes`CREATE INDEX IF NOT EXISTS idx_connection_logs_backend ON connection_logs(backend_id);`,`CREATE INDEX IF NOT EXISTS idx_connection_logs_timestamp ON connection_logs(timestamp);`,@@ -454,6 +497,8 @@ export function getAllSchemaStatements(): string[] {SCHEMA.MINUTE_STATS,SCHEMA.MINUTE_DIM_STATS,SCHEMA.MINUTE_COUNTRY_STATS,+ SCHEMA.HOURLY_DIM_STATS,+ SCHEMA.HOURLY_COUNTRY_STATS,SCHEMA.DOMAIN_PROXY_STATS,SCHEMA.IP_PROXY_STATS,SCHEMA.RULE_CHAIN_TRAFFIC,diff --git a/apps/collector/src/db.ts b/apps/collector/src/db.tsindex 614e15e..3960d05 100644--- a/apps/collector/src/db.ts+++ b/apps/collector/src/db.ts@@ -50,6 +50,7 @@ export class StatsDatabase {// Cached prepared statements for getSummary() — avoids re-compilation per callprivate _summaryStmts: ReturnType<StatsDatabase['prepareSummaryStmts']> | null = null;+ private rangeQueryCache = new Map<string, { value: unknown; expiresAt: number }>();public readonly repos: {auth: AuthRepository;@@ -93,7 +94,7 @@ export class StatsDatabase {this.db.pragma('synchronous = NORMAL');this.db.pragma('wal_autocheckpoint = 1000');this.db.pragma('temp_store = MEMORY');- this.db.pragma('cache_size = -16000'); // 16MB page cache+ this.db.pragma('cache_size = -65536'); // 64MB page cachethis.db.pragma('busy_timeout = 5000');// Apply all schema statements from the single source of truth@@ -111,6 +112,9 @@ export class StatsDatabase {// Migrate existing data if needed (from single-backend schema)this.migrateIfNeeded();++ // Backfill hourly tables from minute data (one-time)+ this.backfillHourlyTables();}// Migrate from old single-backend schema to multi-backend schema@@ -239,6 +243,55 @@ export class StatsDatabase {}}+ /**+ * One-time backfill: populate hourly_dim_stats and hourly_country_stats+ * from existing minute_dim_stats and minute_country_stats data.+ */+ private backfillHourlyTables() {+ const hourlyDimCount = (this.db.prepare(`SELECT COUNT(*) as c FROM hourly_dim_stats`).get() as { c: number }).c;+ const minuteDimCount = (this.db.prepare(`SELECT COUNT(*) as c FROM minute_dim_stats`).get() as { c: number }).c;++ if (hourlyDimCount > 0 || minuteDimCount === 0) return;++ console.log(`[DB] Backfilling hourly tables from ${minuteDimCount} minute_dim_stats rows...`);++ try {+ this.db.exec(`BEGIN TRANSACTION`);++ // Backfill hourly_dim_stats from minute_dim_stats+ this.db.exec(`+ INSERT INTO hourly_dim_stats (backend_id, hour, domain, ip, source_ip, chain, rule, upload, download, connections)+ SELECT backend_id,+ SUBSTR(minute, 1, 13) || ':00:00' as hour,+ domain, ip, source_ip, chain, rule,+ SUM(upload), SUM(download), SUM(connections)+ FROM minute_dim_stats+ GROUP BY backend_id, SUBSTR(minute, 1, 13), domain, ip, source_ip, chain, rule+ `);++ // Backfill hourly_country_stats from minute_country_stats+ const minuteCountryCount = (this.db.prepare(`SELECT COUNT(*) as c FROM minute_country_stats`).get() as { c: number }).c;+ if (minuteCountryCount > 0) {+ this.db.exec(`+ INSERT INTO hourly_country_stats (backend_id, hour, country, country_name, continent, upload, download, connections)+ SELECT backend_id,+ SUBSTR(minute, 1, 13) || ':00:00' as hour,+ country, MAX(country_name), MAX(continent),+ SUM(upload), SUM(download), SUM(connections)+ FROM minute_country_stats+ GROUP BY backend_id, SUBSTR(minute, 1, 13), country+ `);+ }++ this.db.exec(`COMMIT`);+ const hourlyRows = (this.db.prepare(`SELECT COUNT(*) as c FROM hourly_dim_stats`).get() as { c: number }).c;+ console.log(`[DB] Backfill complete: ${hourlyRows} hourly_dim_stats rows created`);+ } catch (error) {+ this.db.exec(`ROLLBACK`);+ console.error('[DB] Hourly backfill failed:', error);+ }+ }+private performMigration() {// Create temporary tables with new schema and migrate data// This is a complex migration - we'll create a default backend and associate all data with it@@ -456,19 +509,83 @@ export class StatsDatabase {// ==================== Domain ====================getDomainByName(backendId: number, domain: string) { return this.repos.domain.getDomainByName(backendId, domain); }- getDomainStats(backendId: number, limit = 100, start?: string, end?: string) { return this.repos.domain.getDomainStats(backendId, limit, start, end); }- getTopDomains(backendId: number, limit = 10, start?: string, end?: string) { return this.repos.domain.getTopDomains(backendId, limit, start, end); }- getDomainStatsPaginated(backendId: number, opts: { offset?: number; limit?: number; sortBy?: string; sortOrder?: string; search?: string; start?: string; end?: string } = {}) { return this.repos.domain.getDomainStatsPaginated(backendId, opts); }+ getDomainStats(backendId: number, limit = 100, start?: string, end?: string) {+ return this.withRangeQueryCache(+ 'domainStats',+ [backendId, limit, start || '', end || ''],+ start,+ end,+ () => this.repos.domain.getDomainStats(backendId, limit, start, end),+ );+ }+ getTopDomains(backendId: number, limit = 10, start?: string, end?: string) {+ return this.withRangeQueryCache(+ 'topDomains',+ [backendId, limit, start || '', end || ''],+ start,+ end,+ () => this.repos.domain.getTopDomains(backendId, limit, start, end),+ );+ }+ getTopDomainsLight(backendId: number, limit = 10, start?: string, end?: string) {+ return this.withRangeQueryCache(+ 'topDomainsLight',+ [backendId, limit, start || '', end || ''],+ start,+ end,+ () => this.repos.domain.getTopDomainsLight(backendId, limit, start, end),+ );+ }+ getDomainStatsPaginated(+ backendId: number,+ opts: { offset?: number; limit?: number; sortBy?: string; sortOrder?: string; search?: string; start?: string; end?: string } = {},+ ) {+ return this.withRangeQueryCache(+ 'domainStatsPaginated',+ [backendId, opts.offset || 0, opts.limit || 50, opts.sortBy || '', opts.sortOrder || '', opts.search || '', opts.start || '', opts.end || ''],+ opts.start,+ opts.end,+ () => this.repos.domain.getDomainStatsPaginated(backendId, opts),+ );+ }getDomainIPDetails(backendId: number, domain: string, start?: string, end?: string, limit?: number, sourceIP?: string, sourceChain?: string) { return this.repos.ip.getDomainIPDetails(backendId, domain, start, end, limit, sourceIP, sourceChain); }getDomainProxyStats(backendId: number, domain: string, start?: string, end?: string, sourceIP?: string, sourceChain?: string) { return this.repos.domain.getDomainProxyStats(backendId, domain, start, end, sourceIP, sourceChain); }// ==================== IP ====================- getIPStats(backendId: number, limit = 100, start?: string, end?: string) { return this.repos.ip.getIPStats(backendId, limit, start, end); }+ getIPStats(backendId: number, limit = 100, start?: string, end?: string) {+ return this.withRangeQueryCache(+ 'ipStats',+ [backendId, limit, start || '', end || ''],+ start,+ end,+ () => this.repos.ip.getIPStats(backendId, limit, start, end),+ );+ }getIPStatsByIPs(backendId: number, ips: string[]) { return this.repos.ip.getIPStatsByIPs(backendId, ips); }- getIPStatsPaginated(backendId: number, opts: { offset?: number; limit?: number; sortBy?: string; sortOrder?: string; search?: string; start?: string; end?: string } = {}) { return this.repos.ip.getIPStatsPaginated(backendId, opts); }+ getIPStatsPaginated(+ backendId: number,+ opts: { offset?: number; limit?: number; sortBy?: string; sortOrder?: string; search?: string; start?: string; end?: string } = {},+ ) {+ return this.withRangeQueryCache(+ 'ipStatsPaginated',+ [backendId, opts.offset || 0, opts.limit || 50, opts.sortBy || '', opts.sortOrder || '', opts.search || '', opts.start || '', opts.end || ''],+ opts.start,+ opts.end,+ () => this.repos.ip.getIPStatsPaginated(backendId, opts),+ );+ }getIPDomainDetails(backendId: number, ip: string, start?: string, end?: string, limit?: number, sourceIP?: string, sourceChain?: string) { return this.repos.ip.getIPDomainDetails(backendId, ip, start, end, limit, sourceIP, sourceChain); }getIPProxyStats(backendId: number, ip: string, start?: string, end?: string, sourceIP?: string, sourceChain?: string) { return this.repos.ip.getIPProxyStats(backendId, ip, start, end, sourceIP, sourceChain); }getTopIPs(backendId: number, limit = 10, start?: string, end?: string) { return this.getIPStats(backendId, limit, start, end); }+ getTopIPsLight(backendId: number, limit = 10, start?: string, end?: string) {+ return this.withRangeQueryCache(+ 'topIPsLight',+ [backendId, limit, start || '', end || ''],+ start,+ end,+ () => this.repos.ip.getTopIPsLight(backendId, limit, start, end),+ );+ }updateASNInfo(ip: string, asn: string, org: string) { this.repos.ip.updateASNInfo(ip, asn, org); }getASNInfo(ips: string[]) { return this.repos.ip.getASNInfo(ips); }getASNInfoForIP(ip: string) { return this.repos.ip.getASNInfoForIP(ip); }@@ -476,29 +593,93 @@ export class StatsDatabase {saveIPGeolocation(ip: string, geo: { country: string; country_name: string; city: string; asn: string; as_name: string; as_domain: string; continent: string; continent_name: string }) { this.repos.ip.saveIPGeolocation(ip, geo); }// ==================== Timeseries ====================- getHourlyStats(backendId: number, hours = 24, start?: string, end?: string) { return this.repos.timeseries.getHourlyStats(backendId, hours, start, end); }+ getHourlyStats(backendId: number, hours = 24, start?: string, end?: string) {+ return this.withRangeQueryCache(+ 'hourlyStats',+ [backendId, hours, start || '', end || ''],+ start,+ end,+ () => this.repos.timeseries.getHourlyStats(backendId, hours, start, end),+ );+ }getTodayTraffic(backendId: number) { return this.repos.timeseries.getTodayTraffic(backendId); }- getTrafficInRange(backendId: number, start?: string, end?: string) { return this.repos.timeseries.getTrafficInRange(backendId, start, end); }- getTrafficTrend(backendId: number, minutes?: number, start?: string, end?: string) { return this.repos.timeseries.getTrafficTrend(backendId, minutes, start, end); }- getTrafficTrendAggregated(backendId: number, minutes?: number, bucketMinutes?: number, start?: string, end?: string) { return this.repos.timeseries.getTrafficTrendAggregated(backendId, minutes, bucketMinutes, start, end); }+ getTrafficInRange(backendId: number, start?: string, end?: string) {+ return this.withRangeQueryCache(+ 'trafficInRange',+ [backendId, start || '', end || ''],+ start,+ end,+ () => this.repos.timeseries.getTrafficInRange(backendId, start, end),+ );+ }+ getTrafficTrend(backendId: number, minutes?: number, start?: string, end?: string) {+ return this.withRangeQueryCache(+ 'trafficTrend',+ [backendId, minutes || 30, start || '', end || ''],+ start,+ end,+ () => this.repos.timeseries.getTrafficTrend(backendId, minutes, start, end),+ );+ }+ getTrafficTrendAggregated(backendId: number, minutes?: number, bucketMinutes?: number, start?: string, end?: string) {+ return this.withRangeQueryCache(+ 'trafficTrendAggregated',+ [backendId, minutes || 30, bucketMinutes || 1, start || '', end || ''],+ start,+ end,+ () => this.repos.timeseries.getTrafficTrendAggregated(backendId, minutes, bucketMinutes, start, end),+ );+ }// ==================== Country ====================- getCountryStats(backendId: number, limit?: number, start?: string, end?: string) { return this.repos.country.getCountryStats(backendId, limit, start, end); }+ getCountryStats(backendId: number, limit?: number, start?: string, end?: string) {+ return this.withRangeQueryCache(+ 'countryStats',+ [backendId, limit || 50, start || '', end || ''],+ start,+ end,+ () => this.repos.country.getCountryStats(backendId, limit, start, end),+ );+ }updateCountryStats(backendId: number, country: string, countryName: string, continent: string, upload: number, download: number) { this.repos.country.updateCountryStats(backendId, country, countryName, continent, upload, download); }batchUpdateCountryStats(backendId: number, results: Array<{ country: string; countryName: string; continent: string; upload: number; download: number; timestampMs?: number }>) { this.repos.country.batchUpdateCountryStats(backendId, results); }// ==================== Device ====================- getDevices(backendId: number, limit?: number, start?: string, end?: string) { return this.repos.device.getDevices(backendId, limit, start, end); }+ getDevices(backendId: number, limit?: number, start?: string, end?: string) {+ return this.withRangeQueryCache(+ 'deviceStats',+ [backendId, limit || 50, start || '', end || ''],+ start,+ end,+ () => this.repos.device.getDevices(backendId, limit, start, end),+ );+ }getDeviceDomains(backendId: number, sourceIP: string, limit?: number, start?: string, end?: string) { return this.repos.device.getDeviceDomains(backendId, sourceIP, limit, start, end); }getDeviceIPs(backendId: number, sourceIP: string, limit?: number, start?: string, end?: string) { return this.repos.device.getDeviceIPs(backendId, sourceIP, limit, start, end); }// ==================== Proxy ====================- getProxyStats(backendId: number, start?: string, end?: string) { return this.repos.proxy.getProxyStats(backendId, start, end); }+ getProxyStats(backendId: number, start?: string, end?: string) {+ return this.withRangeQueryCache(+ 'proxyStats',+ [backendId, start || '', end || ''],+ start,+ end,+ () => this.repos.proxy.getProxyStats(backendId, start, end),+ );+ }getProxyDomains(backendId: number, chain: string, limit?: number, start?: string, end?: string) { return this.repos.proxy.getProxyDomains(backendId, chain, limit, start, end); }getProxyIPs(backendId: number, chain: string, limit?: number, start?: string, end?: string) { return this.repos.proxy.getProxyIPs(backendId, chain, limit, start, end); }// ==================== Rule ====================- getRuleStats(backendId: number, start?: string, end?: string) { return this.repos.rule.getRuleStats(backendId, start, end); }+ getRuleStats(backendId: number, start?: string, end?: string) {+ return this.withRangeQueryCache(+ 'ruleStats',+ [backendId, start || '', end || ''],+ start,+ end,+ () => this.repos.rule.getRuleStats(backendId, start, end),+ );+ }getRuleProxyMap(backendId: number) { return this.repos.rule.getRuleProxyMap(backendId); }getRuleDomains(backendId: number, rule: string, limit?: number, start?: string, end?: string) { return this.repos.rule.getRuleDomains(backendId, rule, limit, start, end); }getRuleIPs(backendId: number, rule: string, limit?: number, start?: string, end?: string) { return this.repos.rule.getRuleIPs(backendId, rule, limit, start, end); }@@ -525,18 +706,125 @@ export class StatsDatabase {return { startMinute: this.toMinuteKey(startDate), endMinute: this.toMinuteKey(endDate) };}+ private toHourKey(date: Date): string {+ return `${date.toISOString().slice(0, 13)}:00:00`;+ }++ private resolveFactTableRange(+ start?: string,+ end?: string,+ ): { table: 'hourly_dim_stats' | 'minute_dim_stats'; startKey: string; endKey: string } {+ if (!start || !end) {+ return { table: 'minute_dim_stats', startKey: '', endKey: '' };+ }+ const startDate = new Date(start);+ const endDate = new Date(end);+ if (Number.isNaN(startDate.getTime()) || Number.isNaN(endDate.getTime())) {+ return { table: 'minute_dim_stats', startKey: this.toMinuteKey(startDate), endKey: this.toMinuteKey(endDate) };+ }+ const rangeMs = endDate.getTime() - startDate.getTime();+ const SIX_HOURS_MS = 6 * 60 * 60 * 1000;+ if (rangeMs > SIX_HOURS_MS) {+ return { table: 'hourly_dim_stats', startKey: this.toHourKey(startDate), endKey: this.toHourKey(endDate) };+ }+ return { table: 'minute_dim_stats', startKey: this.toMinuteKey(startDate), endKey: this.toMinuteKey(endDate) };+ }++ private getRangeCacheTTL(end?: string): number {+ const realtimeTTL = Math.max(+ 1_000,+ parseInt(process.env.DB_RANGE_QUERY_CACHE_TTL_MS || '8000', 10) || 8000,+ );+ const historicalTTL = Math.max(+ realtimeTTL,+ parseInt(process.env.DB_HISTORICAL_QUERY_CACHE_TTL_MS || '300000', 10) || 300000,+ );+ if (!end) return realtimeTTL;++ const endMs = new Date(end).getTime();+ if (Number.isNaN(endMs)) return realtimeTTL;++ const toleranceMs = Math.max(+ 10_000,+ parseInt(process.env.REALTIME_RANGE_END_TOLERANCE_MS || '120000', 10) || 120000,+ );+ return endMs >= Date.now() - toleranceMs ? realtimeTTL : historicalTTL;+ }++ private pruneRangeQueryCache(): void {+ const maxEntries = Math.max(+ 128,+ parseInt(process.env.DB_RANGE_QUERY_CACHE_MAX_ENTRIES || '1024', 10) || 1024,+ );+ const now = Date.now();++ for (const [key, entry] of this.rangeQueryCache) {+ if (entry.expiresAt <= now) {+ this.rangeQueryCache.delete(key);+ }+ }++ while (this.rangeQueryCache.size > maxEntries) {+ const oldestKey = this.rangeQueryCache.keys().next().value as string | undefined;+ if (!oldestKey) break;+ this.rangeQueryCache.delete(oldestKey);+ }+ }++ private withRangeQueryCache<T>(+ prefix: string,+ parts: Array<string | number>,+ start: string | undefined,+ end: string | undefined,+ compute: () => T,+ ): T {+ if (process.env.DB_RANGE_QUERY_CACHE_DISABLED === '1') {+ return compute();+ }++ if (!this.parseMinuteRange(start, end)) {+ return compute();+ }++ const key = `${prefix}:${parts.join('|')}`;+ const now = Date.now();+ const cached = this.rangeQueryCache.get(key);+ if (cached && cached.expiresAt > now) {+ return cached.value as T;+ }++ const value = compute();+ const ttl = this.getRangeCacheTTL(end);+ this.rangeQueryCache.set(key, {+ value,+ expiresAt: now + ttl,+ });+ this.pruneRangeQueryCache();+ return value;+ }+private prepareSummaryStmts() {return {- rangeTraffic: this.db.prepare(`+ rangeCombined: this.db.prepare(`SELECTCOALESCE(SUM(connections), 0) as connections,COALESCE(SUM(upload), 0) as upload,- COALESCE(SUM(download), 0) as download+ COALESCE(SUM(download), 0) as download,+ COUNT(DISTINCT CASE WHEN domain != '' THEN domain END) as uniqueDomains,+ COUNT(DISTINCT CASE WHEN ip != '' THEN ip END) as uniqueIPsFROM minute_dim_statsWHERE backend_id = ? AND minute >= ? AND minute <= ?`),- rangeDomains: this.db.prepare(`SELECT COUNT(DISTINCT domain) as count FROM minute_dim_stats WHERE backend_id = ? AND minute >= ? AND minute <= ? AND domain != ''`),- rangeIPs: this.db.prepare(`SELECT COUNT(DISTINCT ip) as count FROM minute_dim_stats WHERE backend_id = ? AND minute >= ? AND minute <= ? AND ip != ''`),+ rangeCombinedHourly: this.db.prepare(`+ SELECT+ COALESCE(SUM(connections), 0) as connections,+ COALESCE(SUM(upload), 0) as upload,+ COALESCE(SUM(download), 0) as download,+ COUNT(DISTINCT CASE WHEN domain != '' THEN domain END) as uniqueDomains,+ COUNT(DISTINCT CASE WHEN ip != '' THEN ip END) as uniqueIPs+ FROM hourly_dim_stats+ WHERE backend_id = ? AND hour >= ? AND hour <= ?+ `),allTraffic: this.db.prepare(`SELECT COALESCE(SUM(total_connections), 0) as connections, COALESCE(SUM(total_upload), 0) as upload, COALESCE(SUM(total_download), 0) as download FROM ip_stats WHERE backend_id = ?`),allDomains: this.db.prepare(`SELECT COUNT(DISTINCT domain) as count FROM domain_stats WHERE backend_id = ?`),allIPs: this.db.prepare(`SELECT COUNT(DISTINCT ip) as count FROM ip_stats WHERE backend_id = ?`),@@ -556,18 +844,28 @@ export class StatsDatabase {start?: string,end?: string,): { totalConnections: number; totalUpload: number; totalDownload: number; uniqueDomains: number; uniqueIPs: number } {- const range = this.parseMinuteRange(start, end);- const s = this.summaryStmts;- if (range) {- const { connections, upload, download } = s.rangeTraffic.get(backendId, range.startMinute, range.endMinute) as { connections: number; upload: number; download: number };- const uniqueDomains = (s.rangeDomains.get(backendId, range.startMinute, range.endMinute) as { count: number }).count;- const uniqueIPs = (s.rangeIPs.get(backendId, range.startMinute, range.endMinute) as { count: number }).count;- return { totalConnections: connections, totalUpload: upload, totalDownload: download, uniqueDomains, uniqueIPs };- }- const { connections, upload, download } = s.allTraffic.get(backendId) as { connections: number; upload: number; download: number };- const uniqueDomains = (s.allDomains.get(backendId) as { count: number }).count;- const uniqueIPs = (s.allIPs.get(backendId) as { count: number }).count;- return { totalConnections: connections, totalUpload: upload, totalDownload: download, uniqueDomains, uniqueIPs };+ return this.withRangeQueryCache(+ 'summary',+ [backendId, start || '', end || ''],+ start,+ end,+ () => {+ const range = this.parseMinuteRange(start, end);+ const s = this.summaryStmts;+ if (range) {+ const resolved = this.resolveFactTableRange(start, end);+ const stmt = resolved.table === 'hourly_dim_stats' ? s.rangeCombinedHourly : s.rangeCombined;+ const { connections, upload, download, uniqueDomains, uniqueIPs } = stmt.get(+ backendId, resolved.startKey, resolved.endKey,+ ) as { connections: number; upload: number; download: number; uniqueDomains: number; uniqueIPs: number };+ return { totalConnections: connections, totalUpload: upload, totalDownload: download, uniqueDomains, uniqueIPs };+ }+ const { connections, upload, download } = s.allTraffic.get(backendId) as { connections: number; upload: number; download: number };+ const uniqueDomains = (s.allDomains.get(backendId) as { count: number }).count;+ const uniqueIPs = (s.allIPs.get(backendId) as { count: number }).count;+ return { totalConnections: connections, totalUpload: upload, totalDownload: download, uniqueDomains, uniqueIPs };+ },+ );}/**diff --git a/apps/collector/src/modules/auth/auth.controller.ts b/apps/collector/src/modules/auth/auth.controller.tsindex b3fbf17..7c1b483 100644--- a/apps/collector/src/modules/auth/auth.controller.ts+++ b/apps/collector/src/modules/auth/auth.controller.ts@@ -44,11 +44,12 @@ export async function authController(app: FastifyInstance) {if (result.valid) {// Set valid token as HttpOnly cookie- const isProduction = process.env.NODE_ENV === 'production';+ // Only use secure if the connection is actually HTTPS+ const isSecure = request.protocol === 'https';reply.setCookie('neko-session', token, {path: '/',httpOnly: true,- secure: isProduction, // Only use secure in production+ secure: isSecure, // Only use secure on HTTPS connectionssameSite: 'strict',maxAge: 60 * 60 * 24 * 7, // 7 days});diff --git a/apps/collector/src/modules/stats/stats.service.ts b/apps/collector/src/modules/stats/stats.service.tsindex eaff15e..043d6c7 100644--- a/apps/collector/src/modules/stats/stats.service.ts+++ b/apps/collector/src/modules/stats/stats.service.ts@@ -105,12 +105,12 @@ export class StatsService {? this.realtimeStore.applySummaryDelta(backendId, summary): summary;- const dbTopDomains = this.db.getTopDomains(backendId, 10, timeRange.start, timeRange.end);+ const dbTopDomains = this.db.getTopDomainsLight(backendId, 10, timeRange.start, timeRange.end);const topDomains = includeRealtime? this.realtimeStore.mergeTopDomains(backendId, dbTopDomains, 10): dbTopDomains;- const dbTopIPs = this.db.getTopIPs(backendId, 10, timeRange.start, timeRange.end);+ const dbTopIPs = this.db.getTopIPsLight(backendId, 10, timeRange.start, timeRange.end);const topIPs = includeRealtime? this.realtimeStore.mergeTopIPs(backendId, dbTopIPs, 10): dbTopIPs;diff --git a/apps/collector/src/websocket.ts b/apps/collector/src/websocket.tsindex eefd528..3e27034 100644--- a/apps/collector/src/websocket.ts+++ b/apps/collector/src/websocket.ts@@ -119,6 +119,7 @@ export class StatsWebSocketServer {ts: number;}>();private static BASE_SUMMARY_CACHE_TTL_MS = 2000;+ private static BASE_SUMMARY_CACHE_TTL_HISTORICAL_MS = 300000; // 5 minutes for historical rangesconstructor(port: number, db: StatsDatabase) {this.port = port;@@ -644,6 +645,17 @@ export class StatsWebSocketServer {};}+ private getBaseSummaryCacheTTL(range: ClientRange): number {+ if (!range.end) return StatsWebSocketServer.BASE_SUMMARY_CACHE_TTL_MS;+ const endMs = new Date(range.end).getTime();+ if (Number.isNaN(endMs)) return StatsWebSocketServer.BASE_SUMMARY_CACHE_TTL_MS;+ const toleranceMs = parseInt(process.env.REALTIME_RANGE_END_TOLERANCE_MS || '120000', 10);+ const windowMs = Number.isFinite(toleranceMs) ? Math.max(10_000, toleranceMs) : 120_000;+ return endMs >= Date.now() - windowMs+ ? StatsWebSocketServer.BASE_SUMMARY_CACHE_TTL_MS+ : StatsWebSocketServer.BASE_SUMMARY_CACHE_TTL_HISTORICAL_MS;+ }+private resolveBackendId(rawBackendId: number | null): number | null {if (rawBackendId !== null) {return this.db.getBackend(rawBackendId) ? rawBackendId : null;@@ -679,11 +691,13 @@ export class StatsWebSocketServer {!domainsPage &&!ipsPage;+ const cacheTTL = this.getBaseSummaryCacheTTL(range);+const summary = wantsFullSummary? (() => {const baseCacheKey = `${resolvedBackendId}|${range.start || ''}|${range.end || ''}`;const cached = this.baseSummaryCache.get(baseCacheKey);- if (cached && Date.now() - cached.ts < StatsWebSocketServer.BASE_SUMMARY_CACHE_TTL_MS) {+ if (cached && Date.now() - cached.ts < cacheTTL) {return cached.summary;}const result = this.db.getSummary(resolvedBackendId, range.start, range.end);@@ -700,10 +714,10 @@ export class StatsWebSocketServer {// Use cached base summary data to avoid repeated expensive DB queriesconst baseCacheKey = `${resolvedBackendId}|${range.start || ''}|${range.end || ''}`;const baseCached = this.baseSummaryCache.get(baseCacheKey);- const baseCacheValid = baseCached && Date.now() - baseCached.ts < StatsWebSocketServer.BASE_SUMMARY_CACHE_TTL_MS;+ const baseCacheValid = baseCached && Date.now() - baseCached.ts < cacheTTL;const dbTopDomains = wantsFullSummary- ? (baseCacheValid ? baseCached!.topDomains : this.db.getTopDomains(resolvedBackendId, 100, range.start, range.end))+ ? (baseCacheValid ? baseCached!.topDomains : this.db.getTopDomainsLight(resolvedBackendId, 100, range.start, range.end)): undefined;const topDomains = wantsFullSummary && dbTopDomains? includeRealtime@@ -712,7 +726,7 @@ export class StatsWebSocketServer {: [];const dbTopIPs = wantsFullSummary- ? (baseCacheValid ? baseCached!.topIPs : this.db.getTopIPs(resolvedBackendId, 100, range.start, range.end))+ ? (baseCacheValid ? baseCached!.topIPs : this.db.getTopIPsLight(resolvedBackendId, 100, range.start, range.end)): undefined;const topIPs = wantsFullSummary && dbTopIPs? includeRealtime@@ -774,7 +788,7 @@ export class StatsWebSocketServer {});// Evict stale entriesfor (const [key, val] of this.baseSummaryCache) {- if (Date.now() - val.ts > StatsWebSocketServer.BASE_SUMMARY_CACHE_TTL_MS * 2) {+ if (Date.now() - val.ts > cacheTTL * 2) {this.baseSummaryCache.delete(key);}}diff --git a/apps/web/components/common/time-range-picker.tsx b/apps/web/components/common/time-range-picker.tsxindex a96894e..5be50df 100644--- a/apps/web/components/common/time-range-picker.tsx+++ b/apps/web/components/common/time-range-picker.tsx@@ -26,6 +26,7 @@ type PresetType =| "5m"| "15m"| "30m"+ | "1h"| "24h"| "7d"| "30d"@@ -68,6 +69,7 @@ function inferPresetFromRange(range: TimeRange): PresetType {if (diffMin === 5) return "5m";if (diffMin === 15) return "15m";if (diffMin === 30) return "30m";+ if (diffMin === 60) return "1h";if (diffMin === 24 * 60) return "24h";if (diffMin === 7 * 24 * 60) return "7d";if (diffMin === 30 * 24 * 60) return "30d";@@ -147,7 +149,7 @@ export function TimeRangePicker({value: Exclude<PresetType, "custom">;label: string;}> = [- { value: "30m", label: t("preset.30m") },+ { value: "1h", label: t("preset.1h") },{ value: "24h", label: t("preset.24h") },{ value: "7d", label: t("preset.7d") },{ value: "30d", label: t("preset.30d") },@@ -160,6 +162,7 @@ export function TimeRangePicker({{ value: "1m", label: t("preset.1m") },{ value: "5m", label: t("preset.5m") },{ value: "15m", label: t("preset.15m") },+ { value: "30m", label: t("preset.30m") },...stablePresets,];}, [showDebugShortPresets, t]);@@ -191,7 +194,8 @@ export function TimeRangePicker({!showDebugShortPresets &&(inferredPreset === "1m" ||inferredPreset === "5m" ||- inferredPreset === "15m")+ inferredPreset === "15m" ||+ inferredPreset === "30m")? "custom": inferredPreset;setSelectedPreset(normalizedPreset);@@ -453,9 +457,10 @@ export function QuickTimePresets({{ label: "1m", range: getPresetTimeRange("1m") },{ label: "5m", range: getPresetTimeRange("5m") },{ label: "15m", range: getPresetTimeRange("15m") },+ { label: "30m", range: getPresetTimeRange("30m") },]: []),- { label: "30m", range: getPresetTimeRange("30m") },+ { label: "1h", range: getPresetTimeRange("1h") },{ label: "24h", range: getPresetTimeRange("24h") },{ label: "7d", range: getPresetTimeRange("7d") },{ label: "30d", range: getPresetTimeRange("30d") },diff --git a/apps/web/components/common/top-list-item.tsx b/apps/web/components/common/top-list-item.tsxindex 17fdda8..d1bfeb4 100644--- a/apps/web/components/common/top-list-item.tsx+++ b/apps/web/components/common/top-list-item.tsx@@ -14,6 +14,7 @@ interface TopListItemProps {color?: string;showRank?: boolean;valueFormatter?: (value: number) => string;+ titleClassName?: string;}const RANK_COLORS: Record<number, string> = {@@ -32,6 +33,7 @@ export function TopListItem({color = "hsl(var(--primary))",showRank = true,valueFormatter,+ titleClassName,}: TopListItemProps) {const percentage = total > 0 ? (value / total) * 100 : 0;const isTop3 = rank <= 3;@@ -65,7 +67,7 @@ export function TopListItem({<div className="flex-1 min-w-0"><div className="flex items-center justify-between mb-1"><div className="flex items-center gap-2 min-w-0">- <span className="text-sm font-medium truncate" title={title}>+ <span className={cn("text-sm font-medium truncate", titleClassName)} title={title}>{title}</span>{subtitle && (diff --git a/apps/web/components/features/proxies/interactive-proxy-stats.tsx b/apps/web/components/features/proxies/interactive-proxy-stats.tsxindex 9956b19..1c7cc0f 100644--- a/apps/web/components/features/proxies/interactive-proxy-stats.tsx+++ b/apps/web/components/features/proxies/interactive-proxy-stats.tsx@@ -11,6 +11,7 @@ import { ScrollArea } from "@/components/ui/scroll-area";import { CountryFlag, extractCountryCodeFromText, stripLeadingFlagEmoji } from "@/components/features/countries/country-flag";import { formatBytes, formatNumber } from "@/lib/utils";import { cn } from "@/lib/utils";+import { useIsWindows } from "@/lib/hooks/use-is-windows";import { type TimeRange } from "@/lib/api";import { useStableTimeRange } from "@/lib/hooks/use-stable-time-range";import { useStatsWebSocket } from "@/lib/websocket";@@ -103,6 +104,7 @@ export function InteractiveProxyStats({const queryClient = useQueryClient();const stableTimeRange = useStableTimeRange(timeRange);const detailTimeRange = stableTimeRange;+ const isWindows = useIsWindows();const { data: listData, isLoading: listQueryLoading } = useProxies({activeBackendId,@@ -334,7 +336,7 @@ export function InteractiveProxyStats({<button key={item.rawName} onClick={() => handleProxyClick(item.rawName)} className={cn("w-full p-2.5 rounded-xl border text-left transition-all duration-200 overflow-hidden", isSelected ? "border-primary bg-primary/5 ring-1 ring-primary/20" : "border-border/50 bg-card/50 hover:bg-card hover:border-primary/30")}><div className="flex items-center gap-2 mb-1.5 min-w-0"><span className={cn("w-5 h-5 rounded-md text-[10px] font-bold flex items-center justify-center shrink-0", badgeColor)}>{item.rank + 1}</span>- <span className="flex-1 text-sm font-medium truncate min-w-0" title={rawDisplayName}>+ <span className={cn("flex-1 text-sm font-medium truncate min-w-0", isWindows && "emoji-flag-font")} title={rawDisplayName}>{rawDisplayName}</span><span className="text-sm font-bold tabular-nums shrink-0 whitespace-nowrap ml-auto">{formatBytes(item.value)}</span>diff --git a/apps/web/components/features/proxies/proxy-stats-chart.tsx b/apps/web/components/features/proxies/proxy-stats-chart.tsxindex 4a86998..71bc25d 100644--- a/apps/web/components/features/proxies/proxy-stats-chart.tsx+++ b/apps/web/components/features/proxies/proxy-stats-chart.tsx@@ -5,7 +5,8 @@ import { PieChart, Pie, Cell, ResponsiveContainer, Tooltip } from "recharts";import { Server, Link2 } from "lucide-react";import { useTranslations } from "next-intl";import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card";-import { formatBytes, formatNumber } from "@/lib/utils";+import { formatBytes, formatNumber, cn } from "@/lib/utils";+import { useIsWindows } from "@/lib/hooks/use-is-windows";import type { ProxyStats } from "@neko-master/shared";interface ProxyStatsChartProps {@@ -38,6 +39,7 @@ function formatProxyName(name: string): string {export function ProxyStatsChart({ data }: ProxyStatsChartProps) {const t = useTranslations("proxies");+ const isWindows = useIsWindows();const chartData = useMemo(() => {if (!data) return [];@@ -61,7 +63,7 @@ export function ProxyStatsChart({ data }: ProxyStatsChartProps) {const item = payload[0].payload;return (<div className="glass-card p-3 rounded-lg border shadow-lg">- <p className="font-medium text-sm mb-2">{item.name}</p>+ <p className={cn("font-medium text-sm mb-2", isWindows && "emoji-flag-font")}>{item.name}</p><div className="space-y-1 text-xs"><div className="flex justify-between gap-4"><span className="text-muted-foreground">{t("total")}:</span>@@ -132,7 +134,7 @@ export function ProxyStatsChart({ data }: ProxyStatsChartProps) {className="w-3 h-3 rounded-full shrink-0"style={{ backgroundColor: item.color }}/>- <p className="font-medium text-sm truncate" title={item.name}>+ <p className={cn("font-medium text-sm truncate", isWindows && "emoji-flag-font")} title={item.name}>{item.name}</p></div>diff --git a/apps/web/components/features/proxies/proxy-top-grid.tsx b/apps/web/components/features/proxies/proxy-top-grid.tsxindex 56e136f..3137a17 100644--- a/apps/web/components/features/proxies/proxy-top-grid.tsx+++ b/apps/web/components/features/proxies/proxy-top-grid.tsx@@ -10,7 +10,8 @@ import {extractCountryCodeFromText,stripLeadingFlagEmoji,} from "@/components/features/countries";-import { formatBytes, formatNumber } from "@/lib/utils";+import { formatBytes, formatNumber, cn } from "@/lib/utils";+import { useIsWindows } from "@/lib/hooks/use-is-windows";import type { ProxyStats } from "@neko-master/shared";interface ProxyTopGridProps {@@ -60,6 +61,7 @@ export function ProxyTopGrid({const [sortBy, setSortBy] = useState<SortBy>("traffic");const t = useTranslations("topProxies");const proxiesT = useTranslations("proxies");+ const isWindows = useIsWindows();const proxies = useMemo(() => {if (!data) return [];@@ -136,7 +138,7 @@ export function ProxyTopGrid({{/* Name */}<div className="flex-1 min-w-0">- <p className="font-medium text-sm truncate" title={proxy.chain}>+ <p className={cn("font-medium text-sm truncate", isWindows && "emoji-flag-font")} title={proxy.chain}>{proxy.displayName}</p></div>diff --git a/apps/web/components/features/proxies/proxy-top-list.tsx b/apps/web/components/features/proxies/proxy-top-list.tsxindex f349602..1e058af 100644--- a/apps/web/components/features/proxies/proxy-top-list.tsx+++ b/apps/web/components/features/proxies/proxy-top-list.tsx@@ -7,7 +7,8 @@ import { CountryFlag, extractCountryCodeFromText, stripLeadingFlagEmoji } from "import { OverviewCard } from "@/components/common";import { TopListItem } from "@/components/common";import { Button } from "@/components/ui/button";-import { formatBytes, formatNumber } from "@/lib/utils";+import { formatBytes, formatNumber, cn } from "@/lib/utils";+import { useIsWindows } from "@/lib/hooks/use-is-windows";import type { ProxyStats } from "@neko-master/shared";interface ProxyTopListProps {@@ -49,6 +50,7 @@ export function ProxyTopList({ data, limit = 5, onViewAll }: ProxyTopListProps)const [sortBy, setSortBy] = useState<SortBy>("traffic");const t = useTranslations("topProxies");const proxiesT = useTranslations("proxies");+ const isWindows = useIsWindows();const { proxies, totalTraffic, totalConnections } = useMemo(() => {if (!data) return { proxies: [], totalTraffic: 0, totalConnections: 0 };@@ -123,6 +125,7 @@ export function ProxyTopList({ data, limit = 5, onViewAll }: ProxyTopListProps)rank={index + 1}icon={<CountryFlag country={proxy.countryCode} className="h-4 w-6" />}title={proxy.displayName}+ titleClassName={cn(isWindows && "emoji-flag-font")}subtitle={sortBy === "traffic"? `${formatNumber(proxy.totalConnections)} ${proxiesT("connections")}`: `${formatBytes(proxy.total)}`diff --git a/apps/web/components/features/rules/interactive-rule-stats.tsx b/apps/web/components/features/rules/interactive-rule-stats.tsxindex 4d2696f..d6c6bd8 100644--- a/apps/web/components/features/rules/interactive-rule-stats.tsx+++ b/apps/web/components/features/rules/interactive-rule-stats.tsx@@ -38,6 +38,7 @@ import { Favicon } from "@/components/common/favicon";import { DomainPreview } from "@/components/features/domains/domain-preview";import { ProxyChainBadge } from "@/components/features/proxies/proxy-chain-badge";import { DomainExpandedDetails, IPExpandedDetails } from "@/components/features/stats/table/expanded-details";+import { useIsWindows } from "@/lib/hooks/use-is-windows";import { ExpandReveal } from "@/components/ui/expand-reveal";import { UnifiedRuleChainFlow } from "@/components/features/rules/rule-chain-flow";import { InsightChartSkeleton, InsightDetailSectionSkeleton, InsightTableSkeleton, InsightThreePanelSkeleton } from "@/components/ui/insight-skeleton";@@ -172,6 +173,7 @@ export function InteractiveRuleStats({const queryClient = useQueryClient();const stableTimeRange = useStableTimeRange(timeRange, { roundToMinute: true });const detailTimeRange = stableTimeRange;+ const isWindows = useIsWindows();const { data: listData, isLoading: listQueryLoading } = useRules({activeBackendId,@@ -700,7 +702,7 @@ export function InteractiveRuleStats({{idx + 1}</span><span- className="px-1.5 py-0.5 rounded-md text-[10px] font-medium text-white/90 truncate min-w-0"+ className={cn("px-1.5 py-0.5 rounded-md text-[10px] font-medium text-white/90 truncate min-w-0", isWindows && "emoji-flag-font")}style={{ backgroundColor: item.color }}>{item.name}@@ -765,7 +767,7 @@ export function InteractiveRuleStats({</span><span- className="flex-1 text-sm font-medium truncate min-w-0"+ className={cn("flex-1 text-sm font-medium truncate min-w-0", isWindows && "emoji-flag-font")}title={item.name}>{item.name}@@ -818,7 +820,7 @@ export function InteractiveRuleStats({)}>{noTraffic ? "–" : item.rank + 1}</span>- <span className="flex-1 text-sm font-medium line-clamp-2 leading-tight" title={item.name}>+ <span className={cn("flex-1 text-sm font-medium line-clamp-2 leading-tight", isWindows && "emoji-flag-font")} title={item.name}>{item.name}</span></div>@@ -884,7 +886,7 @@ export function InteractiveRuleStats({{t("topDomains")}</CardTitle>{selectedRuleData && (- <span className="text-xs text-muted-foreground">+ <span className={cn("text-xs text-muted-foreground", isWindows && "emoji-flag-font")}>{selectedRuleData.name}</span>)}diff --git a/apps/web/components/features/stats/charts/trend-chart.tsx b/apps/web/components/features/stats/charts/trend-chart.tsxindex 54d7043..fed1e43 100644--- a/apps/web/components/features/stats/charts/trend-chart.tsx+++ b/apps/web/components/features/stats/charts/trend-chart.tsx@@ -17,7 +17,7 @@ import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card";import { cn, formatBytes } from "@/lib/utils";import type { TrafficTrendPoint } from "@neko-master/shared";-type TimeRange = "30m" | "1h" | "24h";+type TimeRange = "30m" | "1h" | "24h" | "today";type TrendGranularity = "minute" | "day";interface TrafficTrendChartProps {@@ -54,6 +54,7 @@ export const TrafficTrendChart = React.memo({ value: "30m", label: t("30m") },{ value: "1h", label: t("1h") },{ value: "24h", label: t("24h") },+ { value: "today", label: t("today") },];return allOptions.filter((item) => timeRangeOptions.includes(item.value));}, [t, timeRangeOptions]);diff --git a/apps/web/components/overview/index.tsx b/apps/web/components/overview/index.tsxindex 9415ade..765df5b 100644--- a/apps/web/components/overview/index.tsx+++ b/apps/web/components/overview/index.tsx@@ -20,7 +20,7 @@ import type {StatsSummary,} from "@neko-master/shared";-type TrendTimeRange = "30m" | "1h" | "24h";+type TrendTimeRange = "30m" | "1h" | "24h" | "today";type TrendGranularity = "minute" | "day";export type GlobalTimePreset = TimePreset;@@ -56,17 +56,25 @@ function getMinuteBucket(durationMs: number): number {return 10;}-function getTrendQuickOptions(durationMs: number): TrendTimeRange[] {+function getTrendQuickOptions(durationMs: number, isToday = false): TrendTimeRange[] {const options: TrendTimeRange[] = [];if (durationMs >= THIRTY_MINUTES_MS) options.push("30m");if (durationMs >= ONE_HOUR_MS) options.push("1h");if (durationMs >= ONE_DAY_MS) options.push("24h");+ if (isToday) options.push("today");return options;}function getQuickRangeMinutes(range: TrendTimeRange): number {if (range === "30m") return 30;if (range === "1h") return 60;+ if (range === "today") {+ // Calculate minutes from midnight to now+ const now = new Date();+ const midnight = new Date(now);+ midnight.setHours(0, 0, 0, 0);+ return Math.round((now.getTime() - midnight.getTime()) / 60000);+ }return 1440;}@@ -145,8 +153,8 @@ export function OverviewTab({);const trendTimeOptions = useMemo(- () => (canUseTrendSelector ? getTrendQuickOptions(globalDurationMs) : []),- [canUseTrendSelector, globalDurationMs],+ () => (canUseTrendSelector ? getTrendQuickOptions(globalDurationMs, timePreset === "today") : []),+ [canUseTrendSelector, globalDurationMs, timePreset],);useEffect(() => {@@ -161,8 +169,15 @@ export function OverviewTab({let queryStart = parsedRange.start;if (canUseTrendSelector) {- const minutes = getQuickRangeMinutes(trendTimeRange);- queryStart = new Date(queryEnd.getTime() - minutes * 60 * 1000);+ if (trendTimeRange === "today") {+ // For "today", use midnight as start time+ const midnight = new Date(queryEnd);+ midnight.setHours(0, 0, 0, 0);+ queryStart = midnight;+ } else {+ const minutes = getQuickRangeMinutes(trendTimeRange);+ queryStart = new Date(queryEnd.getTime() - minutes * 60 * 1000);+ }}if (queryStart > queryEnd) {diff --git a/apps/web/lib/api.ts b/apps/web/lib/api.tsindex a70f49b..fd83fd5 100644--- a/apps/web/lib/api.ts+++ b/apps/web/lib/api.ts@@ -573,37 +573,60 @@ export const api = {// Helper functions for time rangeexport function getPresetTimeRange(- preset: "1m" | "5m" | "15m" | "30m" | "7d" | "30d" | "24h" | "today",+ preset: "1m" | "5m" | "15m" | "30m" | "1h" | "7d" | "30d" | "24h" | "today",): TimeRange {- const end = new Date();+ const now = new Date();+ const end = new Date(now);end.setMilliseconds(0);- const start = new Date(end);++ // Helper to get start of today in local time+ const getStartOfToday = () => {+ const d = new Date(now);+ d.setHours(0, 0, 0, 0);+ return d;+ };++ let start: Date;switch (preset) {case "1m":+ start = new Date(end);start.setMinutes(start.getMinutes() - 1);break;case "5m":+ start = new Date(end);start.setMinutes(start.getMinutes() - 5);break;case "15m":+ start = new Date(end);start.setMinutes(start.getMinutes() - 15);break;case "30m":+ start = new Date(end);start.setMinutes(start.getMinutes() - 30);break;+ case "1h":+ start = new Date(end);+ start.setHours(start.getHours() - 1);+ break;case '7d':+ start = new Date(end);start.setDate(start.getDate() - 7);break;case '30d':+ start = new Date(end);start.setDate(start.getDate() - 30);break;case '24h':+ start = new Date(end);start.setHours(start.getHours() - 24);break;case 'today':- start.setHours(0, 0, 0, 0);+ start = getStartOfToday();break;+ default:+ start = new Date(end);+ start.setHours(start.getHours() - 24);}return {diff --git a/apps/web/lib/websocket.ts b/apps/web/lib/websocket.tsindex e656250..ae87479 100644--- a/apps/web/lib/websocket.ts+++ b/apps/web/lib/websocket.ts@@ -4,10 +4,27 @@ import type { TimeRange } from '@/lib/api';export type ConnectionStatus = 'connecting' | 'connected' | 'disconnected' | 'error';+export interface LiveConnection {+ id: string;+ timestamp: number;+ network: string;+ sourceIP: string;+ sourcePort: string;+ destinationIP: string;+ destinationPort: string;+ domain?: string;+ rule: string;+ rulePayload?: string;+ proxy: string;+ upload: number;+ download: number;+}+interface WebSocketMessage {type: 'stats' | 'ping' | 'pong';backendId?: number;data?: StatsSummary;+ liveConnections?: LiveConnection[];timestamp: string;}@@ -40,9 +57,11 @@ interface UseStatsWebSocketOptions {ipsPageSortBy?: string;ipsPageSortOrder?: "asc" | "desc";ipsPageSearch?: string;+ includeLiveConnections?: boolean;+ liveConnectionsLimit?: number;trackLastMessage?: boolean;enabled?: boolean;- onMessage?: (data: StatsSummary) => void;+ onMessage?: (data: StatsSummary, liveConnections?: LiveConnection[]) => void;onConnect?: () => void;onDisconnect?: () => void;onError?: (error: Event) => void;@@ -155,11 +174,14 @@ export function useStatsWebSocket(options: UseStatsWebSocketOptions = {}) {ipsPageSortBy,ipsPageSortOrder,ipsPageSearch,+ includeLiveConnections,+ liveConnectionsLimit,trackLastMessage = true,enabled = true,} = options;const [status, setStatus] = useState<ConnectionStatus>('disconnected');const [lastMessage, setLastMessage] = useState<StatsSummary | null>(null);+ const [liveConnections, setLiveConnections] = useState<LiveConnection[]>([]);const [latency, setLatency] = useState<number | null>(null);const wsRef = useRef<WebSocket | null>(null);const reconnectTimerRef = useRef<NodeJS.Timeout | null>(null);@@ -250,7 +272,10 @@ export function useStatsWebSocket(options: UseStatsWebSocketOptions = {}) {if (trackLastMessageRef.current) {setLastMessage(message.data);}- onMessageRef.current?.(message.data);+ if (message.liveConnections) {+ setLiveConnections(message.liveConnections);+ }+ onMessageRef.current?.(message.data, message.liveConnections);} else if (message.type === 'pong') {if (lastPingTimeRef.current > 0) {setLatency(Date.now() - lastPingTimeRef.current);@@ -339,6 +364,8 @@ export function useStatsWebSocket(options: UseStatsWebSocketOptions = {}) {ipsPageSortBy,ipsPageSortOrder,ipsPageSearch,+ includeLiveConnections,+ liveConnectionsLimit,timestamp: new Date().toISOString(),}));}, [@@ -371,6 +398,8 @@ export function useStatsWebSocket(options: UseStatsWebSocketOptions = {}) {ipsPageSortBy,ipsPageSortOrder,ipsPageSearch,+ includeLiveConnections,+ liveConnectionsLimit,status,]);@@ -398,6 +427,7 @@ export function useStatsWebSocket(options: UseStatsWebSocketOptions = {}) {return {status,lastMessage,+ liveConnections,latency,connect,disconnect,diff --git a/apps/web/messages/en.json b/apps/web/messages/en.jsonindex 2174913..8f9af46 100644--- a/apps/web/messages/en.json+++ b/apps/web/messages/en.json@@ -65,6 +65,7 @@"5m": "5m","15m": "15m","30m": "30m",+ "1h": "1h","24h": "24h","7d": "7d","30d": "30d",@@ -109,6 +110,7 @@"30m": "30 min","1h": "1 hour","24h": "24 hours",+ "today": "Today","totalDownload": "Total DL","totalUpload": "Total UL","total": "Total",diff --git a/apps/web/messages/zh.json b/apps/web/messages/zh.jsonindex eba3cd1..daf29fb 100644--- a/apps/web/messages/zh.json+++ b/apps/web/messages/zh.json@@ -65,6 +65,7 @@"5m": "5m","15m": "15m","30m": "30m",+ "1h": "1h","24h": "24h","7d": "7d","30d": "30d",@@ -109,6 +110,7 @@"30m": "30分钟","1h": "1小时","24h": "24小时",+ "today": "今天","totalDownload": "总下载","totalUpload": "总上传","total": "总计",diff --git a/package.json b/package.jsonindex e148e3e..8c75dce 100644--- a/package.json+++ b/package.json@@ -1,6 +1,6 @@{"name": "neko-master",- "version": "1.2.7",+ "version": "1.2.8","description": "Modern traffic analytics for edge gateways","private": true,"workspaces": [