A modern and elegant dashboard for network traffic visualization and analysis.
feat: Introduce Surge backend support, enhance data collection with new repositories, and improve UI responsiveness for proxy statistics.
luoleifb97f01389ac
diff --git a/CHANGELOG.en.md b/CHANGELOG.en.mdindex 72517bb..cb77e1f 100644--- a/CHANGELOG.en.md+++ b/CHANGELOG.en.md@@ -7,6 +7,48 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0## [Unreleased]+## [1.2.7] - 2026-02-14++### Added++- **Surge Backend Support** ๐+ - Full support for Surge HTTP REST API data collection+ - Complete rule chain visualization (Rule Chain Flow)+ - Full feature support including proxy distribution charts and domain statistics+ - Intelligent policy cache system with background sync+ - Automatic retry mechanism with exponential backoff for failed API requests+ - Anti-duplicate protection using `recentlyCompleted` Map to prevent double-counting completed connections+- **Responsive Layout Improvements**+ - RULE LIST cards support container queries for adaptive layout switching+ - TOP DOMAINS cards expand to full width in single-column layout with more data items+- **UX Improvements**+ - Added skeleton loading screen for Backends list in Settings dialog++### Fixed++- **Surge collector short-lived connection traffic loss**: Fixed issue where traffic deltas for completed connections (status=Complete) were not being counted; uses `recentlyCompleted` Map to record final traffic and correctly calculate differences+- **Cleanup timer determinism**: Changed `recentlyCompleted` cleanup from `setInterval` to deterministic triggering tied to the polling cycle+- Fixed IPv6 validation logic using Node.js native `net.isIPv4/isIPv6`++### Refactored++- **Database Repository Pattern Refactoring** ๐๏ธ+ - Split the 5400+ line monolithic `db.ts` into 14 independent Repository files+ - Added `database/repositories/` directory with Repository Pattern architecture+ - Slimmed `db.ts` down to ~1000 lines, retaining only DDL, migrations, and one-line delegation methods+ - Extracted Repositories: `base`, `domain`, `ip`, `rule`, `proxy`, `device`, `country`, `timeseries`, `traffic-writer`, `config`, `backend`, `auth`, `surge`+ - `BaseRepository` encapsulates 13 shared utility methods including `parseMinuteRange`, `expandShortChainsForRules`, etc.+- **Code cleanup** (~140 lines removed)+ - Removed unused `parseRule` function, duplicate `buildGatewayHeaders`/`getGatewayBaseUrl`+ - Cleaned up debug `console.log` statements, unused `sleep()`, `DailyStats` import+ - Removed unused `EXTENDED_RETENTION`/`MINIMAL_RETENTION` constants++### Technical Details++- Surge collector uses `/v1/policy_groups/select` endpoint for policy group details+- `BackendRepository` now includes `type: 'clash' | 'surge'` field across create, query, and update flows+- Cleaned up debug code in `/api/gateway/proxies`+## [1.2.6] - 2026-02-13### Securitydiff --git a/CHANGELOG.md b/CHANGELOG.mdindex 00f9f26..1c59ce7 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,6 +7,48 @@## [ๆชๅๅธ]+## [1.2.7] - 2026-02-14++### ๆฐๅข++- **Surge ๅ็ซฏๆฏๆ** ๐+ - ๅฎๅ จๆฏๆ Surge HTTP REST API ๆฐๆฎ้้+ - ๆฏๆ่งๅ้พๅฏ่งๅๅฑ็คบ๏ผRule Chain Flow๏ผ+ - ๆฏๆไปฃ็่็นๅๅธๅพใๅๅ็ป่ฎก็ญๅฎๆดๅ่ฝ+ - ๆบ่ฝ็ญ็ฅ็ผๅญ็ณป็ป๏ผๅๅฐๅๆญฅ Surge ็ญ็ฅ้ ็ฝฎ+ - ่ชๅจ้่ฏๆบๅถ๏ผAPI ่ฏทๆฑๅคฑ่ดฅๆถ้็จๆๆฐ้้ฟ็ญ็ฅ+ - ๅ้ๅค่ฎก็ฎไฟๆค๏ผ้่ฟ `recentlyCompleted` Map ้ฒๆญขๅทฒๅฎๆ่ฟๆฅ่ขซ้ๅค่ฎก็ฎ+- **ๅๅบๅผๅธๅฑไผๅ**+ - RULE LIST ๅก็ๆฏๆๅฎนๅจๆฅ่ฏข่ช้ๅบ๏ผ็ญ็ช็ฉบ้ด่ชๅจๅๆขๅ็ดๅธๅฑ+ - TOP DOMAINS ๅก็ๅจๅๅๅธๅฑๆถ่ชๅจๆๆปกๅฎฝๅบฆๅนถๆพ็คบๆดๅคๆฐๆฎ+- **็จๆทไฝ้ชๆน่ฟ**+ - Settings ๅฏน่ฏๆกๆฐๅข Backends ๅ่กจ้ชจๆถๅฑ๏ผ่งฃๅณ้ฆๆฌกๅ ่ฝฝ็ฝๅฑ้ฎ้ข++### ไฟฎๅค++- **Surge ้้ๅจ็ญ่ฟๆฅๆต้ไธขๅคฑ**๏ผไฟฎๅคๅทฒๅฎๆ่ฟๆฅ๏ผstatus=Complete๏ผ็ๆต้ๅข้ๆช่ขซ่ฎกๅ ฅ็้ฎ้ข๏ผ้่ฟ `recentlyCompleted` Map ่ฎฐๅฝๆ็ปๆต้ๅนถๆญฃ็กฎ่ฎก็ฎๅทฎๅผ+- **ๆธ ็ๅฎๆถๅจ็กฎๅฎๆง**๏ผๅฐ `recentlyCompleted` ็ๆธ ็ไป `setInterval` ๆนไธบไธ่ฝฎ่ฏขๅจๆ็ปๅฎ็็กฎๅฎๆง่งฆๅ+- ไฟฎๅค IPv6 ้ช่ฏ้ป่พ๏ผไฝฟ็จ Node.js ๅ ็ฝฎ `net.isIPv4/isIPv6`++### ้ๆ++- **ๆฐๆฎๅบ Repository ๆจกๅผ้ๆ** ๐๏ธ+ - ๅฐ 5400+ ่ก็ๅไฝ `db.ts` ๆๅไธบ 14 ไธช็ฌ็ซ Repository ๆไปถ+ - ๆฐๅข `database/repositories/` ็ฎๅฝ๏ผ้็จ Repository Pattern ๆถๆ+ - `db.ts` ็ฆ่บซ่ณ ~1000 ่ก๏ผไป ไฟ็ DDLใ่ฟ็งป้ป่พๅไธ่กๅงๆๆนๆณ+ - ๆๅ็ Repository๏ผ`base`ใ`domain`ใ`ip`ใ`rule`ใ`proxy`ใ`device`ใ`country`ใ`timeseries`ใ`traffic-writer`ใ`config`ใ`backend`ใ`auth`ใ`surge`+ - `BaseRepository` ๅฐ่ฃ ไบ `parseMinuteRange`ใ`expandShortChainsForRules` ็ญ 13 ไธชๅ ฑไบซๅทฅๅ ทๆนๆณ+- **ไปฃ็ ๆธ ็**๏ผ~140 ่ก๏ผ+ - ็งป้คๆชไฝฟ็จ็ `parseRule` ๅฝๆฐใ้ๅค็ `buildGatewayHeaders`/`getGatewayBaseUrl`+ - ๆธ ็่ฐ่ฏ `console.log`ใๆชไฝฟ็จ็ `sleep()`ใ`DailyStats` ๅฏผๅ ฅ+ - ็งป้คๆชไฝฟ็จ็ `EXTENDED_RETENTION`/`MINIMAL_RETENTION` ๅธธ้++### ๆๆฏ็ป่++- Surge ้้ๅจไฝฟ็จ `/v1/policy_groups/select` ็ซฏ็น่ทๅ็ญ็ฅ็ป่ฏฆๆ+- `BackendRepository` ๆฐๅข `type: 'clash' | 'surge'` ๅญๆฎต๏ผ่ดฏ็ฉฟๅๅปบใๆฅ่ฏขใๆดๆฐๅ จ้พ่ทฏ+- ๆธ ็ `/api/gateway/proxies` ไธญ็่ฐ่ฏไปฃ็ +## [1.2.6] - 2026-02-13### ๅฎๅ จdiff --git a/README.en.md b/README.en.mdindex 2eef13f..0b88528 100644--- a/README.en.md+++ b/README.en.md@@ -190,10 +190,13 @@ Open <http://localhost:3000> to configure.+### Connect Clash / Mihomo+1. Open <http://localhost:3000>2. The **Gateway Configuration** dialog will appear on first visit3. Fill in your network gateway (e.g., OpenClash) connection info:- **Name**: Custom name (e.g., "Home Gateway")+ - **Type**: Select `Clash / Mihomo`- **Host**: Gateway backend address (e.g., `192.168.101.1`)- **Port**: Gateway backend port (e.g., `9090`)- **Token**: Fill if Secret is configured, otherwise leave empty@@ -202,6 +205,44 @@ Open <http://localhost:3000> to configure.> ๐ก **Get Gateway Address**: Go to your gateway control panel (e.g., OpenClash) โ Enable "External Control" โ Copy API address+### Connect Surge++++Neko Master supports connecting to Surge gateways for complete rule chain visualization and traffic analysis.++#### 1. Enable Surge HTTP API++Enable HTTP remote API in your Surge configuration:++```ini+[General]+http-api = 127.0.0.1:9091+http-api-tls = false+http-api-web-dashboard = true+```++Or configure via Surge's graphical interface:++- **HTTP Remote API**: `Settings` โ `General` โ `HTTP Remote API`+- **Port**: Default `9091`+- **Authentication**: Recommended to set a password for enhanced security++#### 2. Add Surge Backend in Neko Master++1. Open Neko Master settings dialog+2. Click "Add Backend"+3. Fill in the connection info:+ - **Name**: Custom name (e.g., "Surge Home")+ - **Type**: Select `Surge`+ - **Host**: IP address where Surge is running (e.g., `192.168.1.1` or `127.0.0.1`)+ - **Port**: HTTP API port (default `9091`)+ - **Token**: HTTP API password (if configured)+4. Click "Test Connection" to verify the configuration+5. Save the configuration++> ๐ก **Note**: Surge uses HTTP polling to fetch data (compared to Clash's WebSocket real-time stream), with a data refresh delay of approximately 2 seconds.+## ๐ง Port Conflict ResolutionIf you see "port already in use" error, here are the solutions:diff --git a/README.md b/README.mdindex 51fa69b..261ab08 100644--- a/README.md+++ b/README.md@@ -182,10 +182,13 @@ pnpm dev+### ๆฅๅ ฅ Clash / Mihomo+1. ๆๅผ <http://localhost:3000>2. ้ฆๆฌก่ฎฟ้ฎไผๅผนๅบ**็ฝๅ ณ้ ็ฝฎ**ๅฏน่ฏๆก3. ๅกซๅ็ฝ็ป็ฝๅ ณ๏ผๅฆ OpenClash๏ผ่ฟๆฅไฟกๆฏ๏ผ- **ๅ็งฐ**: ่ชๅฎไนๅ็งฐ๏ผๅฆ "Home Gateway"๏ผ+ - **็ฑปๅ**: ้ๆฉ `Clash / Mihomo`- **ๅฐๅ**: ็ฝๅ ณๅ็ซฏๅฐๅ๏ผๅฆ `192.168.101.1`๏ผ- **็ซฏๅฃ**: ็ฝๅ ณๅ็ซฏ็ซฏๅฃ๏ผๅฆ `9090`๏ผ- **Token**: ๅฆๆ้ ็ฝฎไบ Secret ๅๅกซๅ๏ผๅฆๅ็็ฉบ@@ -194,6 +197,43 @@ pnpm dev> ๐ก **่ทๅ็ฝๅ ณๅฐๅ**: ่ฟๅ ฅ็ฝๅ ณๆงๅถ้ขๆฟ๏ผๅฆ OpenClash๏ผ โ ๆๅผใๅค้จๆงๅถใโ ๅคๅถ API ๅฐๅ+### ๆฅๅ ฅ Surge++++Neko Master ๆฏๆๆฅๅ ฅ Surge ็ฝๅ ณ๏ผๅฎ็ฐๅฎๆด็่งๅ้พๅฏ่งๅๅๆต้ๅๆใ++#### 1. ๅผๅฏ Surge HTTP API++ๅจ Surge ้ ็ฝฎไธญๅฏ็จ HTTP ่ฟ็จ API๏ผ++```ini+[General]+http-api = 127.0.0.1:9091+http-api-tls = false+http-api-web-dashboard = true+```++ๆ่ ้่ฟ Surge ็ๅพๅฝข็้ข้ ็ฝฎ๏ผ+- **HTTP ่ฟ็จ API**: `่ฎพ็ฝฎ` โ `ๅธธ่ง` โ `HTTP ่ฟ็จ API`+- **็ซฏๅฃ**: ้ป่ฎค `9091`+- **่ฎค่ฏ**: ๅปบ่ฎฎ่ฎพ็ฝฎๅฏ็ ๅขๅผบๅฎๅ จๆง++#### 2. ๅจ Neko Master ไธญๆทปๅ Surge ๅ็ซฏ++1. ๆๅผ Neko Master ่ฎพ็ฝฎๅฏน่ฏๆก+2. ็นๅปใๆทปๅ ๅ็ซฏใ+3. ๅกซๅ่ฟๆฅไฟกๆฏ๏ผ+ - **ๅ็งฐ**: ่ชๅฎไนๅ็งฐ๏ผๅฆ "Surge Home"๏ผ+ - **็ฑปๅ**: ้ๆฉ `Surge`+ - **ๅฐๅ**: Surge ่ฟ่ก็ IP ๅฐๅ๏ผๅฆ `192.168.1.1` ๆ `127.0.0.1`๏ผ+ - **็ซฏๅฃ**: HTTP API ็ซฏๅฃ๏ผ้ป่ฎค `9091`๏ผ+ - **Token**: HTTP API ๅฏ็ ๏ผๅฆๆ่ฎพ็ฝฎไบ็่ฏ๏ผ+4. ็นๅปใๆต่ฏ่ฟๆฅใ็กฎ่ฎค้ ็ฝฎๆญฃ็กฎ+5. ไฟๅญ้ ็ฝฎ++> ๐ก **ๆณจๆ**: Surge ไฝฟ็จ HTTP ่ฝฎ่ฏขๆนๅผ่ทๅๆฐๆฎ๏ผ็ธๆฏ Clash ็ WebSocket ๅฎๆถๆต๏ผ๏ผๆฐๆฎๅทๆฐๅปถ่ฟ็บฆ 2 ็งใ+## ๐ง ็ซฏๅฃๅฒ็ช่งฃๅณๅฆๆ็ๅฐ้่ฏฏๆ็คบ็ซฏๅฃๅทฒ่ขซๅ ็จ๏ผๆไปฅไธๅ ็ง่งฃๅณๆนๆก๏ผdiff --git a/apps/collector/src/app.ts b/apps/collector/src/app.tsindex 2dcf9e9..d349dfb 100644--- a/apps/collector/src/app.ts+++ b/apps/collector/src/app.ts@@ -9,6 +9,8 @@ import cors from '@fastify/cors';import cookie from '@fastify/cookie';import type { StatsDatabase } from './db.js';import type { RealtimeStore } from './realtime.js';+import { buildGatewayHeaders, getGatewayBaseUrl, parseSurgeRule } from '@neko-master/shared';+import { SurgePolicySyncService } from './modules/surge/surge-policy-sync.js';// Import modulesimport { BackendService, backendController } from './modules/backend/index.js';@@ -28,10 +30,11 @@ export interface AppOptions {db: StatsDatabase;realtimeStore: RealtimeStore;logger?: boolean;+ policySyncService?: SurgePolicySyncService;}export async function createApp(options: AppOptions) {- const { port, db, realtimeStore, logger = false } = options;+ const { port, db, realtimeStore, logger = false, policySyncService } = options;// Create Fastify instanceconst app = Fastify({ logger });@@ -63,13 +66,89 @@ export async function createApp(options: AppOptions) {return statsService.resolveBackendId(backendId);};- const getGatewayBaseUrl = (url: string): string => {- return url- .replace(/^ws:\/\//, 'http://')- .replace(/^wss:\/\//, 'https://')- .replace(/\/connections\/?$/, '');+ // ...++ // Helper to get headers for backend requests+ const getHeaders = (backend: { type: 'clash' | 'surge'; token: string }) => {+ return buildGatewayHeaders(backend);};+ // Compatibility routes: Gateway APIs+ app.get('/api/gateway/proxies', async (request, reply) => {+ const backendId = getBackendIdFromQuery(request.query as Record<string, unknown>);+ if (backendId === null) {+ return reply.status(404).send({ error: 'No backend specified or active' });+ }++ const backend = db.getBackend(backendId);+ if (!backend) {+ return reply.status(404).send({ error: 'Backend not found' });+ }++ const gatewayBaseUrl = getGatewayBaseUrl(backend.url);+ const isSurge = backend.type === 'surge';+ const headers = getHeaders(backend);++ try {+ if (isSurge) {+ // Surge: Get policies list and details+ const res = await fetch(`${gatewayBaseUrl}/v1/policies`, { headers });+ if (!res.ok) {+ return reply.status(res.status).send({ error: `Surge API error: ${res.status}` });+ }++ const data = await res.json() as { proxies: string[]; 'policy-groups': string[] };+ const proxies: Record<string, { name: string; type: string; now?: string }> = {};++ // Get current selection for each policy group+ const policyGroups = data['policy-groups'] || [];+ const groupDetails = await Promise.allSettled(+ policyGroups.map(async (groupName: string) => {+ try {+ const detailRes = await fetch(+ `${gatewayBaseUrl}/v1/policies/${encodeURIComponent(groupName)}`,+ { headers, signal: AbortSignal.timeout(5000) }+ );+ if (!detailRes.ok) return { groupName, now: null };+ const detail = await detailRes.json() as { policy?: string };+ return { groupName, now: detail.policy || null };+ } catch {+ return { groupName, now: null };+ }+ })+ );++ for (const result of groupDetails) {+ if (result.status === 'fulfilled') {+ const { groupName, now } = result.value;+ proxies[groupName] = { name: groupName, type: 'Selector', now: now || '' };+ }+ }++ // Add leaf proxies+ if (data.proxies) {+ for (const name of data.proxies) {+ proxies[name] = { name, type: 'Unknown' };+ }+ }++ return { proxies };+ } else {+ // Clash/OpenClash: Direct proxy to /proxies endpoint+ const res = await fetch(`${gatewayBaseUrl}/proxies`, {+ headers: { ...headers, 'Content-Type': 'application/json' }+ });+ if (!res.ok) {+ return reply.status(res.status).send({ error: `Gateway API error: ${res.status}` });+ }+ return res.json();+ }+ } catch (error: unknown) {+ const message = error instanceof Error ? error.message : 'Failed to reach Gateway API';+ return reply.status(502).send({ error: message });+ }+ });+// Health check endpoint (not part of any module)app.get('/health', async () => ({ status: 'ok' }));@@ -171,7 +250,10 @@ export async function createApp(options: AppOptions) {// Compatibility routes: Gateway APIsapp.get('/api/gateway/providers/proxies', async (request, reply) => {- const backendId = getBackendIdFromQuery(request.query as Record<string, unknown>);+ const query = request.query as Record<string, unknown>;+ const backendId = getBackendIdFromQuery(query);+ const forceRefresh = query.refresh === 'true';+if (backendId === null) {return reply.status(404).send({ error: 'No backend specified or active' });}@@ -182,23 +264,161 @@ export async function createApp(options: AppOptions) {}const gatewayBaseUrl = getGatewayBaseUrl(backend.url);- const headers: Record<string, string> = { 'Content-Type': 'application/json' };- if (backend.token) {- headers.Authorization = `Bearer ${backend.token}`;- }+ const isSurge = backend.type === 'surge';+ const headers = getHeaders(backend);try {- const res = await fetch(`${gatewayBaseUrl}/providers/proxies`, { headers });- if (!res.ok) {- return reply.status(res.status).send({ error: `Gateway API error: ${res.status}` });+ if (isSurge) {+ // Build response from cache or fetch directly+ const providers: Record<string, { proxies: { name: string; type: string; now?: string }[] }> = {};+ let cacheStatus = policySyncService?.getCacheStatus(backendId);++ // Try to use cache first+ if (cacheStatus?.cached && !forceRefresh) {+ const cachedPolicies = db.getSurgePolicyCache(backendId);+ for (const policy of cachedPolicies) {+ if (policy.selectedPolicy) {+ providers[policy.policyGroup] = {+ proxies: [{ name: policy.policyGroup, type: policy.policyType, now: policy.selectedPolicy }]+ };+ }+ }+ }++ // If no cache or force refresh, fetch directly from Surge+ if (Object.keys(providers).length === 0 || forceRefresh) {+ try {+ const res = await fetch(`${gatewayBaseUrl}/v1/policies`, {+ headers,+ signal: AbortSignal.timeout(10000)+ });++ if (!res.ok) {+ throw new Error(`Surge API error: ${res.status}`);+ }++ const data = await res.json() as {+ proxies: string[];+ 'policy-groups': string[];+ };++ const policyGroups = data['policy-groups'] || [];++ // Fetch details for each policy group+ // Surge uses /v1/policy_groups/select?group_name=xxx endpoint+ const groupDetails = await Promise.allSettled(+ policyGroups.map(async (groupName: string) => {+ try {+ const detailRes = await fetch(+ `${gatewayBaseUrl}/v1/policy_groups/select?group_name=${encodeURIComponent(groupName)}`,+ { headers, signal: AbortSignal.timeout(5000) }+ );+ if (!detailRes.ok) return null;+ const detail = await detailRes.json() as { policy?: string; type?: string };+ return {+ name: groupName,+ now: detail.policy || '',+ type: detail.type || 'Select'+ };+ } catch {+ return null;+ }+ })+ );++ // Build providers from fetched data+ let successCount = 0;+ for (const result of groupDetails) {+ if (result.status === 'fulfilled' && result.value && result.value.now) {+ providers[result.value.name] = {+ proxies: [{+ name: result.value.name,+ type: result.value.type,+ now: result.value.now+ }]+ };+ successCount++;+ }+ }++ // Add standalone proxies+ if (data.proxies?.length > 0) {+ providers['default'] = {+ proxies: data.proxies.map(name => ({ name, type: 'Unknown' }))+ };+ }++ // Also update cache in background+ if (policySyncService) {+ policySyncService.syncNow(backendId, gatewayBaseUrl, backend.token || undefined)+ .catch(err => console.error(`[Gateway] Background sync failed:`, err.message));+ }++ } catch (error) {+ console.error(`[Gateway] Failed to fetch from Surge:`, error);+ if (Object.keys(providers).length === 0) {+ return reply.status(502).send({+ error: 'Failed to fetch policies',+ message: error instanceof Error ? error.message : 'Unknown error'+ });+ }+ }+ }++ return {+ providers,+ _cache: cacheStatus ? {+ cached: cacheStatus.cached,+ lastUpdate: cacheStatus.lastUpdate,+ policyCount: cacheStatus.policyCount,+ } : undefined+ };+ } else {+ // Clash/OpenClash: direct proxy+ const res = await fetch(`${gatewayBaseUrl}/providers/proxies`, {+ headers: { ...headers, 'Content-Type': 'application/json' }+ });+ if (!res.ok) {+ return reply.status(res.status).send({ error: `Gateway API error: ${res.status}` });+ }+ return res.json();}- return res.json();} catch (error: unknown) {const message = error instanceof Error ? error.message : 'Failed to reach Gateway API';return reply.status(502).send({ error: message });}});+ // Manual refresh endpoint for Surge policies+ app.post('/api/gateway/providers/proxies/refresh', async (request, reply) => {+ const backendId = getBackendIdFromQuery(request.query as Record<string, unknown>);+ if (backendId === null) {+ return reply.status(404).send({ error: 'No backend specified or active' });+ }++ const backend = db.getBackend(backendId);+ if (!backend || backend.type !== 'surge') {+ return reply.status(400).send({ error: 'Only Surge backend supports this operation' });+ }++ if (!policySyncService) {+ return reply.status(503).send({ error: 'Policy sync service not available' });+ }++ const gatewayBaseUrl = getGatewayBaseUrl(backend.url);+ const result = await policySyncService.syncNow(+ backendId,+ gatewayBaseUrl,+ backend.token || undefined+ );++ return {+ success: result.success,+ message: result.message,+ updated: result.updated,+ };+ });+app.get('/api/gateway/rules', async (request, reply) => {const backendId = getBackendIdFromQuery(request.query as Record<string, unknown>);if (backendId === null) {@@ -211,17 +431,47 @@ export async function createApp(options: AppOptions) {}const gatewayBaseUrl = getGatewayBaseUrl(backend.url);- const headers: Record<string, string> = { 'Content-Type': 'application/json' };- if (backend.token) {- headers.Authorization = `Bearer ${backend.token}`;- }+ const isSurge = backend.type === 'surge';+ const headers = getHeaders(backend);try {- const res = await fetch(`${gatewayBaseUrl}/rules`, { headers });- if (!res.ok) {- return reply.status(res.status).send({ error: `Gateway API error: ${res.status}` });+ if (isSurge) {+ // Surge uses /v1/rules endpoint+ const res = await fetch(`${gatewayBaseUrl}/v1/rules`, { headers });+ if (!res.ok) {+ return reply.status(res.status).send({ error: `Surge API error: ${res.status}` });+ }++ const data = await res.json() as { rules: string[]; 'available-policies': string[] };++ // Parse Surge rules to standard format+ const parsedRules = data.rules+ .map(raw => {+ const parsed = parseSurgeRule(raw);+ return parsed ? { type: parsed.type, payload: parsed.payload, policy: parsed.policy, raw } : null;+ })+ .filter((r): r is NonNullable<typeof r> => r !== null);++ return {+ rules: parsedRules.map(r => ({+ type: r.type,+ payload: r.payload,+ proxy: r.policy,+ size: 0,+ })),+ _source: 'surge' as const,+ _availablePolicies: data['available-policies'],+ };+ } else {+ // Clash/OpenClash uses /rules endpoint+ const res = await fetch(`${gatewayBaseUrl}/rules`, {+ headers: { ...headers, 'Content-Type': 'application/json' }+ });+ if (!res.ok) {+ return reply.status(res.status).send({ error: `Gateway API error: ${res.status}` });+ }+ return res.json();}- return res.json();} catch (error: unknown) {const message = error instanceof Error ? error.message : 'Failed to reach Gateway API';return reply.status(502).send({ error: message });@@ -280,6 +530,9 @@ export async function createApp(options: AppOptions) {await app.listen({ port, host: '0.0.0.0' });console.log(`[API] Server running at http://localhost:${port}`);+ // Start automatic health checks for upstream gateways+ backendService.startHealthChecks();+return app;}@@ -288,11 +541,18 @@ export class APIServer {private db: StatsDatabase;private realtimeStore: RealtimeStore;private port: number;-- constructor(port: number, db: StatsDatabase, realtimeStore: RealtimeStore) {+ private policySyncService?: SurgePolicySyncService;++ constructor(+ port: number,+ db: StatsDatabase,+ realtimeStore: RealtimeStore,+ policySyncService?: SurgePolicySyncService+ ) {this.port = port;this.db = db;this.realtimeStore = realtimeStore;+ this.policySyncService = policySyncService;}async start() {@@ -300,6 +560,7 @@ export class APIServer {port: this.port,db: this.db,realtimeStore: this.realtimeStore,+ policySyncService: this.policySyncService,logger: false,});return this.app;diff --git a/apps/collector/src/data-retention.ts b/apps/collector/src/data-retention.tsindex 0308ff7..fd848d9 100644--- a/apps/collector/src/data-retention.ts+++ b/apps/collector/src/data-retention.ts@@ -28,19 +28,3 @@ export const DEFAULT_RETENTION: RetentionConfig = {autoCleanup: true,cleanupInterval: 24, // Run cleanup every 24 hours};--// For users who want longer retention (higher storage cost)-export const EXTENDED_RETENTION: RetentionConfig = {- connectionLogsDays: 30, // Keep 30 days of raw logs- hourlyStatsDays: 90, // Keep 90 days of hourly data- autoCleanup: true,- cleanupInterval: 24,-};--// Minimal retention for low-resource environments-export const MINIMAL_RETENTION: RetentionConfig = {- connectionLogsDays: 3, // Keep 3 days of raw logs- hourlyStatsDays: 14, // Keep 14 days of hourly data- autoCleanup: true,- cleanupInterval: 12, // Cleanup more frequently-};diff --git a/apps/collector/src/database/repositories/auth.repository.ts b/apps/collector/src/database/repositories/auth.repository.tsnew file mode 100644index 0000000..c7f9171--- /dev/null+++ b/apps/collector/src/database/repositories/auth.repository.ts@@ -0,0 +1,43 @@+/**+ * Auth Repository+ *+ * Handles authentication configuration storage.+ */+import type Database from 'better-sqlite3';+import { BaseRepository } from './base.repository.js';++export class AuthRepository extends BaseRepository {+ constructor(db: Database.Database) {+ super(db);+ }++ getAuthConfig(): { enabled: boolean; tokenHash: string | null; updatedAt: string } {+ const enabledStmt = this.db.prepare(`SELECT value, updated_at FROM auth_config WHERE key = 'enabled'`);+ const tokenStmt = this.db.prepare(`SELECT value, updated_at FROM auth_config WHERE key = 'token_hash'`);++ const enabledRow = enabledStmt.get() as { value: string; updated_at: string } | undefined;+ const tokenRow = tokenStmt.get() as { value: string; updated_at: string } | undefined;++ return {+ enabled: enabledRow?.value === '1',+ tokenHash: tokenRow?.value || null,+ updatedAt: tokenRow?.updated_at || enabledRow?.updated_at || new Date().toISOString(),+ };+ }++ updateAuthConfig(updates: { enabled?: boolean; tokenHash?: string | null }): void {+ if (updates.enabled !== undefined) {+ this.db.prepare(`+ INSERT INTO auth_config (key, value) VALUES ('enabled', ?)+ ON CONFLICT(key) DO UPDATE SET value = excluded.value, updated_at = CURRENT_TIMESTAMP+ `).run(updates.enabled ? '1' : '0');+ }++ if (updates.tokenHash !== undefined) {+ this.db.prepare(`+ INSERT INTO auth_config (key, value) VALUES ('token_hash', ?)+ ON CONFLICT(key) DO UPDATE SET value = excluded.value, updated_at = CURRENT_TIMESTAMP+ `).run(updates.tokenHash || '');+ }+ }+}diff --git a/apps/collector/src/database/repositories/backend.repository.ts b/apps/collector/src/database/repositories/backend.repository.tsindex e298250..4f55a96 100644--- a/apps/collector/src/database/repositories/backend.repository.ts+++ b/apps/collector/src/database/repositories/backend.repository.ts@@ -13,6 +13,7 @@ export interface BackendConfig {name: string;url: string;token: string;+ type: 'clash' | 'surge';enabled: boolean;is_active: boolean;listening: boolean;@@ -30,12 +31,12 @@ export class BackendRepository {/*** Create a new backend configuration*/- createBackend(backend: { name: string; url: string; token?: string }): number {+ createBackend(backend: { name: string; url: string; token?: string; type?: 'clash' | 'surge' }): number {const stmt = this.db.prepare(`- INSERT INTO backend_configs (name, url, token, enabled, is_active, listening)- VALUES (?, ?, ?, 1, 0, 1)+ INSERT INTO backend_configs (name, url, token, type, enabled, is_active, listening)+ VALUES (?, ?, ?, ?, 1, 0, 1)`);- const result = stmt.run(backend.name, backend.url, backend.token || '');+ const result = stmt.run(backend.name, backend.url, backend.token || '', backend.type || 'clash');return Number(result.lastInsertRowid);}@@ -44,7 +45,7 @@ export class BackendRepository {*/getAllBackends(): BackendConfig[] {const stmt = this.db.prepare(`- SELECT id, name, url, token, enabled, is_active, listening, created_at, updated_at+ SELECT id, name, url, token, type, enabled, is_active, listening, created_at, updated_atFROM backend_configsORDER BY created_at ASC`);@@ -56,7 +57,7 @@ export class BackendRepository {*/getBackend(id: number): BackendConfig | undefined {const stmt = this.db.prepare(`- SELECT id, name, url, token, enabled, is_active, listening, created_at, updated_at+ SELECT id, name, url, token, type, enabled, is_active, listening, created_at, updated_atFROM backend_configsWHERE id = ?`);@@ -68,7 +69,7 @@ export class BackendRepository {*/getActiveBackend(): BackendConfig | undefined {const stmt = this.db.prepare(`- SELECT id, name, url, token, enabled, is_active, listening, created_at, updated_at+ SELECT id, name, url, token, type, enabled, is_active, listening, created_at, updated_atFROM backend_configsWHERE is_active = 1LIMIT 1@@ -81,7 +82,7 @@ export class BackendRepository {*/getListeningBackends(): BackendConfig[] {const stmt = this.db.prepare(`- SELECT id, name, url, token, enabled, is_active, listening, created_at, updated_at+ SELECT id, name, url, token, type, enabled, is_active, listening, created_at, updated_atFROM backend_configsWHERE listening = 1 AND enabled = 1`);@@ -107,6 +108,10 @@ export class BackendRepository {sets.push('token = ?');values.push(updates.token);}+ if (updates.type !== undefined) {+ sets.push('type = ?');+ values.push(updates.type);+ }if (updates.enabled !== undefined) {sets.push('enabled = ?');values.push(updates.enabled ? 1 : 0);diff --git a/apps/collector/src/database/repositories/base.repository.ts b/apps/collector/src/database/repositories/base.repository.tsindex e402c40..32cb95f 100644--- a/apps/collector/src/database/repositories/base.repository.ts+++ b/apps/collector/src/database/repositories/base.repository.ts@@ -5,6 +5,7 @@* All specific repositories should extend this class.*/import type Database from 'better-sqlite3';+import type { ProxyStats } from '@neko-master/shared';export abstract class BaseRepository {protected db: Database.Database;@@ -153,6 +154,131 @@ export abstract class BaseRepository {return out;}+ /**+ * Aggregate proxy stats by first hop in chain+ */+ protected aggregateProxyStatsByFirstHop(rows: ProxyStats[]): ProxyStats[] {+ const merged = new Map<string, ProxyStats>();++ for (const row of rows) {+ const hop = this.getChainFirstHop(row.chain || "") || "DIRECT";+ const existing = merged.get(hop);+ if (existing) {+ existing.totalUpload += row.totalUpload;+ existing.totalDownload += row.totalDownload;+ existing.totalConnections += row.totalConnections;+ if (row.lastSeen > existing.lastSeen) {+ existing.lastSeen = row.lastSeen;+ }+ } else {+ merged.set(hop, {+ chain: hop,+ totalUpload: row.totalUpload,+ totalDownload: row.totalDownload,+ totalConnections: row.totalConnections,+ lastSeen: row.lastSeen,+ });+ }+ }++ return Array.from(merged.values()).sort(+ (a, b) => (b.totalDownload + b.totalUpload) - (a.totalDownload + a.totalUpload),+ );+ }++ /**+ * Remap range rows from minute_dim_stats to full chains using baseline data+ */+ protected remapRangeRowsToFullChains(+ rangeRows: Array<{ rule: string; chain: string; totalUpload: number; totalDownload: number; totalConnections: number }>,+ baselineRows: Array<{ rule: string; chain: string; totalUpload: number; totalDownload: number; totalConnections: number }>,+ ): Array<{ rule: string; chain: string; totalUpload: number; totalDownload: number; totalConnections: number }> {+ if (rangeRows.length === 0) return [];+ if (baselineRows.length === 0) return rangeRows;++ const baselineByRuleHop = new Map<string, typeof baselineRows>();+ const baselineByNormalizedRuleHop = new Map<string, typeof baselineRows>();+ for (const row of baselineRows) {+ const hop = this.getChainFirstHop(row.chain);+ const key = `${row.rule}|||${hop}`;+ const list = baselineByRuleHop.get(key);+ if (list) { list.push(row); } else { baselineByRuleHop.set(key, [row]); }++ const normalizedRule = this.normalizeFlowLabel(row.rule);+ if (!normalizedRule) continue;+ const normalizedKey = `${normalizedRule}|||${hop}`;+ const normalizedList = baselineByNormalizedRuleHop.get(normalizedKey);+ if (normalizedList) { normalizedList.push(row); } else { baselineByNormalizedRuleHop.set(normalizedKey, [row]); }+ }++ const mapped: typeof rangeRows = [];+ for (const row of rangeRows) {+ const parts = this.splitChainParts(row.chain);+ const alreadyFull = parts.length > 1 && this.findRuleIndexInChain(parts, row.rule) !== -1;+ if (alreadyFull) { mapped.push(row); continue; }++ const hop = this.getChainFirstHop(row.chain);+ const normalizedRule = this.normalizeFlowLabel(row.rule);+ const candidates = baselineByRuleHop.get(`${row.rule}|||${hop}`) ||+ (normalizedRule ? baselineByNormalizedRuleHop.get(`${normalizedRule}|||${hop}`) : undefined);+ if (!candidates || candidates.length === 0) { mapped.push(row); continue; }++ if (candidates.length === 1) {+ mapped.push({ rule: row.rule, chain: candidates[0].chain, totalUpload: row.totalUpload, totalDownload: row.totalDownload, totalConnections: row.totalConnections });+ continue;+ }++ const weights = candidates.map(c => { const t = c.totalUpload + c.totalDownload; return t > 0 ? t : Math.max(1, c.totalConnections); });+ const uploadParts = this.allocateByWeights(row.totalUpload, weights);+ const downloadParts = this.allocateByWeights(row.totalDownload, weights);+ const connParts = this.allocateByWeights(row.totalConnections, weights);+ for (let i = 0; i < candidates.length; i++) {+ mapped.push({ rule: row.rule, chain: candidates[i].chain, totalUpload: uploadParts[i] || 0, totalDownload: downloadParts[i] || 0, totalConnections: connParts[i] || 0 });+ }+ }+ return mapped;+ }++ /**+ * Expand short chains for rules using rule_chain_traffic+ */+ protected expandShortChainsForRules(backendId: number, chains: string[], rules: string[]): string[] {+ const normalizedChains = this.uniqueNonEmpty(chains);+ if (normalizedChains.length === 0) return [];++ const shortChains = normalizedChains.filter((c) => !c.includes(">"));+ if (shortChains.length === 0) return normalizedChains;++ const normalizedRules = this.uniqueNonEmpty(rules);+ const whereParts: string[] = [];+ const params: Array<string | number> = [backendId];++ if (normalizedRules.length > 0) {+ const rulePlaceholders = normalizedRules.map(() => "?").join(", ");+ whereParts.push(`rule IN (${rulePlaceholders})`);+ params.push(...normalizedRules);+ }++ const chainMatchers: string[] = [];+ for (const chain of shortChains) {+ chainMatchers.push("(chain = ? OR chain LIKE ?)");+ params.push(chain, `${chain} > %`);+ }+ whereParts.push(`(${chainMatchers.join(" OR ")})`);++ const whereClause = whereParts.length > 0 ? `AND ${whereParts.join(" AND ")}` : "";+ const stmt = this.db.prepare(`+ SELECT DISTINCT chain FROM rule_chain_traffic WHERE backend_id = ? ${whereClause} LIMIT 500+ `);++ const rows = stmt.all(...params) as Array<{ chain: string }>;+ const expanded = this.uniqueNonEmpty(rows.map((r) => r.chain));+ if (expanded.length === 0) return normalizedChains;++ const fullInputChains = normalizedChains.filter((c) => c.includes(">"));+ return this.uniqueNonEmpty([...expanded, ...fullInputChains]);+ }+/*** Allocate a total value by weights*/diff --git a/apps/collector/src/database/repositories/config.repository.ts b/apps/collector/src/database/repositories/config.repository.tsnew file mode 100644index 0000000..07e651c--- /dev/null+++ b/apps/collector/src/database/repositories/config.repository.ts@@ -0,0 +1,215 @@+/**+ * Config Repository+ *+ * Handles database maintenance: retention configuration, data cleanup,+ * vacuum, and statistics about stored data.+ */+import type Database from 'better-sqlite3';+import * as fs from 'node:fs';+import { BaseRepository } from './base.repository.js';++interface DatabaseRetentionConfig {+ connectionLogsDays: number;+ hourlyStatsDays: number;+ autoCleanup: boolean;+}++export class ConfigRepository extends BaseRepository {+ private dbPath: string;++ constructor(db: Database.Database, dbPath: string) {+ super(db);+ this.dbPath = dbPath;+ }++ // Retention config+ getRetentionConfig(): DatabaseRetentionConfig {+ const connectionLogsDays = this.db.prepare(+ `SELECT value FROM app_config WHERE key = 'retention.connection_logs_days'`,+ ).get() as { value: string } | undefined;++ const hourlyStatsDays = this.db.prepare(+ `SELECT value FROM app_config WHERE key = 'retention.hourly_stats_days'`,+ ).get() as { value: string } | undefined;++ const autoCleanup = this.db.prepare(+ `SELECT value FROM app_config WHERE key = 'retention.auto_cleanup'`,+ ).get() as { value: string } | undefined;++ return {+ connectionLogsDays: parseInt(connectionLogsDays?.value || '7', 10),+ hourlyStatsDays: parseInt(hourlyStatsDays?.value || '30', 10),+ autoCleanup: autoCleanup?.value === '1',+ };+ }++ updateRetentionConfig(updates: {+ connectionLogsDays?: number;+ hourlyStatsDays?: number;+ autoCleanup?: boolean;+ }): DatabaseRetentionConfig {+ if (updates.connectionLogsDays !== undefined) {+ this.db.prepare(`+ INSERT INTO app_config (key, value) VALUES ('retention.connection_logs_days', ?)+ ON CONFLICT(key) DO UPDATE SET value = excluded.value, updated_at = CURRENT_TIMESTAMP+ `).run(updates.connectionLogsDays.toString());+ }+ if (updates.hourlyStatsDays !== undefined) {+ this.db.prepare(`+ INSERT INTO app_config (key, value) VALUES ('retention.hourly_stats_days', ?)+ ON CONFLICT(key) DO UPDATE SET value = excluded.value, updated_at = CURRENT_TIMESTAMP+ `).run(updates.hourlyStatsDays.toString());+ }+ if (updates.autoCleanup !== undefined) {+ this.db.prepare(`+ INSERT INTO app_config (key, value) VALUES ('retention.auto_cleanup', ?)+ ON CONFLICT(key) DO UPDATE SET value = excluded.value, updated_at = CURRENT_TIMESTAMP+ `).run(updates.autoCleanup ? '1' : '0');+ }+ return this.getRetentionConfig();+ }++ // Data cleanup+ cleanupOldData(backendId: number | null, days: number): {+ deletedConnections: number; deletedLogs: number; deletedDomains: number;+ deletedIPs: number; deletedProxies: number; deletedRules: number;+ } {+ let deletedConnections = 0;+ let deletedLogs = 0;+ let deletedDomains = 0;+ let deletedIPs = 0;+ let deletedProxies = 0;+ let deletedRules = 0;++ if (backendId !== null) {+ if (days === 0) {+ const minuteResult = this.db.prepare(`DELETE FROM minute_stats WHERE backend_id = ?`).run(backendId);+ deletedConnections = minuteResult.changes;+ deletedLogs = minuteResult.changes;+ this.db.prepare(`DELETE FROM minute_dim_stats WHERE backend_id = ?`).run(backendId);+ this.db.prepare(`DELETE FROM minute_country_stats WHERE backend_id = ?`).run(backendId);+ this.db.prepare(`DELETE FROM connection_logs WHERE backend_id = ?`).run(backendId);++ deletedDomains = this.db.prepare(`DELETE FROM domain_stats WHERE backend_id = ?`).run(backendId).changes;+ deletedIPs = this.db.prepare(`DELETE FROM ip_stats WHERE backend_id = ?`).run(backendId).changes;+ deletedProxies = this.db.prepare(`DELETE FROM proxy_stats WHERE backend_id = ?`).run(backendId).changes;+ deletedRules = this.db.prepare(`DELETE FROM rule_stats WHERE backend_id = ?`).run(backendId).changes;++ this.db.prepare(`DELETE FROM country_stats WHERE backend_id = ?`).run(backendId);+ this.db.prepare(`DELETE FROM rule_proxy_map WHERE backend_id = ?`).run(backendId);+ this.db.prepare(`DELETE FROM rule_chain_traffic WHERE backend_id = ?`).run(backendId);+ this.db.prepare(`DELETE FROM rule_domain_traffic WHERE backend_id = ?`).run(backendId);+ this.db.prepare(`DELETE FROM rule_ip_traffic WHERE backend_id = ?`).run(backendId);+ this.db.prepare(`DELETE FROM domain_proxy_stats WHERE backend_id = ?`).run(backendId);+ this.db.prepare(`DELETE FROM ip_proxy_stats WHERE backend_id = ?`).run(backendId);+ this.db.prepare(`DELETE FROM device_stats WHERE backend_id = ?`).run(backendId);+ this.db.prepare(`DELETE FROM device_domain_stats WHERE backend_id = ?`).run(backendId);+ this.db.prepare(`DELETE FROM device_ip_stats WHERE backend_id = ?`).run(backendId);+ this.db.prepare(`DELETE FROM hourly_stats WHERE backend_id = ?`).run(backendId);+ } else {+ const cutoff = new Date(Date.now() - days * 24 * 60 * 60 * 1000);+ const minuteCutoff = cutoff.toISOString().slice(0, 16) + ':00';+ const minuteResult = this.db.prepare(`DELETE FROM minute_stats WHERE backend_id = ? AND minute < ?`).run(backendId, minuteCutoff);+ deletedConnections = minuteResult.changes;+ deletedLogs = minuteResult.changes;+ this.db.prepare(`DELETE FROM minute_dim_stats WHERE backend_id = ? AND minute < ?`).run(backendId, minuteCutoff);+ this.db.prepare(`DELETE FROM minute_country_stats WHERE backend_id = ? AND minute < ?`).run(backendId, minuteCutoff);+ this.db.prepare(`DELETE FROM connection_logs WHERE backend_id = ? AND timestamp < ?`).run(backendId, cutoff.toISOString());+ }+ } else {+ if (days === 0) {+ const minuteResult = this.db.prepare(`DELETE FROM minute_stats`).run();+ deletedConnections = minuteResult.changes;+ deletedLogs = minuteResult.changes;+ this.db.prepare(`DELETE FROM minute_dim_stats`).run();+ this.db.prepare(`DELETE FROM minute_country_stats`).run();+ this.db.prepare(`DELETE FROM connection_logs`).run();++ deletedDomains = this.db.prepare(`DELETE FROM domain_stats`).run().changes;+ deletedIPs = this.db.prepare(`DELETE FROM ip_stats`).run().changes;+ deletedProxies = this.db.prepare(`DELETE FROM proxy_stats`).run().changes;+ deletedRules = this.db.prepare(`DELETE FROM rule_stats`).run().changes;++ this.db.prepare(`DELETE FROM country_stats`).run();+ this.db.prepare(`DELETE FROM rule_proxy_map`).run();+ this.db.prepare(`DELETE FROM rule_chain_traffic`).run();+ this.db.prepare(`DELETE FROM rule_domain_traffic`).run();+ this.db.prepare(`DELETE FROM rule_ip_traffic`).run();+ this.db.prepare(`DELETE FROM domain_proxy_stats`).run();+ this.db.prepare(`DELETE FROM ip_proxy_stats`).run();+ this.db.prepare(`DELETE FROM device_stats`).run();+ this.db.prepare(`DELETE FROM device_domain_stats`).run();+ this.db.prepare(`DELETE FROM device_ip_stats`).run();+ this.db.prepare(`DELETE FROM hourly_stats`).run();+ } else {+ const cutoff = new Date(Date.now() - days * 24 * 60 * 60 * 1000);+ const minuteCutoff = cutoff.toISOString().slice(0, 16) + ':00';+ const minuteResult = this.db.prepare(`DELETE FROM minute_stats WHERE minute < ?`).run(minuteCutoff);+ deletedConnections = minuteResult.changes;+ deletedLogs = minuteResult.changes;+ this.db.prepare(`DELETE FROM minute_dim_stats WHERE minute < ?`).run(minuteCutoff);+ this.db.prepare(`DELETE FROM minute_country_stats WHERE minute < ?`).run(minuteCutoff);+ this.db.prepare(`DELETE FROM connection_logs WHERE timestamp < ?`).run(cutoff.toISOString());+ }+ }++ if (days === 0) {+ this.vacuum();+ }++ return { deletedConnections, deletedLogs, deletedDomains, deletedIPs, deletedProxies, deletedRules };+ }++ cleanupASNCache(days: number): number {+ const cutoff = new Date(Date.now() - days * 24 * 60 * 60 * 1000).toISOString();+ return this.db.prepare(`DELETE FROM asn_cache WHERE queried_at < ?`).run(cutoff).changes;+ }++ getDatabaseSize(): number {+ try {+ return fs.statSync(this.dbPath).size;+ } catch {+ return 0;+ }+ }++ getConnectionLogsCount(backendId: number): number {+ return (this.db.prepare(`SELECT COUNT(*) as count FROM minute_stats WHERE backend_id = ?`).get(backendId) as { count: number }).count;+ }++ getTotalConnectionLogsCount(): number {+ return (this.db.prepare(`SELECT COUNT(*) as count FROM minute_stats`).get() as { count: number }).count;+ }++ vacuum(): void {+ this.db.exec('VACUUM');+ }++ deleteOldMinuteStats(cutoff: string): number {+ const minuteCutoff = cutoff.slice(0, 16) + ':00';+ this.db.prepare(`DELETE FROM minute_dim_stats WHERE minute < ?`).run(minuteCutoff);+ this.db.prepare(`DELETE FROM minute_country_stats WHERE minute < ?`).run(minuteCutoff);+ this.db.prepare(`DELETE FROM connection_logs WHERE timestamp < ?`).run(cutoff);+ return this.db.prepare(`DELETE FROM minute_stats WHERE minute < ?`).run(minuteCutoff).changes;+ }++ deleteOldConnectionLogs(cutoff: string): number {+ return this.deleteOldMinuteStats(cutoff);+ }++ deleteOldHourlyStats(cutoff: string): number {+ return this.db.prepare(`DELETE FROM hourly_stats WHERE hour < ?`).run(cutoff).changes;+ }++ getCleanupStats(): {+ connectionLogsCount: number; hourlyStatsCount: number;+ oldestConnectionLog: string | null; oldestHourlyStat: string | null;+ } {+ return {+ connectionLogsCount: (this.db.prepare('SELECT COUNT(*) as count FROM minute_stats').get() as { count: number }).count,+ hourlyStatsCount: (this.db.prepare('SELECT COUNT(*) as count FROM hourly_stats').get() as { count: number }).count,+ oldestConnectionLog: (this.db.prepare('SELECT MIN(minute) as ts FROM minute_stats').get() as { ts: string | null })?.ts || null,+ oldestHourlyStat: (this.db.prepare('SELECT MIN(hour) as hr FROM hourly_stats').get() as { hr: string | null })?.hr || null,+ };+ }+}diff --git a/apps/collector/src/database/repositories/country.repository.ts b/apps/collector/src/database/repositories/country.repository.tsnew file mode 100644index 0000000..8eb88e5--- /dev/null+++ b/apps/collector/src/database/repositories/country.repository.ts@@ -0,0 +1,135 @@+/**+ * Country Repository+ *+ * Handles country traffic statistics queries and updates.+ */+import type Database from 'better-sqlite3';+import { BaseRepository } from './base.repository.js';++export interface CountryStatsRow {+ country: string;+ countryName: string;+ continent: string;+ totalUpload: number;+ totalDownload: number;+ totalConnections: number;+}++export class CountryRepository extends BaseRepository {+ constructor(db: Database.Database) {+ super(db);+ }++ getCountryStats(backendId: number, limit = 50, start?: string, end?: string): CountryStatsRow[] {+ const range = this.parseMinuteRange(start, end);+ if (range) {+ const minuteCountryStmt = this.db.prepare(`+ SELECT country, MAX(country_name) as countryName, MAX(continent) as continent,+ SUM(upload) as totalUpload, SUM(download) as totalDownload, SUM(connections) as totalConnections+ FROM minute_country_stats+ WHERE backend_id = ? AND minute >= ? AND minute <= ?+ GROUP BY country+ ORDER BY (SUM(upload) + SUM(download)) DESC+ LIMIT ?+ `);+ const minuteCountryRows = minuteCountryStmt.all(backendId, range.startMinute, range.endMinute, limit) as CountryStatsRow[];+ if (minuteCountryRows.length > 0) return minuteCountryRows;++ // Fallback 1: derive from minute_dim_stats + geoip_cache+ const minuteDimFallbackStmt = this.db.prepare(`+ SELECT+ COALESCE(g.country, 'UNKNOWN') as country,+ COALESCE(MAX(g.country_name), 'Unknown') as countryName,+ COALESCE(MAX(g.continent), 'Unknown') as continent,+ SUM(m.upload) as totalUpload, SUM(m.download) as totalDownload, SUM(m.connections) as totalConnections+ FROM minute_dim_stats m+ LEFT JOIN geoip_cache g ON m.ip = g.ip+ WHERE m.backend_id = ? AND m.minute >= ? AND m.minute <= ? AND m.ip != ''+ GROUP BY COALESCE(g.country, 'UNKNOWN')+ ORDER BY (SUM(m.upload) + SUM(m.download)) DESC+ LIMIT ?+ `);+ const minuteDimFallbackRows = minuteDimFallbackStmt.all(backendId, range.startMinute, range.endMinute, limit) as CountryStatsRow[];+ if (minuteDimFallbackRows.length > 0) return minuteDimFallbackRows;++ // Fallback 2: aggregate from minute_stats as UNKNOWN+ const totalStmt = this.db.prepare(`+ SELECT COALESCE(SUM(upload), 0) as upload, COALESCE(SUM(download), 0) as download, COALESCE(SUM(connections), 0) as connections+ FROM minute_stats WHERE backend_id = ? AND minute >= ? AND minute <= ?+ `);+ const total = totalStmt.get(backendId, range.startMinute, range.endMinute) as { upload: number; download: number; connections: number };+ if (total.upload > 0 || total.download > 0 || total.connections > 0) {+ return [{ country: 'UNKNOWN', countryName: 'Unknown', continent: 'Unknown', totalUpload: total.upload, totalDownload: total.download, totalConnections: total.connections }];+ }+ return [];+ }++ const stmt = this.db.prepare(`+ SELECT country, country_name as countryName, continent,+ total_upload as totalUpload, total_download as totalDownload, total_connections as totalConnections+ FROM country_stats WHERE backend_id = ?+ ORDER BY (total_upload + total_download) DESC+ LIMIT ?+ `);+ return stmt.all(backendId, limit) as CountryStatsRow[];+ }++ updateCountryStats(backendId: number, country: string, countryName: string, continent: string, upload: number, download: number): void {+ const stmt = this.db.prepare(`+ INSERT INTO country_stats (backend_id, country, country_name, continent, total_upload, total_download, total_connections, last_seen)+ VALUES (?, ?, ?, ?, ?, ?, 1, CURRENT_TIMESTAMP)+ ON CONFLICT(backend_id, country) DO UPDATE SET+ total_upload = total_upload + ?, total_download = total_download + ?,+ total_connections = total_connections + 1, last_seen = CURRENT_TIMESTAMP+ `);+ stmt.run(backendId, country, countryName, continent, upload, download, upload, download);+ }++ batchUpdateCountryStats(backendId: number, results: Array<{+ country: string; countryName: string; continent: string;+ upload: number; download: number; timestampMs?: number;+ }>): void {+ if (results.length === 0) return;++ const cumulativeStmt = this.db.prepare(`+ INSERT INTO country_stats (backend_id, country, country_name, continent, total_upload, total_download, total_connections, last_seen)+ VALUES (?, ?, ?, ?, ?, ?, 1, CURRENT_TIMESTAMP)+ ON CONFLICT(backend_id, country) DO UPDATE SET+ total_upload = total_upload + ?, total_download = total_download + ?,+ total_connections = total_connections + 1, last_seen = CURRENT_TIMESTAMP+ `);++ const minuteStmt = this.db.prepare(`+ INSERT INTO minute_country_stats (backend_id, minute, country, country_name, continent, upload, download, connections)+ VALUES (@backendId, @minute, @country, @countryName, @continent, @upload, @download, @connections)+ ON CONFLICT(backend_id, minute, country) DO UPDATE SET+ upload = upload + @upload, download = download + @download, connections = connections + @connections+ `);++ const tx = this.db.transaction(() => {+ const minuteMap = new Map<string, {+ minute: string; country: string; countryName: string; continent: string;+ upload: number; download: number; connections: number;+ }>();++ for (const r of results) {+ cumulativeStmt.run(backendId, r.country, r.countryName, r.continent, r.upload, r.download, r.upload, r.download);+ const minute = this.toMinuteKey(new Date(r.timestampMs ?? Date.now()));+ const key = `${minute}:${r.country}`;+ const existing = minuteMap.get(key);+ if (existing) {+ existing.upload += r.upload;+ existing.download += r.download;+ existing.connections++;+ } else {+ minuteMap.set(key, { minute, country: r.country, countryName: r.countryName, continent: r.continent, upload: r.upload, download: r.download, connections: 1 });+ }+ }++ for (const [, item] of minuteMap) {+ minuteStmt.run({ backendId, ...item });+ }+ });+ tx();+ }+}diff --git a/apps/collector/src/database/repositories/device.repository.ts b/apps/collector/src/database/repositories/device.repository.tsnew file mode 100644index 0000000..d0c447e--- /dev/null+++ b/apps/collector/src/database/repositories/device.repository.ts@@ -0,0 +1,124 @@+/**+ * Device Repository+ *+ * Handles device (source IP) statistics and their domain/IP breakdowns.+ */+import type Database from 'better-sqlite3';+import type { DomainStats, IPStats, DeviceStats } from '@neko-master/shared';+import { BaseRepository } from './base.repository.js';++export class DeviceRepository extends BaseRepository {+ constructor(db: Database.Database) {+ super(db);+ }++ getDevices(backendId: number, limit = 50, start?: string, end?: string): DeviceStats[] {+ const range = this.parseMinuteRange(start, end);+ if (range) {+ const stmt = this.db.prepare(`+ SELECT source_ip as sourceIP, SUM(upload) as totalUpload, SUM(download) as totalDownload,+ SUM(connections) as totalConnections, MAX(minute) as lastSeen+ FROM minute_dim_stats+ WHERE backend_id = ? AND minute >= ? AND minute <= ? AND source_ip != ''+ GROUP BY source_ip+ ORDER BY (SUM(upload) + SUM(download)) DESC+ LIMIT ?+ `);+ return stmt.all(backendId, range.startMinute, range.endMinute, limit) as DeviceStats[];+ }++ const stmt = this.db.prepare(`+ SELECT source_ip as sourceIP, total_upload as totalUpload, total_download as totalDownload,+ total_connections as totalConnections, last_seen as lastSeen+ FROM device_stats WHERE backend_id = ?+ ORDER BY (total_upload + total_download) DESC+ LIMIT ?+ `);+ return stmt.all(backendId, limit) as DeviceStats[];+ }++ getDeviceDomains(backendId: number, sourceIP: string, limit = 5000, start?: string, end?: string): DomainStats[] {+ const range = this.parseMinuteRange(start, end);+ if (range) {+ const stmt = this.db.prepare(`+ SELECT domain, SUM(upload) as totalUpload, SUM(download) as totalDownload,+ SUM(connections) as totalConnections, MAX(minute) as lastSeen,+ GROUP_CONCAT(DISTINCT ip) as ips, GROUP_CONCAT(DISTINCT rule) as rules, GROUP_CONCAT(DISTINCT chain) as chains+ FROM minute_dim_stats+ WHERE backend_id = ? AND minute >= ? AND minute <= ? AND source_ip = ? AND domain != ''+ GROUP BY domain ORDER BY (SUM(upload) + SUM(download)) DESC LIMIT ?+ `);+ const rows = stmt.all(backendId, range.startMinute, range.endMinute, sourceIP, limit) as Array<{+ domain: string; totalUpload: number; totalDownload: number; totalConnections: number; lastSeen: string;+ ips: string | null; rules: string | null; chains: string | null;+ }>;+ return rows.map(row => {+ const rules = row.rules ? row.rules.split(',').filter(Boolean) : [];+ const chains = row.chains ? row.chains.split(',').filter(Boolean) : [];+ return { ...row, ips: row.ips ? row.ips.split(',').filter(Boolean) : [], rules, chains: this.expandShortChainsForRules(backendId, chains, rules) };+ }) as DomainStats[];+ }++ const stmt = this.db.prepare(`+ SELECT d.domain, d.total_upload as totalUpload, d.total_download as totalDownload,+ d.total_connections as totalConnections, d.last_seen as lastSeen, g.ips+ FROM device_domain_stats d+ LEFT JOIN domain_stats g ON d.domain = g.domain AND d.backend_id = g.backend_id+ WHERE d.backend_id = ? AND d.source_ip = ?+ ORDER BY (d.total_upload + d.total_download) DESC LIMIT ?+ `);+ const result = stmt.all(backendId, sourceIP, limit) as any[];+ return result.map(r => ({ ...r, ips: r.ips ? r.ips.split(',') : [], rules: [], chains: [] }));+ }++ getDeviceIPs(backendId: number, sourceIP: string, limit = 5000, start?: string, end?: string): IPStats[] {+ const range = this.parseMinuteRange(start, end);+ if (range) {+ const stmt = this.db.prepare(`+ SELECT m.ip, SUM(m.upload) as totalUpload, SUM(m.download) as totalDownload,+ SUM(m.connections) as totalConnections, MAX(m.minute) as lastSeen,+ GROUP_CONCAT(DISTINCT CASE WHEN m.domain != '' THEN m.domain END) as domains,+ COALESCE(i.asn, g.asn) as asn,+ CASE WHEN g.country IS NOT NULL THEN json_array(g.country, COALESCE(g.country_name, g.country), COALESCE(g.city, ''), COALESCE(g.as_name, ''))+ WHEN i.geoip IS NOT NULL THEN json(i.geoip) ELSE NULL END as geoIP,+ GROUP_CONCAT(DISTINCT m.chain) as chains, GROUP_CONCAT(DISTINCT m.rule) as rules+ FROM minute_dim_stats m+ LEFT JOIN ip_stats i ON m.backend_id = i.backend_id AND m.ip = i.ip+ LEFT JOIN geoip_cache g ON m.ip = g.ip+ WHERE m.backend_id = ? AND m.minute >= ? AND m.minute <= ? AND m.source_ip = ? AND m.ip != ''+ GROUP BY m.ip ORDER BY (SUM(m.upload) + SUM(m.download)) DESC LIMIT ?+ `);+ const rows = stmt.all(backendId, range.startMinute, range.endMinute, sourceIP, limit) as Array<{+ ip: string; totalUpload: number; totalDownload: number; totalConnections: number; lastSeen: string;+ domains: string | null; asn: string | null; geoIP: string | null; chains: string | null; rules: string | null;+ }>;+ return rows.map(row => {+ const rules = row.rules ? row.rules.split(',').filter(Boolean) : [];+ const chains = row.chains ? row.chains.split(',').filter(Boolean) : [];+ return {+ ...row, domains: row.domains ? row.domains.split(',').filter(Boolean) : [],+ geoIP: row.geoIP ? JSON.parse(row.geoIP).filter(Boolean) : undefined, asn: row.asn || undefined,+ chains: this.expandShortChainsForRules(backendId, chains, rules),+ };+ }) as IPStats[];+ }++ const stmt = this.db.prepare(`+ SELECT d.ip, d.total_upload as totalUpload, d.total_download as totalDownload,+ d.total_connections as totalConnections, d.last_seen as lastSeen, i.domains,+ COALESCE(i.asn, g.asn) as asn,+ CASE WHEN g.country IS NOT NULL THEN json_array(g.country, COALESCE(g.country_name, g.country), COALESCE(g.city, ''), COALESCE(g.as_name, ''))+ WHEN i.geoip IS NOT NULL THEN json(i.geoip) ELSE NULL END as geoIP+ FROM device_ip_stats d+ LEFT JOIN ip_stats i ON d.ip = i.ip AND d.backend_id = i.backend_id+ LEFT JOIN geoip_cache g ON d.ip = g.ip+ WHERE d.backend_id = ? AND d.source_ip = ?+ ORDER BY (d.total_upload + d.total_download) DESC LIMIT ?+ `);+ const result = stmt.all(backendId, sourceIP, limit) as any[];+ return result.map(r => ({+ ...r, domains: r.domains ? r.domains.split(',') : [],+ geoIP: r.geoIP ? JSON.parse(r.geoIP).filter(Boolean) : undefined, asn: r.asn || undefined,+ }));+ }+}diff --git a/apps/collector/src/database/repositories/domain.repository.ts b/apps/collector/src/database/repositories/domain.repository.tsindex ccc838f..1e486aa 100644--- a/apps/collector/src/database/repositories/domain.repository.ts+++ b/apps/collector/src/database/repositories/domain.repository.ts@@ -671,54 +671,4 @@ export class DomainRepository extends BaseRepository {})) as IPStats[];}- /**- * Expand short chains for rules using rule_chain_traffic- */- private expandShortChainsForRules(- backendId: number,- chains: string[],- rules: string[],- ): string[] {- const normalizedChains = this.uniqueNonEmpty(chains);- if (normalizedChains.length === 0) return [];-- const shortChains = normalizedChains.filter((c) => !c.includes(">"));- if (shortChains.length === 0) return normalizedChains;-- const normalizedRules = this.uniqueNonEmpty(rules);- const whereParts: string[] = [];- const params: Array<string | number> = [backendId];-- if (normalizedRules.length > 0) {- const rulePlaceholders = normalizedRules.map(() => "?").join(", ");- whereParts.push(`rule IN (${rulePlaceholders})`);- params.push(...normalizedRules);- }-- const chainMatchers: string[] = [];- for (const chain of shortChains) {- chainMatchers.push("(chain = ? OR chain LIKE ?)");- params.push(chain, `${chain} > %`);- }- whereParts.push(`(${chainMatchers.join(" OR ")})`);-- const whereClause = whereParts.length > 0 ? `AND ${whereParts.join(" AND ")}` : "";- const stmt = this.db.prepare(`- SELECT DISTINCT chain- FROM rule_chain_traffic- WHERE backend_id = ? ${whereClause}- LIMIT 500- `);-- const rows = stmt.all(...params) as Array<{ chain: string }>;- const expanded = this.uniqueNonEmpty(rows.map((r) => r.chain));-- if (expanded.length === 0) {- return normalizedChains;- }-- // Prefer expanded full chains, but keep already-full values from input too.- const fullInputChains = normalizedChains.filter((c) => c.includes(">"));- return this.uniqueNonEmpty([...expanded, ...fullInputChains]);- }}diff --git a/apps/collector/src/database/repositories/index.ts b/apps/collector/src/database/repositories/index.tsindex e04c6ae..fa46e5a 100644--- a/apps/collector/src/database/repositories/index.ts+++ b/apps/collector/src/database/repositories/index.ts@@ -1,6 +1,6 @@/*** Database Repositories- *+ ** This module exports all repository classes for database operations.* Use these repositories to perform specific domain operations.*/@@ -8,3 +8,13 @@export { BaseRepository } from './base.repository.js';export { DomainRepository } from './domain.repository.js';export { BackendRepository, type BackendConfig } from './backend.repository.js';+export { AuthRepository } from './auth.repository.js';+export { SurgeRepository } from './surge.repository.js';+export { TimeseriesRepository } from './timeseries.repository.js';+export { CountryRepository } from './country.repository.js';+export { DeviceRepository } from './device.repository.js';+export { ProxyRepository } from './proxy.repository.js';+export { RuleRepository } from './rule.repository.js';+export { IPRepository } from './ip.repository.js';+export { ConfigRepository } from './config.repository.js';+export { TrafficWriterRepository, type TrafficUpdate } from './traffic-writer.repository.js';diff --git a/apps/collector/src/database/repositories/ip.repository.ts b/apps/collector/src/database/repositories/ip.repository.tsnew file mode 100644index 0000000..482d0e5--- /dev/null+++ b/apps/collector/src/database/repositories/ip.repository.ts@@ -0,0 +1,432 @@+/**+ * IP Repository+ *+ * Handles IP statistics, ASN/GeoIP lookups, paginated IP queries,+ * and IP-domain/IP-proxy breakdowns.+ */+import type Database from 'better-sqlite3';+import type { DomainStats, IPStats } from '@neko-master/shared';+import { BaseRepository } from './base.repository.js';++interface ProxyTrafficStats {+ chain: string;+ totalUpload: number;+ totalDownload: number;+ totalConnections: number;+}++interface GeoIPInfo {+ country: string;+ country_name: string;+ city: string;+ asn: string;+ as_name: string;+ as_domain: string;+ continent: string;+ continent_name: string;+}++export class IPRepository extends BaseRepository {+ constructor(db: Database.Database) {+ super(db);+ }++ getIPStats(backendId: number, limit = 100, start?: string, end?: string): IPStats[] {+ const range = this.parseMinuteRange(start, end);+ if (range) {+ const stmt = this.db.prepare(`+ SELECT m.ip, GROUP_CONCAT(DISTINCT CASE WHEN m.domain != '' THEN m.domain END) as domains,+ SUM(m.upload) as totalUpload, SUM(m.download) as totalDownload,+ SUM(m.connections) as totalConnections, MAX(m.minute) as lastSeen,+ COALESCE(i.asn, g.asn) as asn,+ CASE WHEN g.country IS NOT NULL THEN json_array(g.country, COALESCE(g.country_name, g.country), COALESCE(g.city, ''), COALESCE(g.as_name, ''))+ WHEN i.geoip IS NOT NULL THEN json(i.geoip) ELSE NULL END as geoIP,+ GROUP_CONCAT(DISTINCT m.chain) as chains, GROUP_CONCAT(DISTINCT m.rule) as rules+ FROM minute_dim_stats m+ LEFT JOIN ip_stats i ON m.backend_id = i.backend_id AND m.ip = i.ip+ LEFT JOIN geoip_cache g ON m.ip = g.ip+ WHERE m.backend_id = ? AND m.minute >= ? AND m.minute <= ? AND m.ip != ''+ GROUP BY m.ip ORDER BY (SUM(m.upload) + SUM(m.download)) DESC LIMIT ?+ `);+ const rows = stmt.all(backendId, range.startMinute, range.endMinute, limit) as Array<{+ ip: string; domains: string; totalUpload: number; totalDownload: number; totalConnections: number;+ lastSeen: string; asn: string | null; geoIP: string | null; chains: string | null; rules: string | null;+ }>;++ return rows.map(row => {+ const rules = row.rules ? row.rules.split(',').filter(Boolean) : [];+ const chains = row.chains ? row.chains.split(',').filter(Boolean) : [];+ return {+ ...row, domains: row.domains ? row.domains.split(',').filter(Boolean) : [],+ geoIP: row.geoIP ? JSON.parse(row.geoIP).filter(Boolean) : undefined,+ asn: row.asn || undefined,+ chains: this.expandShortChainsForRules(backendId, chains, rules),+ };+ }) as IPStats[];+ }++ const stmt = this.db.prepare(`+ SELECT i.ip, i.domains, i.total_upload as totalUpload, i.total_download as totalDownload,+ i.total_connections as totalConnections, i.last_seen as lastSeen,+ COALESCE(i.asn, g.asn) as asn,+ CASE WHEN g.country IS NOT NULL THEN json_array(g.country, COALESCE(g.country_name, g.country), COALESCE(g.city, ''), COALESCE(g.as_name, ''))+ WHEN i.geoip IS NOT NULL THEN json(i.geoip) ELSE NULL END as geoIP,+ i.chains+ FROM ip_stats i LEFT JOIN geoip_cache g ON i.ip = g.ip+ WHERE i.backend_id = ? AND i.ip != ''+ ORDER BY (i.total_upload + i.total_download) DESC LIMIT ?+ `);+ const rows = stmt.all(backendId, limit) as Array<{+ ip: string; domains: string; totalUpload: number; totalDownload: number;+ totalConnections: number; lastSeen: string; asn: string | null; geoIP: string | null; chains: string | null;+ }>;++ return rows.map(row => ({+ ...row, domains: row.domains ? row.domains.split(',') : [],+ geoIP: row.geoIP ? JSON.parse(row.geoIP).filter(Boolean) : undefined,+ asn: row.asn || undefined, chains: row.chains ? row.chains.split(',') : [],+ })) as IPStats[];+ }++ getIPStatsByIPs(backendId: number, ips: string[]): IPStats[] {+ const filteredIps = ips.filter(ip => ip && ip.trim() !== '');+ if (filteredIps.length === 0) return [];++ const placeholders = filteredIps.map(() => '?').join(',');+ const stmt = this.db.prepare(`+ SELECT i.ip, i.domains, i.total_upload as totalUpload, i.total_download as totalDownload,+ i.total_connections as totalConnections, i.last_seen as lastSeen,+ COALESCE(i.asn, g.asn) as asn,+ CASE WHEN g.country IS NOT NULL THEN json_array(g.country, COALESCE(g.country_name, g.country), COALESCE(g.city, ''), COALESCE(g.as_name, ''))+ WHEN i.geoip IS NOT NULL THEN json(i.geoip) ELSE NULL END as geoIP,+ i.chains+ FROM ip_stats i LEFT JOIN geoip_cache g ON i.ip = g.ip+ WHERE i.backend_id = ? AND i.ip IN (${placeholders})+ ORDER BY (i.total_upload + i.total_download) DESC+ `);+ const rows = stmt.all(backendId, ...filteredIps) as Array<{+ ip: string; domains: string; totalUpload: number; totalDownload: number;+ totalConnections: number; lastSeen: string; asn: string | null; geoIP: string | null; chains: string | null;+ }>;++ return rows.map(row => ({+ ...row, domains: row.domains ? row.domains.split(',') : [],+ geoIP: row.geoIP ? JSON.parse(row.geoIP).filter(Boolean) : undefined,+ asn: row.asn || undefined, chains: row.chains ? row.chains.split(',') : [],+ })) as IPStats[];+ }++ getIPStatsPaginated(backendId: number, opts: {+ offset?: number; limit?: number; sortBy?: string; sortOrder?: string;+ search?: string; start?: string; end?: string;+ } = {}): { data: IPStats[]; total: number } {+ const offset = opts.offset ?? 0;+ const limit = Math.min(opts.limit ?? 50, 200);+ const sortOrder = opts.sortOrder === 'asc' ? 'ASC' : 'DESC';+ const search = opts.search?.trim() || '';+ const range = this.parseMinuteRange(opts.start, opts.end);++ const sortColumnMap: Record<string, string> = {+ ip: 'i.ip', totalDownload: 'i.total_download', totalUpload: 'i.total_upload',+ totalTraffic: '(i.total_upload + i.total_download)',+ totalConnections: 'i.total_connections', lastSeen: 'i.last_seen',+ };+ const sortColumn = sortColumnMap[opts.sortBy || 'totalDownload'] || 'i.total_download';++ if (range) {+ const rangeSortColumnMap: Record<string, string> = {+ ip: 'agg.ip', totalDownload: 'agg.totalDownload', totalUpload: 'agg.totalUpload',+ totalTraffic: '(agg.totalUpload + agg.totalDownload)',+ totalConnections: 'agg.totalConnections', lastSeen: 'agg.lastSeen',+ };+ const rangeSortColumn = rangeSortColumnMap[opts.sortBy || 'totalDownload'] || 'agg.totalDownload';++ const whereSearch = search ? "AND (ip LIKE ? OR domain LIKE ?)" : "";+ const baseParams: any[] = [backendId, range.startMinute, range.endMinute];+ const searchParams: any[] = search ? [`%${search}%`, `%${search}%`] : [];++ const countStmt = this.db.prepare(`+ SELECT COUNT(*) as total FROM (+ SELECT ip FROM minute_dim_stats+ WHERE backend_id = ? AND minute >= ? AND minute <= ? AND ip != '' ${whereSearch}+ GROUP BY ip+ )+ `);+ const { total } = countStmt.get(...baseParams, ...searchParams) as { total: number };++ const dataStmt = this.db.prepare(`+ WITH agg AS (+ SELECT ip, GROUP_CONCAT(DISTINCT CASE WHEN domain != '' THEN domain END) as domains,+ GROUP_CONCAT(DISTINCT rule) as rules,+ SUM(upload) as totalUpload, SUM(download) as totalDownload,+ SUM(connections) as totalConnections, MAX(minute) as lastSeen,+ GROUP_CONCAT(DISTINCT chain) as chains+ FROM minute_dim_stats+ WHERE backend_id = ? AND minute >= ? AND minute <= ? AND ip != '' ${whereSearch}+ GROUP BY ip+ )+ SELECT agg.ip, agg.domains, agg.rules, agg.totalUpload, agg.totalDownload,+ agg.totalConnections, agg.lastSeen,+ COALESCE(i.asn, g.asn) as asn,+ CASE WHEN g.country IS NOT NULL THEN json_array(g.country, COALESCE(g.country_name, g.country), COALESCE(g.city, ''), COALESCE(g.as_name, ''))+ WHEN i.geoip IS NOT NULL THEN json(i.geoip) ELSE NULL END as geoIP,+ agg.chains+ FROM agg+ LEFT JOIN ip_stats i ON i.backend_id = ? AND i.ip = agg.ip+ LEFT JOIN geoip_cache g ON g.ip = agg.ip+ ORDER BY ${rangeSortColumn} ${sortOrder} LIMIT ? OFFSET ?+ `);+ const rows = dataStmt.all(...baseParams, ...searchParams, backendId, limit, offset) as Array<{+ ip: string; domains: string; rules: string | null; totalUpload: number; totalDownload: number;+ totalConnections: number; lastSeen: string; asn: string | null; geoIP: string | null; chains: string | null;+ }>;++ const data = rows.map(row => {+ const rules = row.rules ? row.rules.split(',').filter(Boolean) : [];+ const chains = row.chains ? row.chains.split(',').filter(Boolean) : [];+ return {+ ip: row.ip, domains: row.domains ? row.domains.split(',').filter(Boolean) : [],+ totalUpload: row.totalUpload, totalDownload: row.totalDownload,+ totalConnections: row.totalConnections, lastSeen: row.lastSeen,+ geoIP: row.geoIP ? JSON.parse(row.geoIP).filter(Boolean) : undefined,+ asn: row.asn || undefined,+ chains: this.expandShortChainsForRules(backendId, chains, rules),+ };+ }) as IPStats[];++ return { data, total };+ }++ const whereClause = search+ ? "WHERE i.backend_id = ? AND i.ip != '' AND (i.ip LIKE ? OR i.domains LIKE ?)"+ : "WHERE i.backend_id = ? AND i.ip != ''";+ const params: any[] = search ? [backendId, `%${search}%`, `%${search}%`] : [backendId];++ const countStmt = this.db.prepare(`SELECT COUNT(*) as total FROM ip_stats i ${whereClause}`);+ const { total } = countStmt.get(...params) as { total: number };++ const dataStmt = this.db.prepare(`+ SELECT i.ip, i.domains, i.total_upload as totalUpload, i.total_download as totalDownload,+ i.total_connections as totalConnections, i.last_seen as lastSeen, i.rules,+ COALESCE(i.asn, g.asn) as asn,+ CASE WHEN g.country IS NOT NULL THEN json_array(g.country, COALESCE(g.country_name, g.country), COALESCE(g.city, ''), COALESCE(g.as_name, ''))+ WHEN i.geoip IS NOT NULL THEN json(i.geoip) ELSE NULL END as geoIP,+ i.chains+ FROM ip_stats i LEFT JOIN geoip_cache g ON i.ip = g.ip+ ${whereClause} ORDER BY ${sortColumn} ${sortOrder} LIMIT ? OFFSET ?+ `);+ const rows = dataStmt.all(...params, limit, offset) as Array<{+ ip: string; domains: string; totalUpload: number; totalDownload: number;+ totalConnections: number; lastSeen: string; rules: string | null;+ asn: string | null; geoIP: string | null; chains: string | null;+ }>;++ const data = rows.map(row => {+ const rules = row.rules ? row.rules.split(',').filter(Boolean) : [];+ const chains = row.chains ? row.chains.split(',').filter(Boolean) : [];+ return {+ ip: row.ip, domains: row.domains ? row.domains.split(',') : [],+ totalUpload: row.totalUpload, totalDownload: row.totalDownload,+ totalConnections: row.totalConnections, lastSeen: row.lastSeen,+ geoIP: row.geoIP ? JSON.parse(row.geoIP).filter(Boolean) : undefined,+ asn: row.asn || undefined,+ chains: this.expandShortChainsForRules(backendId, chains, rules),+ };+ }) as IPStats[];++ return { data, total };+ }++ getDomainIPDetails(+ backendId: number, domain: string, start?: string, end?: string,+ limit = 100, sourceIP?: string, sourceChain?: string,+ ): IPStats[] {+ const range = this.parseMinuteRange(start, end);+ if (range || sourceIP || sourceChain) {+ const conditions = ["m.backend_id = ?", "m.domain = ?", "m.ip != ''"];+ const params: Array<string | number> = [backendId, domain];+ if (range) {+ conditions.push("m.minute >= ?", "m.minute <= ?");+ params.push(range.startMinute, range.endMinute);+ }+ if (sourceIP) {+ conditions.push("m.source_ip = ?");+ params.push(sourceIP);+ }+ if (sourceChain) {+ conditions.push("(m.chain = ? OR m.chain LIKE ?)");+ params.push(sourceChain, `${sourceChain} > %`);+ }++ const stmt = this.db.prepare(`+ SELECT m.ip, GROUP_CONCAT(DISTINCT CASE WHEN m.domain != '' THEN m.domain END) as domains,+ SUM(m.upload) as totalUpload, SUM(m.download) as totalDownload,+ SUM(m.connections) as totalConnections, MAX(m.minute) as lastSeen,+ COALESCE(i.asn, g.asn) as asn,+ CASE WHEN g.country IS NOT NULL THEN json_array(g.country, COALESCE(g.country_name, g.country), COALESCE(g.city, ''), COALESCE(g.as_name, ''))+ WHEN i.geoip IS NOT NULL THEN json(i.geoip) ELSE NULL END as geoIP,+ GROUP_CONCAT(DISTINCT m.chain) as chains, GROUP_CONCAT(DISTINCT m.rule) as rules+ FROM minute_dim_stats m+ LEFT JOIN ip_stats i ON m.backend_id = i.backend_id AND m.ip = i.ip+ LEFT JOIN geoip_cache g ON m.ip = g.ip+ WHERE ${conditions.join(" AND ")}+ GROUP BY m.ip ORDER BY (SUM(m.upload) + SUM(m.download)) DESC LIMIT ?+ `);+ const rows = stmt.all(...params, limit) as Array<{+ ip: string; domains: string; totalUpload: number; totalDownload: number;+ totalConnections: number; lastSeen: string; asn: string | null;+ geoIP: string | null; chains: string | null; rules: string | null;+ }>;++ return rows.map(row => {+ const rules = row.rules ? row.rules.split(',').filter(Boolean) : [];+ const chains = row.chains ? row.chains.split(',').filter(Boolean) : [];+ return {+ ...row, domains: row.domains ? row.domains.split(',').filter(Boolean) : [],+ geoIP: row.geoIP ? JSON.parse(row.geoIP).filter(Boolean) : undefined,+ asn: row.asn || undefined,+ chains: this.expandShortChainsForRules(backendId, chains, rules),+ };+ }) as IPStats[];+ }++ // Fallback: look up domain ips from domain_stats, then fetch IP details+ const domainRow = this.db.prepare(+ `SELECT ips FROM domain_stats WHERE backend_id = ? AND domain = ?`,+ ).get(backendId, domain) as { ips: string | null } | undefined;++ if (!domainRow || !domainRow.ips) return [];+ const ipList = domainRow.ips.split(',').filter(Boolean).slice(0, limit);+ if (ipList.length === 0) return [];++ return this.getIPStatsByIPs(backendId, ipList);+ }++ getIPDomainDetails(+ backendId: number, ip: string, start?: string, end?: string,+ limit = 100, sourceIP?: string, sourceChain?: string,+ ): DomainStats[] {+ const range = this.parseMinuteRange(start, end);+ const conditions = ["backend_id = ?", "ip = ?", "domain != ''"];+ const params: Array<string | number> = [backendId, ip];+ if (range) {+ conditions.push("minute >= ?", "minute <= ?");+ params.push(range.startMinute, range.endMinute);+ }+ if (sourceIP) {+ conditions.push("source_ip = ?");+ params.push(sourceIP);+ }+ if (sourceChain) {+ conditions.push("(chain = ? OR chain LIKE ?)");+ params.push(sourceChain, `${sourceChain} > %`);+ }++ const stmt = this.db.prepare(`+ SELECT domain, GROUP_CONCAT(DISTINCT ip) as ips,+ SUM(upload) as totalUpload, SUM(download) as totalDownload,+ SUM(connections) as totalConnections, MAX(minute) as lastSeen,+ GROUP_CONCAT(DISTINCT CASE WHEN rule != '' THEN rule END) as rules,+ GROUP_CONCAT(DISTINCT chain) as chains+ FROM minute_dim_stats WHERE ${conditions.join(" AND ")}+ GROUP BY domain ORDER BY (SUM(upload) + SUM(download)) DESC LIMIT ?+ `);++ const rows = stmt.all(...params, limit) as Array<{+ domain: string; ips: string | null; totalUpload: number; totalDownload: number;+ totalConnections: number; lastSeen: string; rules: string | null; chains: string | null;+ }>;++ return rows.map(row => {+ const rules = row.rules ? row.rules.split(',').filter(Boolean) : [];+ const chains = row.chains ? row.chains.split(',').filter(Boolean) : [];+ return {+ domain: row.domain, ips: row.ips ? row.ips.split(',').filter(Boolean) : [],+ totalUpload: row.totalUpload, totalDownload: row.totalDownload,+ totalConnections: row.totalConnections, lastSeen: row.lastSeen, rules,+ chains: this.expandShortChainsForRules(backendId, chains, rules),+ };+ }) as DomainStats[];+ }++ getIPProxyStats(+ backendId: number, ip: string, start?: string, end?: string,+ sourceIP?: string, sourceChain?: string,+ ): ProxyTrafficStats[] {+ const range = this.parseMinuteRange(start, end);+ if (range || sourceIP || sourceChain) {+ const conditions = ["backend_id = ?", "ip = ?"];+ const params: Array<string | number> = [backendId, ip];+ if (range) {+ conditions.push("minute >= ?", "minute <= ?");+ params.push(range.startMinute, range.endMinute);+ }+ if (sourceIP) {+ conditions.push("source_ip = ?");+ params.push(sourceIP);+ }+ if (sourceChain) {+ conditions.push("(chain = ? OR chain LIKE ?)");+ params.push(sourceChain, `${sourceChain} > %`);+ }++ const stmt = this.db.prepare(`+ SELECT chain, SUM(upload) as totalUpload, SUM(download) as totalDownload, SUM(connections) as totalConnections+ FROM minute_dim_stats WHERE ${conditions.join(" AND ")}+ GROUP BY chain ORDER BY (SUM(upload) + SUM(download)) DESC+ `);+ return stmt.all(...params) as ProxyTrafficStats[];+ }++ const stmt = this.db.prepare(`+ SELECT chain, total_upload as totalUpload, total_download as totalDownload, total_connections as totalConnections+ FROM ip_proxy_stats WHERE backend_id = ? AND ip = ?+ ORDER BY (total_upload + total_download) DESC+ `);+ return stmt.all(backendId, ip) as ProxyTrafficStats[];+ }++ // ASN methods+ updateASNInfo(ip: string, asn: string, org: string): void {+ const stmt = this.db.prepare(`+ INSERT INTO asn_cache (ip, asn, org, queried_at) VALUES (?, ?, ?, CURRENT_TIMESTAMP)+ ON CONFLICT(ip) DO UPDATE SET asn = ?, org = ?, queried_at = CURRENT_TIMESTAMP+ `);+ stmt.run(ip, asn, org, asn, org);+ }++ getASNInfo(ips: string[]): Array<{ ip: string; asn: string; org: string }> {+ const stmt = this.db.prepare(`+ SELECT ip, asn, org FROM asn_cache WHERE ip IN (${ips.map(() => '?').join(',')})+ `);+ return stmt.all(...ips) as Array<{ ip: string; asn: string; org: string }>;+ }++ getASNInfoForIP(ip: string): { ip: string; asn: string; org: string } | undefined {+ const stmt = this.db.prepare(`SELECT ip, asn, org FROM asn_cache WHERE ip = ?`);+ return stmt.get(ip) as { ip: string; asn: string; org: string } | undefined;+ }++ // GeoIP methods+ getIPGeolocation(ip: string): GeoIPInfo | undefined {+ const stmt = this.db.prepare(`+ SELECT country, country_name, city, asn, as_name, as_domain, continent, continent_name+ FROM geoip_cache WHERE ip = ?+ `);+ return stmt.get(ip) as GeoIPInfo | undefined;+ }++ saveIPGeolocation(ip: string, geo: GeoIPInfo): void {+ const stmt = this.db.prepare(`+ INSERT INTO geoip_cache (ip, country, country_name, city, asn, as_name, as_domain, continent, continent_name, queried_at)+ VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, CURRENT_TIMESTAMP)+ ON CONFLICT(ip) DO UPDATE SET+ country = ?, country_name = ?, city = ?, asn = ?, as_name = ?, as_domain = ?,+ continent = ?, continent_name = ?, queried_at = CURRENT_TIMESTAMP+ `);+ stmt.run(+ ip, geo.country, geo.country_name, geo.city, geo.asn, geo.as_name, geo.as_domain, geo.continent, geo.continent_name,+ geo.country, geo.country_name, geo.city, geo.asn, geo.as_name, geo.as_domain, geo.continent, geo.continent_name,+ );+ }+}diff --git a/apps/collector/src/database/repositories/proxy.repository.ts b/apps/collector/src/database/repositories/proxy.repository.tsnew file mode 100644index 0000000..758f719--- /dev/null+++ b/apps/collector/src/database/repositories/proxy.repository.ts@@ -0,0 +1,118 @@+/**+ * Proxy Repository+ *+ * Handles proxy/chain statistics and their domain/IP breakdowns.+ */+import type Database from 'better-sqlite3';+import type { DomainStats, IPStats, ProxyStats } from '@neko-master/shared';+import { BaseRepository } from './base.repository.js';++export class ProxyRepository extends BaseRepository {+ constructor(db: Database.Database) {+ super(db);+ }++ getProxyStats(backendId: number, start?: string, end?: string): ProxyStats[] {+ const range = this.parseMinuteRange(start, end);+ if (range) {+ const stmt = this.db.prepare(`+ SELECT chain, SUM(upload) as totalUpload, SUM(download) as totalDownload,+ SUM(connections) as totalConnections, MAX(minute) as lastSeen+ FROM minute_dim_stats WHERE backend_id = ? AND minute >= ? AND minute <= ?+ GROUP BY chain ORDER BY (SUM(upload) + SUM(download)) DESC+ `);+ return this.aggregateProxyStatsByFirstHop(stmt.all(backendId, range.startMinute, range.endMinute) as ProxyStats[]);+ }++ const stmt = this.db.prepare(`+ SELECT chain, total_upload as totalUpload, total_download as totalDownload,+ total_connections as totalConnections, last_seen as lastSeen+ FROM proxy_stats WHERE backend_id = ? ORDER BY (total_upload + total_download) DESC+ `);+ return this.aggregateProxyStatsByFirstHop(stmt.all(backendId) as ProxyStats[]);+ }++ getProxyDomains(backendId: number, chain: string, limit = 50, start?: string, end?: string): DomainStats[] {+ const range = this.parseMinuteRange(start, end);+ if (range) {+ const stmt = this.db.prepare(`+ SELECT domain, SUM(upload) as totalUpload, SUM(download) as totalDownload,+ SUM(connections) as totalConnections, MAX(minute) as lastSeen, GROUP_CONCAT(DISTINCT ip) as ips+ FROM minute_dim_stats+ WHERE backend_id = ? AND minute >= ? AND minute <= ? AND (chain = ? OR chain LIKE ?) AND domain != ''+ GROUP BY domain ORDER BY (SUM(upload) + SUM(download)) DESC LIMIT ?+ `);+ const rows = stmt.all(backendId, range.startMinute, range.endMinute, chain, `${chain} > %`, limit) as Array<{+ domain: string; totalUpload: number; totalDownload: number; totalConnections: number; lastSeen: string; ips: string | null;+ }>;+ return rows.map(row => ({ ...row, ips: row.ips ? row.ips.split(',').filter(Boolean) : [], rules: [], chains: [chain] })) as DomainStats[];+ }++ const stmt = this.db.prepare(`+ SELECT dps.domain, dps.total_upload as totalUpload, dps.total_download as totalDownload,+ dps.total_connections as totalConnections, dps.last_seen as lastSeen, ds.ips+ FROM domain_proxy_stats dps+ LEFT JOIN domain_stats ds ON dps.backend_id = ds.backend_id AND dps.domain = ds.domain+ WHERE dps.backend_id = ? AND (dps.chain = ? OR dps.chain LIKE ?)+ ORDER BY (dps.total_upload + dps.total_download) DESC LIMIT ?+ `);+ const rows = stmt.all(backendId, chain, `${chain} > %`, limit) as Array<{+ domain: string; totalUpload: number; totalDownload: number; totalConnections: number; lastSeen: string; ips: string | null;+ }>;+ return rows.map(row => ({ ...row, ips: row.ips ? row.ips.split(',').filter(Boolean) : [], rules: [], chains: [chain] })) as DomainStats[];+ }++ getProxyIPs(backendId: number, chain: string, limit = 50, start?: string, end?: string): IPStats[] {+ const range = this.parseMinuteRange(start, end);+ if (range) {+ const stmt = this.db.prepare(`+ SELECT m.ip, SUM(m.upload) as totalUpload, SUM(m.download) as totalDownload,+ SUM(m.connections) as totalConnections, MAX(m.minute) as lastSeen,+ GROUP_CONCAT(DISTINCT CASE WHEN m.domain != '' THEN m.domain END) as domains,+ COALESCE(i.asn, g.asn) as asn,+ CASE WHEN g.country IS NOT NULL THEN json_array(g.country, COALESCE(g.country_name, g.country), COALESCE(g.city, ''), COALESCE(g.as_name, ''))+ WHEN i.geoip IS NOT NULL THEN json(i.geoip) ELSE NULL END as geoIP+ FROM minute_dim_stats m+ LEFT JOIN ip_stats i ON m.backend_id = i.backend_id AND m.ip = i.ip+ LEFT JOIN geoip_cache g ON m.ip = g.ip+ WHERE m.backend_id = ? AND m.minute >= ? AND m.minute <= ? AND (m.chain = ? OR m.chain LIKE ?) AND m.ip != ''+ GROUP BY m.ip ORDER BY (SUM(m.upload) + SUM(m.download)) DESC LIMIT ?+ `);+ const rows = stmt.all(backendId, range.startMinute, range.endMinute, chain, `${chain} > %`, limit) as Array<{+ ip: string; totalUpload: number; totalDownload: number; totalConnections: number; lastSeen: string;+ domains: string | null; asn: string | null; geoIP: string | null;+ }>;+ return rows.map(row => ({+ ...row, domains: row.domains ? row.domains.split(',').filter(Boolean) : [], chains: [chain],+ asn: row.asn || undefined, geoIP: row.geoIP ? JSON.parse(row.geoIP).filter(Boolean) : undefined,+ })) as IPStats[];+ }++ const stmt = this.db.prepare(`+ SELECT ips.ip, ips.total_upload as totalUpload, ips.total_download as totalDownload,+ ips.total_connections as totalConnections, ips.last_seen as lastSeen, ips.domains+ FROM ip_proxy_stats ips+ WHERE ips.backend_id = ? AND (ips.chain = ? OR ips.chain LIKE ?) AND ips.ip != ''+ ORDER BY (ips.total_upload + ips.total_download) DESC LIMIT ?+ `);+ const rows = stmt.all(backendId, chain, `${chain} > %`, limit) as Array<{+ ip: string; totalUpload: number; totalDownload: number; totalConnections: number; lastSeen: string; domains: string | null;+ }>;++ const ipList = rows.map(r => r.ip).filter(ip => ip && ip.trim() !== '');+ if (ipList.length === 0) return [];++ const placeholders = ipList.map(() => '?').join(',');+ const geoStmt = this.db.prepare(`+ SELECT ip, CASE WHEN country IS NOT NULL THEN json_array(country, COALESCE(country_name, country), COALESCE(city, ''), COALESCE(as_name, '')) ELSE NULL END as geoIP+ FROM geoip_cache WHERE ip IN (${placeholders})+ `);+ const geoRows = geoStmt.all(...ipList) as Array<{ ip: string; geoIP: string | null }>;+ const geoMap = new Map(geoRows.map(r => [r.ip, r.geoIP]));++ return rows.map(row => ({+ ...row, domains: row.domains ? row.domains.split(',').filter(Boolean) : [], chains: [chain],+ geoIP: geoMap.get(row.ip) ? JSON.parse(geoMap.get(row.ip)!).filter(Boolean) : undefined,+ })) as IPStats[];+ }+}diff --git a/apps/collector/src/database/repositories/rule.repository.ts b/apps/collector/src/database/repositories/rule.repository.tsnew file mode 100644index 0000000..9efefce--- /dev/null+++ b/apps/collector/src/database/repositories/rule.repository.ts@@ -0,0 +1,484 @@+/**+ * Rule Repository+ *+ * Handles rule statistics, rule-proxy mappings, rule chain flow visualization,+ * and per-rule domain/IP breakdowns.+ */+import type Database from 'better-sqlite3';+import type { DomainStats, IPStats, ProxyStats, RuleStats } from '@neko-master/shared';+import { BaseRepository } from './base.repository.js';++interface ProxyTrafficStats {+ chain: string;+ totalUpload: number;+ totalDownload: number;+ totalConnections: number;+}++export class RuleRepository extends BaseRepository {+ constructor(db: Database.Database) {+ super(db);+ }++ getRuleStats(backendId: number, start?: string, end?: string): RuleStats[] {+ const range = this.parseMinuteRange(start, end);+ if (range) {+ const stmt = this.db.prepare(`+ SELECT rule, MAX(chain) as finalProxy,+ SUM(upload) as totalUpload, SUM(download) as totalDownload,+ SUM(connections) as totalConnections, MAX(minute) as lastSeen+ FROM minute_dim_stats+ WHERE backend_id = ? AND minute >= ? AND minute <= ?+ GROUP BY rule ORDER BY (SUM(upload) + SUM(download)) DESC+ `);+ return stmt.all(backendId, range.startMinute, range.endMinute) as RuleStats[];+ }++ const stmt = this.db.prepare(`+ SELECT rule, final_proxy as finalProxy, total_upload as totalUpload, total_download as totalDownload,+ total_connections as totalConnections, last_seen as lastSeen+ FROM rule_stats WHERE backend_id = ? ORDER BY (total_upload + total_download) DESC+ `);+ return stmt.all(backendId) as RuleStats[];+ }++ getRuleProxyMap(backendId: number): Array<{ rule: string; proxies: string[] }> {+ const stmt = this.db.prepare(`+ SELECT rule, proxy FROM rule_proxy_map WHERE backend_id = ? ORDER BY rule, proxy+ `);+ const rows = stmt.all(backendId) as Array<{ rule: string; proxy: string }>;++ const map = new Map<string, string[]>();+ for (const row of rows) {+ if (!map.has(row.rule)) map.set(row.rule, []);+ map.get(row.rule)!.push(row.proxy);+ }+ return Array.from(map.entries()).map(([rule, proxies]) => ({ rule, proxies }));+ }++ getRuleDomains(backendId: number, rule: string, limit = 50, start?: string, end?: string): DomainStats[] {+ const range = this.parseMinuteRange(start, end);+ if (range) {+ const stmt = this.db.prepare(`+ SELECT domain, SUM(upload) as totalUpload, SUM(download) as totalDownload,+ SUM(connections) as totalConnections, MAX(minute) as lastSeen,+ GROUP_CONCAT(DISTINCT ip) as ips, GROUP_CONCAT(DISTINCT chain) as chains+ FROM minute_dim_stats+ WHERE backend_id = ? AND minute >= ? AND minute <= ? AND rule = ? AND domain != ''+ GROUP BY domain ORDER BY (SUM(upload) + SUM(download)) DESC LIMIT ?+ `);+ const rows = stmt.all(backendId, range.startMinute, range.endMinute, rule, limit) as Array<{+ domain: string; totalUpload: number; totalDownload: number; totalConnections: number;+ lastSeen: string; ips: string | null; chains: string | null;+ }>;++ return rows.map(row => {+ const chains = row.chains ? row.chains.split(',').filter(Boolean) : [];+ return {+ domain: row.domain, totalUpload: row.totalUpload, totalDownload: row.totalDownload,+ totalConnections: row.totalConnections, lastSeen: row.lastSeen,+ ips: row.ips ? row.ips.split(',').filter(Boolean) : [],+ chains: this.expandShortChainsForRules(backendId, chains, [rule]), rules: [rule],+ };+ }) as DomainStats[];+ }++ const stmt = this.db.prepare(`+ SELECT rdt.domain, rdt.total_upload as totalUpload, rdt.total_download as totalDownload,+ rdt.total_connections as totalConnections, rdt.last_seen as lastSeen, ds.ips, ds.chains+ FROM rule_domain_traffic rdt+ LEFT JOIN domain_stats ds ON rdt.backend_id = ds.backend_id AND rdt.domain = ds.domain+ WHERE rdt.backend_id = ? AND rdt.rule = ?+ ORDER BY (rdt.total_upload + rdt.total_download) DESC LIMIT ?+ `);+ const rows = stmt.all(backendId, rule, limit) as Array<{+ domain: string; totalUpload: number; totalDownload: number; totalConnections: number;+ lastSeen: string; ips: string | null; chains: string | null;+ }>;++ return rows.map(row => ({+ domain: row.domain, totalUpload: row.totalUpload, totalDownload: row.totalDownload,+ totalConnections: row.totalConnections, lastSeen: row.lastSeen,+ ips: row.ips ? row.ips.split(',').filter(Boolean) : [],+ chains: this.expandShortChainsForRules(backendId, row.chains ? row.chains.split(',').filter(Boolean) : [], [rule]),+ rules: [rule],+ })) as DomainStats[];+ }++ getRuleIPs(backendId: number, rule: string, limit = 50, start?: string, end?: string): IPStats[] {+ const range = this.parseMinuteRange(start, end);+ if (range) {+ const stmt = this.db.prepare(`+ SELECT m.ip, SUM(m.upload) as totalUpload, SUM(m.download) as totalDownload,+ SUM(m.connections) as totalConnections, MAX(m.minute) as lastSeen,+ GROUP_CONCAT(DISTINCT CASE WHEN m.domain != '' THEN m.domain END) as domains,+ GROUP_CONCAT(DISTINCT m.chain) as chains,+ COALESCE(i.asn, g.asn) as asn,+ CASE WHEN g.country IS NOT NULL THEN json_array(g.country, COALESCE(g.country_name, g.country), COALESCE(g.city, ''), COALESCE(g.as_name, ''))+ WHEN i.geoip IS NOT NULL THEN json(i.geoip) ELSE NULL END as geoIPData+ FROM minute_dim_stats m+ LEFT JOIN ip_stats i ON m.backend_id = i.backend_id AND m.ip = i.ip+ LEFT JOIN geoip_cache g ON m.ip = g.ip+ WHERE m.backend_id = ? AND m.minute >= ? AND m.minute <= ? AND m.rule = ? AND m.ip != ''+ GROUP BY m.ip ORDER BY (SUM(m.upload) + SUM(m.download)) DESC LIMIT ?+ `);+ const rows = stmt.all(backendId, range.startMinute, range.endMinute, rule, limit) as Array<{+ ip: string; totalUpload: number; totalDownload: number; totalConnections: number;+ lastSeen: string; domains: string | null; chains: string | null; asn: string | null; geoIPData: string | null;+ }>;++ return rows.map(row => {+ const chains = row.chains ? row.chains.split(',').filter(Boolean) : [];+ return {+ ip: row.ip, totalUpload: row.totalUpload, totalDownload: row.totalDownload,+ totalConnections: row.totalConnections, lastSeen: row.lastSeen,+ domains: row.domains ? row.domains.split(',').filter(Boolean) : [],+ chains: this.expandShortChainsForRules(backendId, chains, [rule]),+ asn: row.asn || undefined,+ geoIP: row.geoIPData ? JSON.parse(row.geoIPData).filter(Boolean) : undefined,+ };+ }) as IPStats[];+ }++ const stmt = this.db.prepare(`+ SELECT rit.ip, rit.total_upload as totalUpload, rit.total_download as totalDownload,+ rit.total_connections as totalConnections, rit.last_seen as lastSeen,+ i.domains, i.chains, COALESCE(i.asn, g.asn) as asn,+ CASE WHEN g.country IS NOT NULL THEN json_array(g.country, COALESCE(g.country_name, g.country), COALESCE(g.city, ''), COALESCE(g.as_name, ''))+ ELSE NULL END as geoIPData+ FROM rule_ip_traffic rit+ LEFT JOIN ip_stats i ON rit.backend_id = i.backend_id AND rit.ip = i.ip+ LEFT JOIN geoip_cache g ON rit.ip = g.ip+ WHERE rit.backend_id = ? AND rit.rule = ?+ ORDER BY (rit.total_upload + rit.total_download) DESC LIMIT ?+ `);+ const rows = stmt.all(backendId, rule, limit) as Array<{+ ip: string; totalUpload: number; totalDownload: number; totalConnections: number;+ lastSeen: string; domains: string | null; chains: string | null; asn: string | null; geoIPData: string | null;+ }>;++ return rows.map(row => ({+ ip: row.ip, totalUpload: row.totalUpload, totalDownload: row.totalDownload,+ totalConnections: row.totalConnections, lastSeen: row.lastSeen,+ domains: row.domains ? row.domains.split(',').filter(Boolean) : [],+ chains: this.expandShortChainsForRules(backendId, row.chains ? row.chains.split(',').filter(Boolean) : [], [rule]),+ asn: row.asn || undefined,+ geoIP: row.geoIPData ? JSON.parse(row.geoIPData).filter(Boolean) : undefined,+ })) as IPStats[];+ }++ getRuleDomainProxyStats(backendId: number, rule: string, domain: string, start?: string, end?: string): ProxyTrafficStats[] {+ const range = this.parseMinuteRange(start, end);+ if (range) {+ const stmt = this.db.prepare(`+ SELECT chain, SUM(upload) as totalUpload, SUM(download) as totalDownload, SUM(connections) as totalConnections+ FROM minute_dim_stats+ WHERE backend_id = ? AND minute >= ? AND minute <= ? AND rule = ? AND domain = ?+ GROUP BY chain ORDER BY (SUM(upload) + SUM(download)) DESC+ `);+ return stmt.all(backendId, range.startMinute, range.endMinute, rule, domain) as ProxyTrafficStats[];+ }+ return [];+ }++ getRuleDomainIPDetails(backendId: number, rule: string, domain: string, start?: string, end?: string, limit = 100): IPStats[] {+ const range = this.parseMinuteRange(start, end);+ const conditions = ["m.backend_id = ?", "m.rule = ?", "m.domain = ?", "m.ip != ''"];+ const params: Array<string | number> = [backendId, rule, domain];+ if (range) {+ conditions.push("m.minute >= ?", "m.minute <= ?");+ params.push(range.startMinute, range.endMinute);+ }++ const stmt = this.db.prepare(`+ SELECT m.ip, GROUP_CONCAT(DISTINCT CASE WHEN m.domain != '' THEN m.domain END) as domains,+ SUM(m.upload) as totalUpload, SUM(m.download) as totalDownload,+ SUM(m.connections) as totalConnections, MAX(m.minute) as lastSeen,+ COALESCE(i.asn, g.asn) as asn,+ CASE WHEN g.country IS NOT NULL THEN json_array(g.country, COALESCE(g.country_name, g.country), COALESCE(g.city, ''), COALESCE(g.as_name, ''))+ WHEN i.geoip IS NOT NULL THEN json(i.geoip) ELSE NULL END as geoIP,+ GROUP_CONCAT(DISTINCT m.chain) as chains+ FROM minute_dim_stats m+ LEFT JOIN ip_stats i ON m.backend_id = i.backend_id AND m.ip = i.ip+ LEFT JOIN geoip_cache g ON m.ip = g.ip+ WHERE ${conditions.join(" AND ")}+ GROUP BY m.ip ORDER BY (SUM(m.upload) + SUM(m.download)) DESC LIMIT ?+ `);++ const rows = stmt.all(...params, limit) as Array<{+ ip: string; domains: string; totalUpload: number; totalDownload: number;+ totalConnections: number; lastSeen: string; asn: string | null; geoIP: string | null; chains: string | null;+ }>;++ return rows.map(row => {+ const chains = row.chains ? row.chains.split(',').filter(Boolean) : [];+ return {+ ...row, domains: row.domains ? row.domains.split(',').filter(Boolean) : [],+ geoIP: row.geoIP ? JSON.parse(row.geoIP).filter(Boolean) : undefined,+ asn: row.asn || undefined,+ chains: this.expandShortChainsForRules(backendId, chains, [rule]),+ };+ }) as IPStats[];+ }++ getRuleIPProxyStats(backendId: number, rule: string, ip: string, start?: string, end?: string): ProxyTrafficStats[] {+ const range = this.parseMinuteRange(start, end);+ if (range) {+ const stmt = this.db.prepare(`+ SELECT chain, SUM(upload) as totalUpload, SUM(download) as totalDownload, SUM(connections) as totalConnections+ FROM minute_dim_stats+ WHERE backend_id = ? AND minute >= ? AND minute <= ? AND rule = ? AND ip = ?+ GROUP BY chain ORDER BY (SUM(upload) + SUM(download)) DESC+ `);+ return stmt.all(backendId, range.startMinute, range.endMinute, rule, ip) as ProxyTrafficStats[];+ }+ return [];+ }++ getRuleIPDomainDetails(backendId: number, rule: string, ip: string, start?: string, end?: string, limit = 100): DomainStats[] {+ const range = this.parseMinuteRange(start, end);+ const conditions = ["backend_id = ?", "rule = ?", "ip = ?", "domain != ''"];+ const params: Array<string | number> = [backendId, rule, ip];+ if (range) {+ conditions.push("minute >= ?", "minute <= ?");+ params.push(range.startMinute, range.endMinute);+ }++ const stmt = this.db.prepare(`+ SELECT domain, GROUP_CONCAT(DISTINCT ip) as ips,+ SUM(upload) as totalUpload, SUM(download) as totalDownload,+ SUM(connections) as totalConnections, MAX(minute) as lastSeen,+ GROUP_CONCAT(DISTINCT CASE WHEN rule != '' THEN rule END) as rules,+ GROUP_CONCAT(DISTINCT chain) as chains+ FROM minute_dim_stats WHERE ${conditions.join(" AND ")}+ GROUP BY domain ORDER BY (SUM(upload) + SUM(download)) DESC LIMIT ?+ `);++ const rows = stmt.all(...params, limit) as Array<{+ domain: string; ips: string | null; totalUpload: number; totalDownload: number;+ totalConnections: number; lastSeen: string; rules: string | null; chains: string | null;+ }>;++ return rows.map(row => {+ const rules = row.rules ? row.rules.split(',').filter(Boolean) : [];+ const chains = row.chains ? row.chains.split(',').filter(Boolean) : [];+ return {+ domain: row.domain, ips: row.ips ? row.ips.split(',').filter(Boolean) : [],+ totalUpload: row.totalUpload, totalDownload: row.totalDownload,+ totalConnections: row.totalConnections, lastSeen: row.lastSeen, rules,+ chains: this.expandShortChainsForRules(backendId, chains, rules),+ };+ }) as DomainStats[];+ }++ getRuleChainFlow(+ backendId: number, rule: string, start?: string, end?: string,+ ): { nodes: Array<{ name: string; totalUpload: number; totalDownload: number; totalConnections: number }>; links: Array<{ source: number; target: number }> } {+ const range = this.parseMinuteRange(start, end);+ let rows: Array<{ chain: string; totalUpload: number; totalDownload: number; totalConnections: number }>;++ if (range) {+ const stmt = this.db.prepare(`+ SELECT chain, SUM(upload) as totalUpload, SUM(download) as totalDownload, SUM(connections) as totalConnections+ FROM minute_dim_stats+ WHERE backend_id = ? AND minute >= ? AND minute <= ? AND rule = ? AND chain != ''+ GROUP BY chain+ `);+ rows = stmt.all(backendId, range.startMinute, range.endMinute, rule) as typeof rows;++ const baselineStmt = this.db.prepare(`+ SELECT rule, chain, total_upload as totalUpload, total_download as totalDownload, total_connections as totalConnections+ FROM rule_chain_traffic WHERE backend_id = ? AND rule = ?+ `);+ const baselineRows = baselineStmt.all(backendId, rule) as Array<{+ rule: string; chain: string; totalUpload: number; totalDownload: number; totalConnections: number;+ }>;+ const remapped = this.remapRangeRowsToFullChains(+ rows.map(r => ({ rule, chain: r.chain, totalUpload: r.totalUpload, totalDownload: r.totalDownload, totalConnections: r.totalConnections })),+ baselineRows,+ );+ rows = remapped.map(r => ({ chain: r.chain, totalUpload: r.totalUpload, totalDownload: r.totalDownload, totalConnections: r.totalConnections }));+ } else {+ const stmt = this.db.prepare(`+ SELECT chain, total_upload as totalUpload, total_download as totalDownload, total_connections as totalConnections+ FROM rule_chain_traffic WHERE backend_id = ? AND rule = ?+ `);+ rows = stmt.all(backendId, rule) as typeof rows;+ }++ const nodeMap = new Map<string, { name: string; totalUpload: number; totalDownload: number; totalConnections: number }>();+ const linkSet = new Set<string>();++ for (const row of rows) {+ const flowPath = this.buildRuleFlowPath(rule, row.chain);+ if (flowPath.length < 2) continue;+ for (let i = 0; i < flowPath.length; i++) {+ const nodeName = flowPath[i];+ if (!nodeMap.has(nodeName)) {+ nodeMap.set(nodeName, { name: nodeName, totalUpload: 0, totalDownload: 0, totalConnections: 0 });+ }+ const node = nodeMap.get(nodeName)!;+ node.totalUpload += row.totalUpload;+ node.totalDownload += row.totalDownload;+ node.totalConnections += row.totalConnections;+ }+ for (let i = 0; i < flowPath.length - 1; i++) {+ linkSet.add(`${flowPath[i]}|${flowPath[i + 1]}`);+ }+ }++ const nodes = Array.from(nodeMap.values());+ const nodeIndexMap = new Map(nodes.map((n, i) => [n.name, i]));+ const links = Array.from(linkSet).map(linkStr => {+ const [sourceName, targetName] = linkStr.split('|');+ return { source: nodeIndexMap.get(sourceName)!, target: nodeIndexMap.get(targetName)! };+ });++ return { nodes, links };+ }++ getAllRuleChainFlows(+ backendId: number, start?: string, end?: string,+ ): {+ nodes: Array<{ name: string; layer: number; nodeType: 'rule' | 'group' | 'proxy'; totalUpload: number; totalDownload: number; totalConnections: number; rules: string[] }>;+ links: Array<{ source: number; target: number; rules: string[] }>;+ rulePaths: Record<string, { nodeIndices: number[]; linkIndices: number[] }>;+ maxLayer: number;+ } {+ const range = this.parseMinuteRange(start, end);+ let rows: Array<{ rule: string; chain: string; totalUpload: number; totalDownload: number; totalConnections: number }>;++ if (range) {+ const stmt = this.db.prepare(`+ SELECT rule, chain, SUM(upload) as totalUpload, SUM(download) as totalDownload, SUM(connections) as totalConnections+ FROM minute_dim_stats+ WHERE backend_id = ? AND minute >= ? AND minute <= ? AND rule != '' AND chain != ''+ GROUP BY rule, chain ORDER BY rule, chain+ `);+ rows = stmt.all(backendId, range.startMinute, range.endMinute) as typeof rows;++ const baselineStmt = this.db.prepare(`+ SELECT rule, chain, total_upload as totalUpload, total_download as totalDownload, total_connections as totalConnections+ FROM rule_chain_traffic WHERE backend_id = ?+ `);+ const baselineRows = baselineStmt.all(backendId) as typeof rows;+ rows = this.remapRangeRowsToFullChains(rows, baselineRows);+ } else {+ const stmt = this.db.prepare(`+ SELECT rule, chain, total_upload as totalUpload, total_download as totalDownload, total_connections as totalConnections+ FROM rule_chain_traffic WHERE backend_id = ? ORDER BY rule, chain+ `);+ rows = stmt.all(backendId) as typeof rows;+ }++ const nodeMap = new Map<string, {+ totalUpload: number; totalDownload: number; totalConnections: number;+ rules: Set<string>; layer: number;+ }>();+ const linkMap = new Map<string, Set<string>>();+ const rulePathNodes = new Map<string, Set<string>>();+ const rulePathLinks = new Map<string, Set<string>>();++ for (const row of rows) {+ const rule = row.rule;+ if (!rulePathNodes.has(rule)) {+ rulePathNodes.set(rule, new Set());+ rulePathLinks.set(rule, new Set());+ }++ const flowPath = this.buildRuleFlowPath(rule, row.chain);+ if (flowPath.length < 2) continue;++ for (let i = 0; i < flowPath.length; i++) {+ const nodeName = flowPath[i];+ if (!nodeMap.has(nodeName)) {+ nodeMap.set(nodeName, { totalUpload: 0, totalDownload: 0, totalConnections: 0, rules: new Set(), layer: i });+ }+ const node = nodeMap.get(nodeName)!;+ node.totalUpload += row.totalUpload;+ node.totalDownload += row.totalDownload;+ node.totalConnections += row.totalConnections;+ node.rules.add(rule);+ node.layer = Math.max(node.layer, i);+ rulePathNodes.get(rule)!.add(nodeName);+ }++ for (let i = 0; i < flowPath.length - 1; i++) {+ const linkKey = `${flowPath[i]}|${flowPath[i + 1]}`;+ if (!linkMap.has(linkKey)) linkMap.set(linkKey, new Set());+ linkMap.get(linkKey)!.add(rule);+ rulePathLinks.get(rule)!.add(linkKey);+ }+ }++ // Determine node types and fix layer assignments+ const nodeEntries = Array.from(nodeMap.entries());+ const nodeTypeMap = new Map<string, 'rule' | 'group' | 'proxy'>();+ let computedMaxLayer = 0;++ const isBuiltInPolicy = (name: string): boolean =>+ name === 'DIRECT' || name === 'REJECT' || name === 'REJECT-TINY';++ for (const [name, data] of nodeEntries) {+ const hasOutgoing = Array.from(linkMap.keys()).some(k => k.startsWith(name + '|'));+ const hasIncoming = Array.from(linkMap.keys()).some(k => k.endsWith('|' + name));++ if (!hasIncoming) {+ nodeTypeMap.set(name, 'rule');+ data.layer = 0;+ } else if (!hasOutgoing && !isBuiltInPolicy(name)) {+ nodeTypeMap.set(name, 'proxy');+ } else {+ nodeTypeMap.set(name, 'group');+ }+ computedMaxLayer = Math.max(computedMaxLayer, data.layer);+ }++ // Force all proxy nodes to the rightmost column+ for (const [name, data] of nodeEntries) {+ if (nodeTypeMap.get(name) === 'proxy') data.layer = computedMaxLayer;+ }++ // Stable ordering for ReactFlow+ const nodeTypeOrder = (type: 'rule' | 'group' | 'proxy'): number =>+ type === 'rule' ? 0 : type === 'group' ? 1 : 2;++ const sortedNodeEntries = [...nodeEntries].sort(([nameA, dataA], [nameB, dataB]) => {+ if (dataA.layer !== dataB.layer) return dataA.layer - dataB.layer;+ const typeDiff = nodeTypeOrder(nodeTypeMap.get(nameA)!) - nodeTypeOrder(nodeTypeMap.get(nameB)!);+ if (typeDiff !== 0) return typeDiff;+ return nameA.localeCompare(nameB);+ });++ const nodes = sortedNodeEntries.map(([name, data]) => ({+ name, layer: data.layer, nodeType: nodeTypeMap.get(name)!,+ totalUpload: data.totalUpload, totalDownload: data.totalDownload,+ totalConnections: data.totalConnections, rules: Array.from(data.rules),+ }));++ const nodeIndexMap = new Map(nodes.map((n, i) => [n.name, i]));++ const links = Array.from(linkMap.entries())+ .sort(([keyA], [keyB]) => keyA.localeCompare(keyB))+ .map(([key, rules]) => {+ const [sourceName, targetName] = key.split('|');+ return { source: nodeIndexMap.get(sourceName)!, target: nodeIndexMap.get(targetName)!, rules: Array.from(rules) };+ });++ const rulePaths: Record<string, { nodeIndices: number[]; linkIndices: number[] }> = {};+ for (const [rule, nodeNames] of rulePathNodes) {+ const nodeIndices = Array.from(nodeNames).map(n => nodeIndexMap.get(n)!).filter(i => i !== undefined);+ const linkIndices: number[] = [];+ const linkKeys = rulePathLinks.get(rule)!;+ links.forEach((link, idx) => {+ const sourceName = nodes[link.source].name;+ const targetName = nodes[link.target].name;+ if (linkKeys.has(`${sourceName}|${targetName}`)) linkIndices.push(idx);+ });+ rulePaths[rule] = { nodeIndices, linkIndices };+ }++ const maxLayer = nodes.reduce((max, n) => Math.max(max, n.layer), 0);+ return { nodes, links, rulePaths, maxLayer };+ }+}diff --git a/apps/collector/src/database/repositories/surge.repository.ts b/apps/collector/src/database/repositories/surge.repository.tsnew file mode 100644index 0000000..ec5e063--- /dev/null+++ b/apps/collector/src/database/repositories/surge.repository.ts@@ -0,0 +1,83 @@+/**+ * Surge Repository+ *+ * Handles Surge policy cache storage and retrieval.+ */+import type Database from 'better-sqlite3';+import { BaseRepository } from './base.repository.js';++export class SurgeRepository extends BaseRepository {+ constructor(db: Database.Database) {+ super(db);+ }++ getSurgePolicyCache(backendId: number): Array<{+ policyGroup: string;+ selectedPolicy: string | null;+ policyType: string;+ allPolicies: string[] | null;+ updatedAt: string;+ }> {+ const stmt = this.db.prepare(`+ SELECT policy_group as policyGroup,+ selected_policy as selectedPolicy,+ policy_type as policyType,+ all_policies as allPolicies,+ updated_at as updatedAt+ FROM surge_policy_cache+ WHERE backend_id = ?+ ORDER BY policy_group+ `);+ const rows = stmt.all(backendId) as any[];+ return rows.map(r => ({+ ...r,+ allPolicies: r.allPolicies ? JSON.parse(r.allPolicies) : null,+ }));+ }++ updateSurgePolicyCache(+ backendId: number,+ policies: Array<{+ policyGroup: string;+ selectedPolicy: string | null;+ policyType?: string;+ allPolicies?: string[];+ }>,+ ): void {+ const insert = this.db.prepare(`+ INSERT INTO surge_policy_cache+ (backend_id, policy_group, selected_policy, policy_type, all_policies, updated_at)+ VALUES+ (?, ?, ?, ?, ?, datetime('now'))+ ON CONFLICT(backend_id, policy_group) DO UPDATE SET+ selected_policy = excluded.selected_policy,+ policy_type = excluded.policy_type,+ all_policies = excluded.all_policies,+ updated_at = excluded.updated_at+ `);++ const transaction = this.db.transaction((items: typeof policies) => {+ for (const item of items) {+ insert.run(+ backendId,+ item.policyGroup,+ item.selectedPolicy,+ item.policyType || 'Select',+ item.allPolicies ? JSON.stringify(item.allPolicies) : null,+ );+ }+ });++ transaction(policies);+ }++ getSurgePolicyCacheLastUpdate(backendId: number): string | null {+ const stmt = this.db.prepare(`SELECT MAX(updated_at) as lastUpdate FROM surge_policy_cache WHERE backend_id = ?`);+ const row = stmt.get(backendId) as any;+ return row?.lastUpdate || null;+ }++ clearSurgePolicyCache(backendId: number): void {+ this.db.prepare(`DELETE FROM surge_policy_cache WHERE backend_id = ?`).run(backendId);+ }+}diff --git a/apps/collector/src/database/repositories/timeseries.repository.ts b/apps/collector/src/database/repositories/timeseries.repository.tsnew file mode 100644index 0000000..3deb545--- /dev/null+++ b/apps/collector/src/database/repositories/timeseries.repository.ts@@ -0,0 +1,127 @@+/**+ * Timeseries Repository+ *+ * Handles time-based traffic queries: hourly stats, today traffic,+ * traffic in range, traffic trend and aggregated trend.+ */+import type Database from 'better-sqlite3';+import type { HourlyStats } from '@neko-master/shared';+import { BaseRepository } from './base.repository.js';++export class TimeseriesRepository extends BaseRepository {+ constructor(db: Database.Database) {+ super(db);+ }++ getHourlyStats(backendId: number, hours = 24, start?: string, end?: string): HourlyStats[] {+ const range = this.parseMinuteRange(start, end);+ if (range) {+ const stmt = this.db.prepare(`+ SELECT+ substr(minute, 1, 13) || ':00:00' as hour,+ SUM(upload) as upload,+ SUM(download) as download,+ SUM(connections) as connections+ FROM minute_stats+ WHERE backend_id = ? AND minute >= ? AND minute <= ?+ GROUP BY substr(minute, 1, 13)+ ORDER BY hour DESC+ LIMIT ?+ `);+ return stmt.all(backendId, range.startMinute, range.endMinute, hours) as HourlyStats[];+ }++ const stmt = this.db.prepare(`+ SELECT hour, upload, download, connections+ FROM hourly_stats+ WHERE backend_id = ?+ ORDER BY hour DESC+ LIMIT ?+ `);+ return stmt.all(backendId, hours) as HourlyStats[];+ }++ getTodayTraffic(backendId: number): { upload: number; download: number } {+ const today = new Date().toISOString().split('T')[0];+ const stmt = this.db.prepare(`+ SELECT COALESCE(SUM(upload), 0) as upload, COALESCE(SUM(download), 0) as download+ FROM hourly_stats+ WHERE backend_id = ? AND hour >= ?+ `);+ return stmt.get(backendId, today) as { upload: number; download: number };+ }++ getTrafficInRange(backendId: number, start?: string, end?: string): { upload: number; download: number } {+ const range = this.parseMinuteRange(start, end);+ if (!range) {+ return this.getTodayTraffic(backendId);+ }++ const stmt = this.db.prepare(`+ SELECT+ COALESCE(SUM(upload), 0) as upload,+ COALESCE(SUM(download), 0) as download+ FROM minute_stats+ WHERE backend_id = ? AND minute >= ? AND minute <= ?+ `);+ return stmt.get(backendId, range.startMinute, range.endMinute) as { upload: number; download: number };+ }++ getTrafficTrend(+ backendId: number,+ minutes = 30,+ start?: string,+ end?: string,+ ): Array<{ time: string; upload: number; download: number }> {+ const range = this.parseMinuteRange(start, end);+ const cutoff = new Date(Date.now() - minutes * 60 * 1000);+ const cutoffStr = cutoff.toISOString().slice(0, 16) + ':00';+ const stmt = this.db.prepare(`+ SELECT minute as time, upload, download+ FROM minute_stats+ WHERE backend_id = ? AND minute >= ? AND minute <= ?+ ORDER BY minute ASC+ `);+ return stmt.all(+ backendId,+ range?.startMinute || cutoffStr,+ range?.endMinute || this.toMinuteKey(new Date()),+ ) as Array<{ time: string; upload: number; download: number }>;+ }++ getTrafficTrendAggregated(+ backendId: number,+ minutes = 30,+ bucketMinutes = 1,+ start?: string,+ end?: string,+ ): Array<{ time: string; upload: number; download: number }> {+ const range = this.parseMinuteRange(start, end);+ const cutoff = new Date(Date.now() - minutes * 60 * 1000);+ const cutoffStr = cutoff.toISOString().slice(0, 16) + ':00';+ const endMinute = range?.endMinute || this.toMinuteKey(new Date());++ if (bucketMinutes <= 1) {+ const stmt = this.db.prepare(`+ SELECT minute as time, upload, download+ FROM minute_stats+ WHERE backend_id = ? AND minute >= ? AND minute <= ?+ ORDER BY minute ASC+ `);+ return stmt.all(backendId, range?.startMinute || cutoffStr, endMinute) as Array<{ time: string; upload: number; download: number }>;+ }++ const bucketExpr = `strftime('%Y-%m-%dT%H:%M:00', datetime((strftime('%s', datetime(minute)) / ${bucketMinutes * 60}) * ${bucketMinutes * 60}, 'unixepoch'))`;+ const stmt = this.db.prepare(`+ SELECT+ ${bucketExpr} as time,+ SUM(upload) as upload,+ SUM(download) as download+ FROM minute_stats+ WHERE backend_id = ? AND minute >= ? AND minute <= ?+ GROUP BY ${bucketExpr}+ ORDER BY time ASC+ `);+ return stmt.all(backendId, range?.startMinute || cutoffStr, endMinute) as Array<{ time: string; upload: number; download: number }>;+ }+}diff --git a/apps/collector/src/database/repositories/traffic-writer.repository.ts b/apps/collector/src/database/repositories/traffic-writer.repository.tsnew file mode 100644index 0000000..60aa4d6--- /dev/null+++ b/apps/collector/src/database/repositories/traffic-writer.repository.ts@@ -0,0 +1,459 @@+/**+ * Traffic Writer Repository+ *+ * Handles writing traffic data to the database. Contains the two main+ * write methods: updateTrafficStats (single) and batchUpdateTrafficStats (batch).+ */+import type Database from 'better-sqlite3';+import { BaseRepository } from './base.repository.js';++export interface TrafficUpdate {+ domain: string;+ ip: string;+ chain: string;+ chains: string[];+ rule: string;+ rulePayload: string;+ upload: number;+ download: number;+ sourceIP?: string;+ timestampMs?: number;+}++export class TrafficWriterRepository extends BaseRepository {+ constructor(db: Database.Database) {+ super(db);+ }++ updateTrafficStats(backendId: number, update: TrafficUpdate) {+ const now = new Date();+ const timestamp = now.toISOString();+ const hour = timestamp.slice(0, 13) + ':00:00';++ if (update.upload === 0 && update.download === 0) return;++ const ruleName = update.chains.length > 1 ? update.chains[update.chains.length - 1] :+ update.rulePayload ? `${update.rule}(${update.rulePayload})` : update.rule;+ const finalProxy = update.chains.length > 0 ? update.chains[0] : 'DIRECT';+ const fullChain = update.chains.join(' > ') || update.chain || 'DIRECT';++ const transaction = this.db.transaction(() => {+ const domainName = update.domain || 'unknown';+ if (domainName !== 'unknown') {+ this.db.prepare(`+ INSERT INTO domain_stats (backend_id, domain, ips, total_upload, total_download, total_connections, last_seen, rules, chains)+ VALUES (@backendId, @domain, @ip, @upload, @download, 1, @timestamp, @rule, @chain)+ ON CONFLICT(backend_id, domain) DO UPDATE SET+ ips = CASE WHEN domain_stats.ips IS NULL THEN @ip WHEN INSTR(domain_stats.ips, @ip) > 0 THEN domain_stats.ips ELSE domain_stats.ips || ',' || @ip END,+ total_upload = total_upload + @upload, total_download = total_download + @download,+ total_connections = total_connections + 1, last_seen = @timestamp,+ rules = CASE WHEN domain_stats.rules IS NULL THEN @rule WHEN INSTR(domain_stats.rules, @rule) > 0 THEN domain_stats.rules ELSE domain_stats.rules || ',' || @rule END,+ chains = CASE WHEN domain_stats.chains IS NULL THEN @chain WHEN INSTR(domain_stats.chains, @chain) > 0 THEN domain_stats.chains ELSE domain_stats.chains || ',' || @chain END+ `).run({ backendId, domain: domainName, ip: update.ip, upload: update.upload, download: update.download, timestamp, rule: ruleName, chain: fullChain });+ }++ this.db.prepare(`+ INSERT INTO ip_stats (backend_id, ip, domains, total_upload, total_download, total_connections, last_seen, chains, rules)+ VALUES (@backendId, @ip, @domain, @upload, @download, 1, @timestamp, @chain, @rule)+ ON CONFLICT(backend_id, ip) DO UPDATE SET+ domains = CASE WHEN ip_stats.domains IS NULL THEN @domain WHEN INSTR(ip_stats.domains, @domain) > 0 THEN ip_stats.domains ELSE ip_stats.domains || ',' || @domain END,+ total_upload = total_upload + @upload, total_download = total_download + @download,+ total_connections = total_connections + 1, last_seen = @timestamp,+ chains = CASE WHEN ip_stats.chains IS NULL THEN @chain WHEN INSTR(ip_stats.chains, @chain) > 0 THEN ip_stats.chains ELSE ip_stats.chains || ',' || @chain END,+ rules = CASE WHEN ip_stats.rules IS NULL THEN @rule WHEN INSTR(ip_stats.rules, @rule) > 0 THEN ip_stats.rules ELSE ip_stats.rules || ',' || @rule END+ `).run({ backendId, ip: update.ip, domain: update.domain || 'unknown', upload: update.upload, download: update.download, timestamp, chain: fullChain, rule: ruleName });++ this.db.prepare(`+ INSERT INTO proxy_stats (backend_id, chain, total_upload, total_download, total_connections, last_seen)+ VALUES (@backendId, @chain, @upload, @download, 1, @timestamp)+ ON CONFLICT(backend_id, chain) DO UPDATE SET+ total_upload = total_upload + @upload, total_download = total_download + @download,+ total_connections = total_connections + 1, last_seen = @timestamp+ `).run({ backendId, chain: fullChain, upload: update.upload, download: update.download, timestamp });++ this.db.prepare(`+ INSERT INTO rule_stats (backend_id, rule, final_proxy, total_upload, total_download, total_connections, last_seen)+ VALUES (@backendId, @rule, @finalProxy, @upload, @download, 1, @timestamp)+ ON CONFLICT(backend_id, rule) DO UPDATE SET+ final_proxy = @finalProxy, total_upload = total_upload + @upload, total_download = total_download + @download,+ total_connections = total_connections + 1, last_seen = @timestamp+ `).run({ backendId, rule: ruleName, finalProxy, upload: update.upload, download: update.download, timestamp });++ this.db.prepare(`+ INSERT INTO rule_chain_traffic (backend_id, rule, chain, total_upload, total_download, total_connections, last_seen)+ VALUES (@backendId, @rule, @chain, @upload, @download, 1, @timestamp)+ ON CONFLICT(backend_id, rule, chain) DO UPDATE SET+ total_upload = total_upload + @upload, total_download = total_download + @download,+ total_connections = total_connections + 1, last_seen = @timestamp+ `).run({ backendId, rule: ruleName, chain: fullChain, upload: update.upload, download: update.download, timestamp });++ if (domainName !== 'unknown') {+ this.db.prepare(`+ INSERT INTO rule_domain_traffic (backend_id, rule, domain, total_upload, total_download, total_connections, last_seen)+ VALUES (@backendId, @rule, @domain, @upload, @download, 1, @timestamp)+ ON CONFLICT(backend_id, rule, domain) DO UPDATE SET+ total_upload = total_upload + @upload, total_download = total_download + @download,+ total_connections = total_connections + 1, last_seen = @timestamp+ `).run({ backendId, rule: ruleName, domain: domainName, upload: update.upload, download: update.download, timestamp });+ }++ this.db.prepare(`+ INSERT INTO rule_ip_traffic (backend_id, rule, ip, total_upload, total_download, total_connections, last_seen)+ VALUES (@backendId, @rule, @ip, @upload, @download, 1, @timestamp)+ ON CONFLICT(backend_id, rule, ip) DO UPDATE SET+ total_upload = total_upload + @upload, total_download = total_download + @download,+ total_connections = total_connections + 1, last_seen = @timestamp+ `).run({ backendId, rule: ruleName, ip: update.ip, upload: update.upload, download: update.download, timestamp });++ if (update.chains.length > 1) {+ this.db.prepare(`INSERT OR IGNORE INTO rule_proxy_map (backend_id, rule, proxy) VALUES (@backendId, @rule, @proxy)`)+ .run({ backendId, rule: update.chains[update.chains.length - 1], proxy: update.chains[0] });+ }++ this.db.prepare(`+ INSERT INTO hourly_stats (backend_id, hour, upload, download, connections) VALUES (@backendId, @hour, @upload, @download, 1)+ ON CONFLICT(backend_id, hour) DO UPDATE SET upload = upload + @upload, download = download + @download, connections = connections + 1+ `).run({ backendId, hour, upload: update.upload, download: update.download });++ const minute = timestamp.slice(0, 16) + ':00';+ this.db.prepare(`+ INSERT INTO minute_stats (backend_id, minute, upload, download, connections) VALUES (@backendId, @minute, @upload, @download, 1)+ ON CONFLICT(backend_id, minute) DO UPDATE SET upload = upload + @upload, download = download + @download, connections = connections + 1+ `).run({ backendId, minute, upload: update.upload, download: update.download });++ if (domainName !== 'unknown') {+ this.db.prepare(`+ INSERT INTO domain_proxy_stats (backend_id, domain, chain, total_upload, total_download, total_connections, last_seen)+ VALUES (@backendId, @domain, @chain, @upload, @download, 1, @timestamp)+ ON CONFLICT(backend_id, domain, chain) DO UPDATE SET+ total_upload = total_upload + @upload, total_download = total_download + @download,+ total_connections = total_connections + 1, last_seen = @timestamp+ `).run({ backendId, domain: domainName, chain: fullChain, upload: update.upload, download: update.download, timestamp });+ }++ this.db.prepare(`+ INSERT INTO ip_proxy_stats (backend_id, ip, chain, total_upload, total_download, total_connections, last_seen, domains)+ VALUES (@backendId, @ip, @chain, @upload, @download, 1, @timestamp, @domain)+ ON CONFLICT(backend_id, ip, chain) DO UPDATE SET+ total_upload = total_upload + @upload, total_download = total_download + @download,+ total_connections = total_connections + 1, last_seen = @timestamp,+ domains = CASE WHEN ip_proxy_stats.domains IS NULL THEN @domain WHEN @domain = 'unknown' THEN ip_proxy_stats.domains+ WHEN INSTR(ip_proxy_stats.domains, @domain) > 0 THEN ip_proxy_stats.domains ELSE ip_proxy_stats.domains || ',' || @domain END+ `).run({ backendId, ip: update.ip, chain: fullChain, upload: update.upload, download: update.download, timestamp, domain: update.domain || 'unknown' });+ });++ transaction();+ }++ batchUpdateTrafficStats(backendId: number, updates: TrafficUpdate[]) {+ if (updates.length === 0) return;++ const now = new Date();+ const timestamp = now.toISOString();++ // Aggregate updates by domain, ip, chain to reduce UPSERT conflicts+ const domainMap = new Map<string, TrafficUpdate & { count: number }>();+ const ipMap = new Map<string, TrafficUpdate & { count: number }>();+ const chainMap = new Map<string, { chains: string[]; upload: number; download: number; count: number }>();+ const ruleProxyMap = new Map<string, { rule: string; proxy: string; upload: number; download: number; count: number }>();+ const hourlyMap = new Map<string, { upload: number; download: number; connections: number }>();+ const ruleChainMap = new Map<string, { rule: string; chain: string; upload: number; download: number; count: number }>();+ const ruleDomainMap = new Map<string, { rule: string; domain: string; upload: number; download: number; count: number }>();+ const ruleIPMap = new Map<string, { rule: string; ip: string; upload: number; download: number; count: number }>();+ const minuteMap = new Map<string, { upload: number; download: number; connections: number }>();+ const minuteDimMap = new Map<string, {+ minute: string; domain: string; ip: string; sourceIP: string;+ chain: string; rule: string; upload: number; download: number; connections: number;+ }>();+ const domainProxyMap = new Map<string, { domain: string; chain: string; upload: number; download: number; count: number }>();+ const ipProxyMap = new Map<string, { ip: string; chain: string; upload: number; download: number; count: number; domains: Set<string> }>();+ const deviceMap = new Map<string, { sourceIP: string; upload: number; download: number; count: number }>();+ const deviceDomainMap = new Map<string, { sourceIP: string; domain: string; upload: number; download: number; count: number }>();+ const deviceIPMap = new Map<string, { sourceIP: string; ip: string; upload: number; download: number; count: number }>();++ for (const update of updates) {+ if (update.upload === 0 && update.download === 0) continue;++ const ruleName = update.chains.length > 1 ? update.chains[update.chains.length - 1] :+ update.rulePayload ? `${update.rule}(${update.rulePayload})` : update.rule;+ const finalProxy = update.chains.length > 0 ? update.chains[0] : 'DIRECT';+ const fullChain = update.chains.join(' > ') || update.chain || 'DIRECT';+ const eventDate = new Date(update.timestampMs ?? now.getTime());+ const hourKey = this.toHourKey(eventDate);+ const minuteKey = this.toMinuteKey(eventDate);++ // Aggregate domain stats+ if (update.domain) {+ const domainKey = `${update.domain}:${update.ip}:${fullChain}`;+ const existing = domainMap.get(domainKey);+ if (existing) { existing.upload += update.upload; existing.download += update.download; existing.count++; }+ else { domainMap.set(domainKey, { ...update, count: 1 }); }+ }++ // Aggregate IP stats+ const ipKey = `${update.ip}:${update.domain}:${fullChain}`;+ const existingIp = ipMap.get(ipKey);+ if (existingIp) { existingIp.upload += update.upload; existingIp.download += update.download; existingIp.count++; }+ else { ipMap.set(ipKey, { ...update, rule: ruleName, count: 1 }); }++ // Aggregate chain stats+ const existingChain = chainMap.get(fullChain);+ if (existingChain) { existingChain.upload += update.upload; existingChain.download += update.download; existingChain.count++; }+ else { chainMap.set(fullChain, { chains: update.chains, upload: update.upload, download: update.download, count: 1 }); }++ // Aggregate rule stats+ const ruleKey = `${ruleName}:${finalProxy}`;+ const existingRule = ruleProxyMap.get(ruleKey);+ if (existingRule) { existingRule.upload += update.upload; existingRule.download += update.download; existingRule.count++; }+ else { ruleProxyMap.set(ruleKey, { rule: ruleName, proxy: finalProxy, upload: update.upload, download: update.download, count: 1 }); }++ // Aggregate hourly stats+ const existingHour = hourlyMap.get(hourKey);+ if (existingHour) { existingHour.upload += update.upload; existingHour.download += update.download; existingHour.connections++; }+ else { hourlyMap.set(hourKey, { upload: update.upload, download: update.download, connections: 1 }); }++ // Aggregate rule_chain_traffic+ const fullChainForRule = update.chains.join(' > ');+ const ruleChainKey = `${ruleName}:${fullChainForRule}`;+ const existingRuleChain = ruleChainMap.get(ruleChainKey);+ if (existingRuleChain) { existingRuleChain.upload += update.upload; existingRuleChain.download += update.download; existingRuleChain.count++; }+ else { ruleChainMap.set(ruleChainKey, { rule: ruleName, chain: fullChainForRule, upload: update.upload, download: update.download, count: 1 }); }++ // Aggregate rule_domain_traffic+ if (update.domain) {+ const rdKey = `${ruleName}:${update.domain}`;+ const existingRD = ruleDomainMap.get(rdKey);+ if (existingRD) { existingRD.upload += update.upload; existingRD.download += update.download; existingRD.count++; }+ else { ruleDomainMap.set(rdKey, { rule: ruleName, domain: update.domain, upload: update.upload, download: update.download, count: 1 }); }+ }++ // Aggregate rule_ip_traffic+ const riKey = `${ruleName}:${update.ip}`;+ const existingRI = ruleIPMap.get(riKey);+ if (existingRI) { existingRI.upload += update.upload; existingRI.download += update.download; existingRI.count++; }+ else { ruleIPMap.set(riKey, { rule: ruleName, ip: update.ip, upload: update.upload, download: update.download, count: 1 }); }++ // Aggregate minute_stats+ const existingMinute = minuteMap.get(minuteKey);+ if (existingMinute) { existingMinute.upload += update.upload; existingMinute.download += update.download; existingMinute.connections++; }+ else { minuteMap.set(minuteKey, { upload: update.upload, download: update.download, connections: 1 }); }++ // Aggregate minute_dim_stats+ const dimKey = `${minuteKey}:${update.domain || ''}:${update.ip || ''}:${update.sourceIP || ''}:${fullChain}:${ruleName}`;+ const existingDim = minuteDimMap.get(dimKey);+ if (existingDim) { existingDim.upload += update.upload; existingDim.download += update.download; existingDim.connections++; }+ else { minuteDimMap.set(dimKey, { minute: minuteKey, domain: update.domain || '', ip: update.ip || '', sourceIP: update.sourceIP || '', chain: fullChain, rule: ruleName, upload: update.upload, download: update.download, connections: 1 }); }++ // Aggregate domain_proxy_stats+ if (update.domain) {+ const dpKey = `${update.domain}:${fullChain}`;+ const existingDP = domainProxyMap.get(dpKey);+ if (existingDP) { existingDP.upload += update.upload; existingDP.download += update.download; existingDP.count++; }+ else { domainProxyMap.set(dpKey, { domain: update.domain, chain: fullChain, upload: update.upload, download: update.download, count: 1 }); }+ }++ // Aggregate ip_proxy_stats+ const ipPKey = `${update.ip}:${fullChain}`;+ const existingIPP = ipProxyMap.get(ipPKey);+ if (existingIPP) {+ existingIPP.upload += update.upload; existingIPP.download += update.download; existingIPP.count++;+ if (update.domain && update.domain !== 'unknown') existingIPP.domains.add(update.domain);+ } else {+ const domains = new Set<string>();+ if (update.domain && update.domain !== 'unknown') domains.add(update.domain);+ ipProxyMap.set(ipPKey, { ip: update.ip, chain: fullChain, upload: update.upload, download: update.download, count: 1, domains });+ }++ // Aggregate device stats+ if (update.sourceIP) {+ const sourceIP = update.sourceIP;+ const existingDevice = deviceMap.get(sourceIP);+ if (existingDevice) { existingDevice.upload += update.upload; existingDevice.download += update.download; existingDevice.count++; }+ else { deviceMap.set(sourceIP, { sourceIP, upload: update.upload, download: update.download, count: 1 }); }++ if (update.domain) {+ const ddKey = `${sourceIP}:${update.domain}`;+ const existingDD = deviceDomainMap.get(ddKey);+ if (existingDD) { existingDD.upload += update.upload; existingDD.download += update.download; existingDD.count++; }+ else { deviceDomainMap.set(ddKey, { sourceIP, domain: update.domain, upload: update.upload, download: update.download, count: 1 }); }+ }++ if (update.ip) {+ const diKey = `${sourceIP}:${update.ip}`;+ const existingDI = deviceIPMap.get(diKey);+ if (existingDI) { existingDI.upload += update.upload; existingDI.download += update.download; existingDI.count++; }+ else { deviceIPMap.set(diKey, { sourceIP, ip: update.ip, upload: update.upload, download: update.download, count: 1 }); }+ }+ }+ }++ // Sub-transaction 1: Core aggregation tables+ const tx1 = this.db.transaction(() => {+ const domainStmt = this.db.prepare(`+ INSERT INTO domain_stats (backend_id, domain, ips, total_upload, total_download, total_connections, last_seen, rules, chains)+ VALUES (@backendId, @domain, @ip, @upload, @download, @count, @timestamp, @rule, @chain)+ ON CONFLICT(backend_id, domain) DO UPDATE SET+ ips = CASE WHEN domain_stats.ips IS NULL THEN @ip WHEN LENGTH(domain_stats.ips) > 4000 THEN domain_stats.ips WHEN INSTR(domain_stats.ips, @ip) > 0 THEN domain_stats.ips ELSE domain_stats.ips || ',' || @ip END,+ total_upload = total_upload + @upload, total_download = total_download + @download,+ total_connections = total_connections + @count, last_seen = @timestamp,+ rules = CASE WHEN domain_stats.rules IS NULL THEN @rule WHEN LENGTH(domain_stats.rules) > 4000 THEN domain_stats.rules WHEN INSTR(domain_stats.rules, @rule) > 0 THEN domain_stats.rules ELSE domain_stats.rules || ',' || @rule END,+ chains = CASE WHEN domain_stats.chains IS NULL THEN @chain WHEN LENGTH(domain_stats.chains) > 4000 THEN domain_stats.chains WHEN INSTR(domain_stats.chains, @chain) > 0 THEN domain_stats.chains ELSE domain_stats.chains || ',' || @chain END+ `);+ for (const [, data] of domainMap) {+ const ruleName = data.chains.length > 1 ? data.chains[data.chains.length - 1] : data.rulePayload ? `${data.rule}(${data.rulePayload})` : data.rule;+ const fullChain = data.chains.join(' > ');+ domainStmt.run({ backendId, domain: data.domain, ip: data.ip, upload: data.upload, download: data.download, count: data.count, timestamp, rule: ruleName, chain: fullChain });+ }++ const ipStmt = this.db.prepare(`+ INSERT INTO ip_stats (backend_id, ip, domains, total_upload, total_download, total_connections, last_seen, chains, rules)+ VALUES (@backendId, @ip, @domain, @upload, @download, @count, @timestamp, @chain, @rule)+ ON CONFLICT(backend_id, ip) DO UPDATE SET+ domains = CASE WHEN ip_stats.domains IS NULL THEN @domain WHEN LENGTH(ip_stats.domains) > 4000 THEN ip_stats.domains WHEN INSTR(ip_stats.domains, @domain) > 0 THEN ip_stats.domains ELSE ip_stats.domains || ',' || @domain END,+ total_upload = total_upload + @upload, total_download = total_download + @download,+ total_connections = total_connections + @count, last_seen = @timestamp,+ chains = CASE WHEN ip_stats.chains IS NULL THEN @chain WHEN LENGTH(ip_stats.chains) > 4000 THEN ip_stats.chains WHEN INSTR(ip_stats.chains, @chain) > 0 THEN ip_stats.chains ELSE ip_stats.chains || ',' || @chain END,+ rules = CASE WHEN ip_stats.rules IS NULL THEN @rule WHEN LENGTH(ip_stats.rules) > 4000 THEN ip_stats.rules WHEN INSTR(ip_stats.rules, @rule) > 0 THEN ip_stats.rules ELSE ip_stats.rules || ',' || @rule END+ `);+ for (const [, data] of ipMap) {+ const fullChain = data.chains.join(' > ');+ ipStmt.run({ backendId, ip: data.ip, domain: data.domain || 'unknown', upload: data.upload, download: data.download, count: data.count, timestamp, chain: fullChain, rule: data.rule });+ }++ const proxyStmt = this.db.prepare(`+ INSERT INTO proxy_stats (backend_id, chain, total_upload, total_download, total_connections, last_seen)+ VALUES (@backendId, @chain, @upload, @download, @count, @timestamp)+ ON CONFLICT(backend_id, chain) DO UPDATE SET+ total_upload = total_upload + @upload, total_download = total_download + @download,+ total_connections = total_connections + @count, last_seen = @timestamp+ `);+ for (const [chain, data] of chainMap) { proxyStmt.run({ backendId, chain, upload: data.upload, download: data.download, count: data.count, timestamp }); }++ const ruleStmt = this.db.prepare(`+ INSERT INTO rule_stats (backend_id, rule, final_proxy, total_upload, total_download, total_connections, last_seen)+ VALUES (@backendId, @rule, @proxy, @upload, @download, @count, @timestamp)+ ON CONFLICT(backend_id, rule) DO UPDATE SET+ final_proxy = @proxy, total_upload = total_upload + @upload, total_download = total_download + @download,+ total_connections = total_connections + @count, last_seen = @timestamp+ `);+ for (const [, data] of ruleProxyMap) { ruleStmt.run({ backendId, rule: data.rule, proxy: data.proxy, upload: data.upload, download: data.download, count: data.count, timestamp }); }++ const ruleProxyStmt = this.db.prepare(`INSERT OR IGNORE INTO rule_proxy_map (backend_id, rule, proxy) VALUES (@backendId, @rule, @proxy)`);+ for (const [, data] of ruleProxyMap) { ruleProxyStmt.run({ backendId, rule: data.rule, proxy: data.proxy }); }++ const hourlyStmt = this.db.prepare(`+ INSERT INTO hourly_stats (backend_id, hour, upload, download, connections) VALUES (@backendId, @hour, @upload, @download, @connections)+ ON CONFLICT(backend_id, hour) DO UPDATE SET upload = upload + @upload, download = download + @download, connections = connections + @connections+ `);+ for (const [hour, data] of hourlyMap) { hourlyStmt.run({ backendId, hour, upload: data.upload, download: data.download, connections: data.connections }); }++ const ruleChainStmt = this.db.prepare(`+ INSERT INTO rule_chain_traffic (backend_id, rule, chain, total_upload, total_download, total_connections, last_seen)+ VALUES (@backendId, @rule, @chain, @upload, @download, @count, @timestamp)+ ON CONFLICT(backend_id, rule, chain) DO UPDATE SET+ total_upload = total_upload + @upload, total_download = total_download + @download,+ total_connections = total_connections + @count, last_seen = @timestamp+ `);+ for (const [, data] of ruleChainMap) { ruleChainStmt.run({ backendId, rule: data.rule, chain: data.chain, upload: data.upload, download: data.download, count: data.count, timestamp }); }+ });+ tx1();++ // Sub-transaction 2: Detail tables + minute/hourly tables+ const tx2 = this.db.transaction(() => {+ const ruleDomainStmt = this.db.prepare(`+ INSERT INTO rule_domain_traffic (backend_id, rule, domain, total_upload, total_download, total_connections, last_seen)+ VALUES (@backendId, @rule, @domain, @upload, @download, @count, @timestamp)+ ON CONFLICT(backend_id, rule, domain) DO UPDATE SET+ total_upload = total_upload + @upload, total_download = total_download + @download,+ total_connections = total_connections + @count, last_seen = @timestamp+ `);+ for (const [, data] of ruleDomainMap) { ruleDomainStmt.run({ backendId, rule: data.rule, domain: data.domain, upload: data.upload, download: data.download, count: data.count, timestamp }); }++ const ruleIPStmt = this.db.prepare(`+ INSERT INTO rule_ip_traffic (backend_id, rule, ip, total_upload, total_download, total_connections, last_seen)+ VALUES (@backendId, @rule, @ip, @upload, @download, @count, @timestamp)+ ON CONFLICT(backend_id, rule, ip) DO UPDATE SET+ total_upload = total_upload + @upload, total_download = total_download + @download,+ total_connections = total_connections + @count, last_seen = @timestamp+ `);+ for (const [, data] of ruleIPMap) { ruleIPStmt.run({ backendId, rule: data.rule, ip: data.ip, upload: data.upload, download: data.download, count: data.count, timestamp }); }++ const minuteStmt = this.db.prepare(`+ INSERT INTO minute_stats (backend_id, minute, upload, download, connections) VALUES (@backendId, @minute, @upload, @download, @connections)+ ON CONFLICT(backend_id, minute) DO UPDATE SET upload = upload + @upload, download = download + @download, connections = connections + @connections+ `);+ for (const [minute, data] of minuteMap) { minuteStmt.run({ backendId, minute, upload: data.upload, download: data.download, connections: data.connections }); }++ const minuteDimStmt = this.db.prepare(`+ INSERT INTO minute_dim_stats (backend_id, minute, domain, ip, source_ip, chain, rule, upload, download, connections)+ VALUES (@backendId, @minute, @domain, @ip, @sourceIP, @chain, @rule, @upload, @download, @connections)+ ON CONFLICT(backend_id, minute, domain, ip, source_ip, chain, rule) DO UPDATE SET+ upload = upload + @upload, download = download + @download, connections = connections + @connections+ `);+ for (const [, data] of minuteDimMap) { minuteDimStmt.run({ backendId, minute: data.minute, domain: data.domain, ip: data.ip, sourceIP: data.sourceIP, chain: data.chain, rule: data.rule, upload: data.upload, download: data.download, connections: data.connections }); }++ const domainProxyStmt = this.db.prepare(`+ INSERT INTO domain_proxy_stats (backend_id, domain, chain, total_upload, total_download, total_connections, last_seen)+ VALUES (@backendId, @domain, @chain, @upload, @download, @count, @timestamp)+ ON CONFLICT(backend_id, domain, chain) DO UPDATE SET+ total_upload = total_upload + @upload, total_download = total_download + @download,+ total_connections = total_connections + @count, last_seen = @timestamp+ `);+ for (const [, data] of domainProxyMap) { domainProxyStmt.run({ backendId, domain: data.domain, chain: data.chain, upload: data.upload, download: data.download, count: data.count, timestamp }); }++ const ipProxyStmt = this.db.prepare(`+ INSERT INTO ip_proxy_stats (backend_id, ip, chain, total_upload, total_download, total_connections, last_seen)+ VALUES (@backendId, @ip, @chain, @upload, @download, @count, @timestamp)+ ON CONFLICT(backend_id, ip, chain) DO UPDATE SET+ total_upload = total_upload + @upload, total_download = total_download + @download,+ total_connections = total_connections + @count, last_seen = @timestamp+ `);+ const ipProxyDomainStmt = this.db.prepare(`+ UPDATE ip_proxy_stats SET domains = CASE+ WHEN domains IS NULL OR domains = '' THEN @domain+ WHEN LENGTH(domains) > 4000 THEN domains+ WHEN INSTR(',' || domains || ',', ',' || @domain || ',') > 0 THEN domains+ ELSE domains || ',' || @domain END+ WHERE backend_id = @backendId AND ip = @ip AND chain = @chain+ `);+ for (const [, data] of ipProxyMap) {+ ipProxyStmt.run({ backendId, ip: data.ip, chain: data.chain, upload: data.upload, download: data.download, count: data.count, timestamp });+ if (data.domains.size > 0) {+ for (const domain of data.domains) { ipProxyDomainStmt.run({ backendId, ip: data.ip, chain: data.chain, domain }); }+ }+ }+ });+ tx2();++ // Sub-transaction 3: Device tables+ const tx3 = this.db.transaction(() => {+ const deviceStmt = this.db.prepare(`+ INSERT INTO device_stats (backend_id, source_ip, total_upload, total_download, total_connections, last_seen)+ VALUES (@backendId, @sourceIP, @upload, @download, @count, @timestamp)+ ON CONFLICT(backend_id, source_ip) DO UPDATE SET+ total_upload = total_upload + @upload, total_download = total_download + @download,+ total_connections = total_connections + @count, last_seen = @timestamp+ `);+ for (const [, data] of deviceMap) { deviceStmt.run({ backendId, sourceIP: data.sourceIP, upload: data.upload, download: data.download, count: data.count, timestamp }); }++ const deviceDomainStmt = this.db.prepare(`+ INSERT INTO device_domain_stats (backend_id, source_ip, domain, total_upload, total_download, total_connections, last_seen)+ VALUES (@backendId, @sourceIP, @domain, @upload, @download, @count, @timestamp)+ ON CONFLICT(backend_id, source_ip, domain) DO UPDATE SET+ total_upload = total_upload + @upload, total_download = total_download + @download,+ total_connections = total_connections + @count, last_seen = @timestamp+ `);+ for (const [, data] of deviceDomainMap) { deviceDomainStmt.run({ backendId, sourceIP: data.sourceIP, domain: data.domain, upload: data.upload, download: data.download, count: data.count, timestamp }); }++ const deviceIPStmt = this.db.prepare(`+ INSERT INTO device_ip_stats (backend_id, source_ip, ip, total_upload, total_download, total_connections, last_seen)+ VALUES (@backendId, @sourceIP, @ip, @upload, @download, @count, @timestamp)+ ON CONFLICT(backend_id, source_ip, ip) DO UPDATE SET+ total_upload = total_upload + @upload, total_download = total_download + @download,+ total_connections = total_connections + @count, last_seen = @timestamp+ `);+ for (const [, data] of deviceIPMap) { deviceIPStmt.run({ backendId, sourceIP: data.sourceIP, ip: data.ip, upload: data.upload, download: data.download, count: data.count, timestamp }); }+ });+ tx3();+ }+}diff --git a/apps/collector/src/db.ts b/apps/collector/src/db.tsindex 5fade08..edfcc54 100644--- a/apps/collector/src/db.ts+++ b/apps/collector/src/db.ts@@ -1,13 +1,21 @@import Database from 'better-sqlite3';import path from 'path';import fs from 'fs';-import type { Connection, DomainStats, IPStats, HourlyStats, DailyStats, ProxyStats, RuleStats, ProxyTrafficStats, DeviceStats } from '@neko-master/shared';-// Retention config stored in database (doesn't include cleanupInterval)-interface DatabaseRetentionConfig {- connectionLogsDays: number;- hourlyStatsDays: number;- autoCleanup: boolean;-}+import type { Connection, DomainStats, IPStats, HourlyStats, ProxyStats, RuleStats, ProxyTrafficStats, DeviceStats } from '@neko-master/shared';+import {+ AuthRepository,+ SurgeRepository,+ TimeseriesRepository,+ CountryRepository,+ DeviceRepository,+ ProxyRepository,+ RuleRepository,+ IPRepository,+ ConfigRepository,+ TrafficWriterRepository,+ DomainRepository,+ BackendRepository,+} from './database/repositories/index.js';export interface TrafficUpdate {domain: string;@@ -27,6 +35,7 @@ export interface BackendConfig {name: string;url: string;token: string;+ type: 'clash' | 'surge';enabled: boolean;is_active: boolean;listening: boolean;@@ -38,10 +47,40 @@ export class StatsDatabase {private db: Database.Database;private dbPath: string;+ public readonly repos: {+ auth: AuthRepository;+ surge: SurgeRepository;+ timeseries: TimeseriesRepository;+ country: CountryRepository;+ device: DeviceRepository;+ proxy: ProxyRepository;+ rule: RuleRepository;+ ip: IPRepository;+ config: ConfigRepository;+ trafficWriter: TrafficWriterRepository;+ domain: DomainRepository;+ backend: BackendRepository;+ };+constructor(dbPath = 'stats.db') {this.dbPath = path.resolve(dbPath);this.db = new Database(this.dbPath);this.init();++ this.repos = {+ auth: new AuthRepository(this.db),+ surge: new SurgeRepository(this.db),+ timeseries: new TimeseriesRepository(this.db),+ country: new CountryRepository(this.db),+ device: new DeviceRepository(this.db),+ proxy: new ProxyRepository(this.db),+ rule: new RuleRepository(this.db),+ ip: new IPRepository(this.db),+ config: new ConfigRepository(this.db, this.dbPath),+ trafficWriter: new TrafficWriterRepository(this.db),+ domain: new DomainRepository(this.db),+ backend: new BackendRepository(this.db),+ };}private init() {@@ -403,6 +442,7 @@ export class StatsDatabase {name TEXT NOT NULL,url TEXT NOT NULL,token TEXT DEFAULT '',+ type TEXT DEFAULT 'clash',enabled BOOLEAN DEFAULT 1,is_active BOOLEAN DEFAULT 0,listening BOOLEAN DEFAULT 1,@@ -414,6 +454,14 @@ export class StatsDatabase {// Create unique index on namethis.db.exec(`CREATE UNIQUE INDEX IF NOT EXISTS idx_backend_configs_name ON backend_configs(name);`);+ // Migration: Add type column if not exists (for older databases)+ try {+ this.db.exec(`ALTER TABLE backend_configs ADD COLUMN type TEXT DEFAULT 'clash'`);+ console.log('[DB] Migration: Added type column to backend_configs');+ } catch {+ // Column already exists, ignore+ }+// App configuration - stores app-level settings like retention policythis.db.exec(`CREATE TABLE IF NOT EXISTS app_config (@@ -431,6 +479,22 @@ export class StatsDatabase {('retention.auto_cleanup', '1');`);+ // Surge policy cache - stores Surge policy group selections+ this.db.exec(`+ CREATE TABLE IF NOT EXISTS surge_policy_cache (+ backend_id INTEGER NOT NULL,+ policy_group TEXT NOT NULL,+ selected_policy TEXT,+ policy_type TEXT DEFAULT 'Select',+ all_policies TEXT,+ updated_at DATETIME DEFAULT CURRENT_TIMESTAMP,+ PRIMARY KEY (backend_id, policy_group),+ FOREIGN KEY (backend_id) REFERENCES backend_configs(id) ON DELETE CASCADE+ );+ `);+ this.db.exec(`CREATE INDEX IF NOT EXISTS idx_surge_policy_backend ON surge_policy_cache(backend_id);`);+ this.db.exec(`CREATE INDEX IF NOT EXISTS idx_surge_policy_updated ON surge_policy_cache(updated_at);`);+// Auth configuration table - stores authentication settingsthis.db.exec(`CREATE TABLE IF NOT EXISTS auth_config (@@ -479,6 +543,14 @@ export class StatsDatabase {this.db.exec(`ALTER TABLE backend_configs ADD COLUMN listening BOOLEAN DEFAULT 1;`);}+ // Check if backend_configs has type column+ const hasType = backendInfo.some(col => col.name === 'type');++ if (!hasType) {+ console.log('[DB] Adding type column to backend_configs...');+ this.db.exec(`ALTER TABLE backend_configs ADD COLUMN type TEXT DEFAULT 'clash';`);+ }+// Check if ip_stats has chains column (added for proxy display per IP)const ipStatsInfo = this.db.prepare(`PRAGMA table_info(ip_stats)`).all() as { name: string }[];const hasIPChains = ipStatsInfo.some(col => col.name === 'chains');@@ -780,4530 +852,152 @@ export class StatsDatabase {}}- // Update traffic stats with delta values for a specific backend- updateTrafficStats(backendId: number, update: TrafficUpdate) {- const now = new Date();- const timestamp = now.toISOString();- const hour = timestamp.slice(0, 13) + ':00:00';-- // Only process if there's actual traffic- if (update.upload === 0 && update.download === 0) return;-- // Get rule name from chains (friendly name) or fallback to rule + rulePayload- const ruleName = update.chains.length > 1 ? update.chains[update.chains.length - 1] :- update.rulePayload ? `${update.rule}(${update.rulePayload})` : update.rule;- const finalProxy = update.chains.length > 0 ? update.chains[0] : 'DIRECT';- const fullChain = update.chains.join(' > ') || update.chain || 'DIRECT';-- const transaction = this.db.transaction(() => {- // Update domain stats with backend_id (skip if domain is unknown)- const domainName = update.domain || 'unknown';- if (domainName !== 'unknown') {- const domainStmt = this.db.prepare(`- INSERT INTO domain_stats (backend_id, domain, ips, total_upload, total_download, total_connections, last_seen, rules, chains)- VALUES (@backendId, @domain, @ip, @upload, @download, 1, @timestamp, @rule, @chain)- ON CONFLICT(backend_id, domain) DO UPDATE SET- ips = CASE- WHEN domain_stats.ips IS NULL THEN @ip- WHEN INSTR(domain_stats.ips, @ip) > 0 THEN domain_stats.ips- ELSE domain_stats.ips || ',' || @ip- END,- total_upload = total_upload + @upload,- total_download = total_download + @download,- total_connections = total_connections + 1,- last_seen = @timestamp,- rules = CASE- WHEN domain_stats.rules IS NULL THEN @rule- WHEN INSTR(domain_stats.rules, @rule) > 0 THEN domain_stats.rules- ELSE domain_stats.rules || ',' || @rule- END,- chains = CASE- WHEN domain_stats.chains IS NULL THEN @chain- WHEN INSTR(domain_stats.chains, @chain) > 0 THEN domain_stats.chains- ELSE domain_stats.chains || ',' || @chain- END- `);- domainStmt.run({- backendId,- domain: domainName,- ip: update.ip,- upload: update.upload,- download: update.download,- timestamp,- rule: ruleName,- chain: fullChain- });- }-- // Update IP stats with backend_id- const ipStmt = this.db.prepare(`- INSERT INTO ip_stats (backend_id, ip, domains, total_upload, total_download, total_connections, last_seen, chains, rules)- VALUES (@backendId, @ip, @domain, @upload, @download, 1, @timestamp, @chain, @rule)- ON CONFLICT(backend_id, ip) DO UPDATE SET- domains = CASE- WHEN ip_stats.domains IS NULL THEN @domain- WHEN INSTR(ip_stats.domains, @domain) > 0 THEN ip_stats.domains- ELSE ip_stats.domains || ',' || @domain- END,- total_upload = total_upload + @upload,- total_download = total_download + @download,- total_connections = total_connections + 1,- last_seen = @timestamp,- chains = CASE- WHEN ip_stats.chains IS NULL THEN @chain- WHEN INSTR(ip_stats.chains, @chain) > 0 THEN ip_stats.chains- ELSE ip_stats.chains || ',' || @chain- END,- rules = CASE- WHEN ip_stats.rules IS NULL THEN @rule- WHEN INSTR(ip_stats.rules, @rule) > 0 THEN ip_stats.rules- ELSE ip_stats.rules || ',' || @rule- END- `);- ipStmt.run({- backendId,- ip: update.ip,- domain: update.domain || 'unknown',- upload: update.upload,- download: update.download,- timestamp,- chain: fullChain,- rule: ruleName- });-- // Update proxy stats with backend_id- const proxyStmt = this.db.prepare(`- INSERT INTO proxy_stats (backend_id, chain, total_upload, total_download, total_connections, last_seen)- VALUES (@backendId, @chain, @upload, @download, 1, @timestamp)- ON CONFLICT(backend_id, chain) DO UPDATE SET- total_upload = total_upload + @upload,- total_download = total_download + @download,- total_connections = total_connections + 1,- last_seen = @timestamp- `);- proxyStmt.run({- backendId,- chain: fullChain,- upload: update.upload,- download: update.download,- timestamp- });-- // Update rule stats with backend_id- const ruleStmt = this.db.prepare(`- INSERT INTO rule_stats (backend_id, rule, final_proxy, total_upload, total_download, total_connections, last_seen)- VALUES (@backendId, @rule, @finalProxy, @upload, @download, 1, @timestamp)- ON CONFLICT(backend_id, rule) DO UPDATE SET- final_proxy = @finalProxy,- total_upload = total_upload + @upload,- total_download = total_download + @download,- total_connections = total_connections + 1,- last_seen = @timestamp- `);- ruleStmt.run({- backendId,- rule: ruleName,- finalProxy,- upload: update.upload,- download: update.download,- timestamp- });-- // Update rule_chain_traffic (all connections have a chain)- const ruleChainStmt = this.db.prepare(`- INSERT INTO rule_chain_traffic (backend_id, rule, chain, total_upload, total_download, total_connections, last_seen)- VALUES (@backendId, @rule, @chain, @upload, @download, 1, @timestamp)- ON CONFLICT(backend_id, rule, chain) DO UPDATE SET- total_upload = total_upload + @upload,- total_download = total_download + @download,- total_connections = total_connections + 1,- last_seen = @timestamp- `);- ruleChainStmt.run({- backendId,- rule: ruleName,- chain: fullChain,- upload: update.upload,- download: update.download,- timestamp- });-- // Update rule_domain_traffic (only when domain is known)- if (domainName !== 'unknown') {- const ruleDomainStmt = this.db.prepare(`- INSERT INTO rule_domain_traffic (backend_id, rule, domain, total_upload, total_download, total_connections, last_seen)- VALUES (@backendId, @rule, @domain, @upload, @download, 1, @timestamp)- ON CONFLICT(backend_id, rule, domain) DO UPDATE SET- total_upload = total_upload + @upload,- total_download = total_download + @download,- total_connections = total_connections + 1,- last_seen = @timestamp- `);- ruleDomainStmt.run({- backendId,- rule: ruleName,- domain: domainName,- upload: update.upload,- download: update.download,- timestamp- });- }-- // Update rule_ip_traffic (all connections have an IP)- const ruleIPStmt = this.db.prepare(`- INSERT INTO rule_ip_traffic (backend_id, rule, ip, total_upload, total_download, total_connections, last_seen)- VALUES (@backendId, @rule, @ip, @upload, @download, 1, @timestamp)- ON CONFLICT(backend_id, rule, ip) DO UPDATE SET- total_upload = total_upload + @upload,- total_download = total_download + @download,- total_connections = total_connections + 1,- last_seen = @timestamp- `);- ruleIPStmt.run({- backendId,- rule: ruleName,- ip: update.ip,- upload: update.upload,- download: update.download,- timestamp- });-- // Update rule-proxy mapping with backend_id- if (update.chains.length > 1) {- const rule = update.chains[update.chains.length - 1];- const proxy = update.chains[0];- const ruleProxyStmt = this.db.prepare(`- INSERT OR IGNORE INTO rule_proxy_map (backend_id, rule, proxy)- VALUES (@backendId, @rule, @proxy)- `);- ruleProxyStmt.run({ backendId, rule, proxy });- }-- // Update hourly stats with backend_id- const hourlyStmt = this.db.prepare(`- INSERT INTO hourly_stats (backend_id, hour, upload, download, connections)- VALUES (@backendId, @hour, @upload, @download, 1)- ON CONFLICT(backend_id, hour) DO UPDATE SET- upload = upload + @upload,- download = download + @download,- connections = connections + 1- `);- hourlyStmt.run({ backendId, hour, upload: update.upload, download: update.download });-- // UPSERT minute_stats- const minute = timestamp.slice(0, 16) + ':00'; // 'YYYY-MM-DDTHH:MM:00'- const minuteStmt = this.db.prepare(`- INSERT INTO minute_stats (backend_id, minute, upload, download, connections)- VALUES (@backendId, @minute, @upload, @download, 1)- ON CONFLICT(backend_id, minute) DO UPDATE SET- upload = upload + @upload,- download = download + @download,- connections = connections + 1- `);- minuteStmt.run({ backendId, minute, upload: update.upload, download: update.download });-- // UPSERT domain_proxy_stats (only when domain is known)- if (domainName !== 'unknown') {- const domainProxyStmt = this.db.prepare(`- INSERT INTO domain_proxy_stats (backend_id, domain, chain, total_upload, total_download, total_connections, last_seen)- VALUES (@backendId, @domain, @chain, @upload, @download, 1, @timestamp)- ON CONFLICT(backend_id, domain, chain) DO UPDATE SET- total_upload = total_upload + @upload,- total_download = total_download + @download,- total_connections = total_connections + 1,- last_seen = @timestamp- `);- domainProxyStmt.run({- backendId,- domain: domainName,- chain: fullChain,- upload: update.upload,- download: update.download,- timestamp- });- }-- // UPSERT ip_proxy_stats- const ipProxyStmt = this.db.prepare(`- INSERT INTO ip_proxy_stats (backend_id, ip, chain, total_upload, total_download, total_connections, last_seen, domains)- VALUES (@backendId, @ip, @chain, @upload, @download, 1, @timestamp, @domain)- ON CONFLICT(backend_id, ip, chain) DO UPDATE SET- total_upload = total_upload + @upload,- total_download = total_download + @download,- total_connections = total_connections + 1,- last_seen = @timestamp,- domains = CASE- WHEN ip_proxy_stats.domains IS NULL THEN @domain- WHEN @domain = 'unknown' THEN ip_proxy_stats.domains- WHEN INSTR(ip_proxy_stats.domains, @domain) > 0 THEN ip_proxy_stats.domains- ELSE ip_proxy_stats.domains || ',' || @domain- END- `);- ipProxyStmt.run({- backendId,- ip: update.ip,- chain: fullChain,- upload: update.upload,- download: update.download,- timestamp,- domain: update.domain || 'unknown'- });- });-- transaction();- }-- // Batch update traffic stats - processes multiple updates in a single transaction- batchUpdateTrafficStats(backendId: number, updates: TrafficUpdate[]) {- if (updates.length === 0) return;-- const now = new Date();- const timestamp = now.toISOString();-- // Aggregate updates by domain, ip, chain to reduce UPSERT conflicts- const domainMap = new Map<string, TrafficUpdate & { count: number }>();- const ipMap = new Map<string, TrafficUpdate & { count: number }>();- const chainMap = new Map<string, { chains: string[]; upload: number; download: number; count: number }>();- const ruleProxyMap = new Map<string, { rule: string; proxy: string; upload: number; download: number; count: number }>();- const hourlyMap = new Map<string, { upload: number; download: number; connections: number }>();- const ruleChainMap = new Map<string, { rule: string; chain: string; upload: number; download: number; count: number }>();- const ruleDomainMap = new Map<string, { rule: string; domain: string; upload: number; download: number; count: number }>();- const ruleIPMap = new Map<string, { rule: string; ip: string; upload: number; download: number; count: number }>();- const minuteMap = new Map<string, { upload: number; download: number; connections: number }>();- const minuteDimMap = new Map<string, {- minute: string;- domain: string;- ip: string;- sourceIP: string;- chain: string;- rule: string;- upload: number;- download: number;- connections: number;- }>();- const domainProxyMap = new Map<string, { domain: string; chain: string; upload: number; download: number; count: number }>();- const ipProxyMap = new Map<string, { ip: string; chain: string; upload: number; download: number; count: number; domains: Set<string> }>();- const deviceMap = new Map<string, { sourceIP: string; upload: number; download: number; count: number }>();- const deviceDomainMap = new Map<string, { sourceIP: string; domain: string; upload: number; download: number; count: number }>();- const deviceIPMap = new Map<string, { sourceIP: string; ip: string; upload: number; download: number; count: number }>();-- for (const update of updates) {- if (update.upload === 0 && update.download === 0) continue;-- // Use the last element of chains as the rule name (friendly name from Gateway config)- // e.g., "ๆผ็ฝไน้ฑผ", "ๅพฎ่ฝฏๆๅก", "TikTok"- const ruleName = update.chains.length > 1 ? update.chains[update.chains.length - 1] :- update.rulePayload ? `${update.rule}(${update.rulePayload})` : update.rule;- const finalProxy = update.chains.length > 0 ? update.chains[0] : 'DIRECT';- const fullChain = update.chains.join(' > ') || update.chain || 'DIRECT';- const eventDate = new Date(update.timestampMs ?? now.getTime());- const hourKey = this.toHourKey(eventDate);- const minuteKey = this.toMinuteKey(eventDate);-- // Aggregate domain stats- if (update.domain) {- const domainKey = `${update.domain}:${update.ip}:${fullChain}`;- const existing = domainMap.get(domainKey);- if (existing) {- existing.upload += update.upload;- existing.download += update.download;- existing.count++;- } else {- domainMap.set(domainKey, { ...update, count: 1 });- }- }-- // Aggregate IP stats- const ipKey = `${update.ip}:${update.domain}:${fullChain}`;- const existingIp = ipMap.get(ipKey);- if (existingIp) {- existingIp.upload += update.upload;- existingIp.download += update.download;- existingIp.count++;- } else {- ipMap.set(ipKey, { ...update, rule: ruleName, count: 1 });- }-- // Aggregate chain stats- const chainKey = fullChain;- const existingChain = chainMap.get(chainKey);- if (existingChain) {- existingChain.upload += update.upload;- existingChain.download += update.download;- existingChain.count++;- } else {- chainMap.set(chainKey, {- chains: update.chains,- upload: update.upload,- download: update.download,- count: 1- });- }-- // Aggregate rule stats- const ruleKey = `${ruleName}:${finalProxy}`;- const existingRule = ruleProxyMap.get(ruleKey);- if (existingRule) {- existingRule.upload += update.upload;- existingRule.download += update.download;- existingRule.count++;- } else {- ruleProxyMap.set(ruleKey, { rule: ruleName, proxy: finalProxy, upload: update.upload, download: update.download, count: 1 });- }-- // Aggregate hourly stats- const existingHour = hourlyMap.get(hourKey);- if (existingHour) {- existingHour.upload += update.upload;- existingHour.download += update.download;- existingHour.connections++;- } else {- hourlyMap.set(hourKey, {- upload: update.upload,- download: update.download,- connections: 1- });- }-- // Aggregate rule_chain_traffic- const fullChainForRule = update.chains.join(' > ');- const ruleChainKey = `${ruleName}:${fullChainForRule}`;- const existingRuleChain = ruleChainMap.get(ruleChainKey);- if (existingRuleChain) {- existingRuleChain.upload += update.upload;- existingRuleChain.download += update.download;- existingRuleChain.count++;- } else {- ruleChainMap.set(ruleChainKey, { rule: ruleName, chain: fullChainForRule, upload: update.upload, download: update.download, count: 1 });- }-- // Aggregate rule_domain_traffic (only when domain exists)- if (update.domain) {- const ruleDomainKey = `${ruleName}:${update.domain}`;- const existingRuleDomain = ruleDomainMap.get(ruleDomainKey);- if (existingRuleDomain) {- existingRuleDomain.upload += update.upload;- existingRuleDomain.download += update.download;- existingRuleDomain.count++;- } else {- ruleDomainMap.set(ruleDomainKey, { rule: ruleName, domain: update.domain, upload: update.upload, download: update.download, count: 1 });- }- }-- // Aggregate rule_ip_traffic- const ruleIPKey = `${ruleName}:${update.ip}`;- const existingRuleIP = ruleIPMap.get(ruleIPKey);- if (existingRuleIP) {- existingRuleIP.upload += update.upload;- existingRuleIP.download += update.download;- existingRuleIP.count++;- } else {- ruleIPMap.set(ruleIPKey, { rule: ruleName, ip: update.ip, upload: update.upload, download: update.download, count: 1 });- }-- // Aggregate minute_stats- const existingMinute = minuteMap.get(minuteKey);- if (existingMinute) {- existingMinute.upload += update.upload;- existingMinute.download += update.download;- existingMinute.connections++;- } else {- minuteMap.set(minuteKey, { upload: update.upload, download: update.download, connections: 1 });- }-- // Aggregate minute-level dimension facts for range queries- const dimDomain = update.domain || '';- const dimIP = update.ip || '';- const dimSourceIP = update.sourceIP || '';- const dimChain = fullChain;- const dimKey = `${minuteKey}:${dimDomain}:${dimIP}:${dimSourceIP}:${dimChain}:${ruleName}`;- const existingDim = minuteDimMap.get(dimKey);- if (existingDim) {- existingDim.upload += update.upload;- existingDim.download += update.download;- existingDim.connections++;- } else {- minuteDimMap.set(dimKey, {- minute: minuteKey,- domain: dimDomain,- ip: dimIP,- sourceIP: dimSourceIP,- chain: dimChain,- rule: ruleName,- upload: update.upload,- download: update.download,- connections: 1,- });- }-- // Aggregate domain_proxy_stats (only when domain exists)- const proxyChain = fullChain;- if (update.domain) {- const dpKey = `${update.domain}:${proxyChain}`;- const existingDP = domainProxyMap.get(dpKey);- if (existingDP) {- existingDP.upload += update.upload;- existingDP.download += update.download;- existingDP.count++;- } else {- domainProxyMap.set(dpKey, { domain: update.domain, chain: proxyChain, upload: update.upload, download: update.download, count: 1 });- }- }-- // Aggregate ip_proxy_stats- const ipPKey = `${update.ip}:${proxyChain}`;- const existingIPP = ipProxyMap.get(ipPKey);- if (existingIPP) {- existingIPP.upload += update.upload;- existingIPP.download += update.download;- existingIPP.count++;- if (update.domain && update.domain !== 'unknown') {- existingIPP.domains.add(update.domain);- }- } else {- const domains = new Set<string>();- if (update.domain && update.domain !== 'unknown') {- domains.add(update.domain);- }- ipProxyMap.set(ipPKey, { ip: update.ip, chain: proxyChain, upload: update.upload, download: update.download, count: 1, domains });- }-- // Aggregate device stats- if (update.sourceIP) {- const sourceIP = update.sourceIP;-- // Device stats- const deviceKey = sourceIP;- const existingDevice = deviceMap.get(deviceKey);- if (existingDevice) {- existingDevice.upload += update.upload;- existingDevice.download += update.download;- existingDevice.count++;- } else {- deviceMap.set(deviceKey, { sourceIP, upload: update.upload, download: update.download, count: 1 });- }-- // Device domain stats- if (update.domain) {- const deviceDomainKey = `${sourceIP}:${update.domain}`;- const existingDeviceDomain = deviceDomainMap.get(deviceDomainKey);- if (existingDeviceDomain) {- existingDeviceDomain.upload += update.upload;- existingDeviceDomain.download += update.download;- existingDeviceDomain.count++;- } else {- deviceDomainMap.set(deviceDomainKey, { sourceIP, domain: update.domain, upload: update.upload, download: update.download, count: 1 });- }- }-- // Device IP stats- if (update.ip) {- const deviceIPKey = `${sourceIP}:${update.ip}`;- const existingDeviceIP = deviceIPMap.get(deviceIPKey);- if (existingDeviceIP) {- existingDeviceIP.upload += update.upload;- existingDeviceIP.download += update.download;- existingDeviceIP.count++;- } else {- deviceIPMap.set(deviceIPKey, { sourceIP, ip: update.ip, upload: update.upload, download: update.download, count: 1 });- }- }- }- }-- // Execute batch upserts in sub-transactions to reduce SQLite lock contention.- // Sub-transaction 1: Core aggregation tables (domain_stats, ip_stats, proxy_stats, rule_stats)- const tx1 = this.db.transaction(() => {- // Domain stats- const domainStmt = this.db.prepare(`- INSERT INTO domain_stats (backend_id, domain, ips, total_upload, total_download, total_connections, last_seen, rules, chains)- VALUES (@backendId, @domain, @ip, @upload, @download, @count, @timestamp, @rule, @chain)- ON CONFLICT(backend_id, domain) DO UPDATE SET- ips = CASE- WHEN domain_stats.ips IS NULL THEN @ip- WHEN LENGTH(domain_stats.ips) > 4000 THEN domain_stats.ips- WHEN INSTR(domain_stats.ips, @ip) > 0 THEN domain_stats.ips- ELSE domain_stats.ips || ',' || @ip- END,- total_upload = total_upload + @upload,- total_download = total_download + @download,- total_connections = total_connections + @count,- last_seen = @timestamp,- rules = CASE- WHEN domain_stats.rules IS NULL THEN @rule- WHEN LENGTH(domain_stats.rules) > 4000 THEN domain_stats.rules- WHEN INSTR(domain_stats.rules, @rule) > 0 THEN domain_stats.rules- ELSE domain_stats.rules || ',' || @rule- END,- chains = CASE- WHEN domain_stats.chains IS NULL THEN @chain- WHEN LENGTH(domain_stats.chains) > 4000 THEN domain_stats.chains- WHEN INSTR(domain_stats.chains, @chain) > 0 THEN domain_stats.chains- ELSE domain_stats.chains || ',' || @chain- END- `);-- for (const [key, data] of domainMap) {- const ruleName = data.chains.length > 1 ? data.chains[data.chains.length - 1] :- data.rulePayload ? `${data.rule}(${data.rulePayload})` : data.rule;- // Store full chain path (joined with > for clarity)- const fullChain = data.chains.join(' > ');- domainStmt.run({- backendId,- domain: data.domain,- ip: data.ip,- upload: data.upload,- download: data.download,- count: data.count,- timestamp,- rule: ruleName,- chain: fullChain- });- }-- // IP stats- const ipStmt = this.db.prepare(`- INSERT INTO ip_stats (backend_id, ip, domains, total_upload, total_download, total_connections, last_seen, chains, rules)- VALUES (@backendId, @ip, @domain, @upload, @download, @count, @timestamp, @chain, @rule)- ON CONFLICT(backend_id, ip) DO UPDATE SET- domains = CASE- WHEN ip_stats.domains IS NULL THEN @domain- WHEN LENGTH(ip_stats.domains) > 4000 THEN ip_stats.domains- WHEN INSTR(ip_stats.domains, @domain) > 0 THEN ip_stats.domains- ELSE ip_stats.domains || ',' || @domain- END,- total_upload = total_upload + @upload,- total_download = total_download + @download,- total_connections = total_connections + @count,- last_seen = @timestamp,- chains = CASE- WHEN ip_stats.chains IS NULL THEN @chain- WHEN LENGTH(ip_stats.chains) > 4000 THEN ip_stats.chains- WHEN INSTR(ip_stats.chains, @chain) > 0 THEN ip_stats.chains- ELSE ip_stats.chains || ',' || @chain- END,- rules = CASE- WHEN ip_stats.rules IS NULL THEN @rule- WHEN LENGTH(ip_stats.rules) > 4000 THEN ip_stats.rules- WHEN INSTR(ip_stats.rules, @rule) > 0 THEN ip_stats.rules- ELSE ip_stats.rules || ',' || @rule- END- `);-- for (const [key, data] of ipMap) {- // Store full chain path (joined with > for clarity)- const fullChain = data.chains.join(' > ');- ipStmt.run({- backendId,- ip: data.ip,- domain: data.domain || 'unknown',- upload: data.upload,- download: data.download,- count: data.count,- timestamp,- chain: fullChain,- rule: data.rule- });- }-- // Chain/Proxy stats- const proxyStmt = this.db.prepare(`- INSERT INTO proxy_stats (backend_id, chain, total_upload, total_download, total_connections, last_seen)- VALUES (@backendId, @chain, @upload, @download, @count, @timestamp)- ON CONFLICT(backend_id, chain) DO UPDATE SET- total_upload = total_upload + @upload,- total_download = total_download + @download,- total_connections = total_connections + @count,- last_seen = @timestamp- `);-- for (const [chain, data] of chainMap) {- proxyStmt.run({- backendId,- chain,- upload: data.upload,- download: data.download,- count: data.count,- timestamp- });- }-- // Rule stats- const ruleStmt = this.db.prepare(`- INSERT INTO rule_stats (backend_id, rule, final_proxy, total_upload, total_download, total_connections, last_seen)- VALUES (@backendId, @rule, @proxy, @upload, @download, @count, @timestamp)- ON CONFLICT(backend_id, rule) DO UPDATE SET- final_proxy = @proxy,- total_upload = total_upload + @upload,- total_download = total_download + @download,- total_connections = total_connections + @count,- last_seen = @timestamp- `);-- for (const [key, data] of ruleProxyMap) {- ruleStmt.run({- backendId,- rule: data.rule,- proxy: data.proxy,- upload: data.upload,- download: data.download,- count: data.count,- timestamp- });- }-- // Rule proxy map (only insert, ignore duplicates)- const ruleProxyStmt = this.db.prepare(`- INSERT OR IGNORE INTO rule_proxy_map (backend_id, rule, proxy)- VALUES (@backendId, @rule, @proxy)- `);-- for (const [key, data] of ruleProxyMap) {- ruleProxyStmt.run({- backendId,- rule: data.rule,- proxy: data.proxy- });- }-- // Hourly stats- const hourlyStmt = this.db.prepare(`- INSERT INTO hourly_stats (backend_id, hour, upload, download, connections)- VALUES (@backendId, @hour, @upload, @download, @connections)- ON CONFLICT(backend_id, hour) DO UPDATE SET- upload = upload + @upload,- download = download + @download,- connections = connections + @connections- `);-- for (const [hour, data] of hourlyMap) {- hourlyStmt.run({- backendId,- hour,- upload: data.upload,- download: data.download,- connections: data.connections- });- }-- // Rule chain traffic- const ruleChainStmt = this.db.prepare(`- INSERT INTO rule_chain_traffic (backend_id, rule, chain, total_upload, total_download, total_connections, last_seen)- VALUES (@backendId, @rule, @chain, @upload, @download, @count, @timestamp)- ON CONFLICT(backend_id, rule, chain) DO UPDATE SET- total_upload = total_upload + @upload,- total_download = total_download + @download,- total_connections = total_connections + @count,- last_seen = @timestamp- `);-- for (const [key, data] of ruleChainMap) {- ruleChainStmt.run({- backendId,- rule: data.rule,- chain: data.chain,- upload: data.upload,- download: data.download,- count: data.count,- timestamp- });- }- });- tx1();-- // Sub-transaction 2: Detail tables + minute/hourly tables- const tx2 = this.db.transaction(() => {-- // Rule domain traffic- const ruleDomainStmt = this.db.prepare(`- INSERT INTO rule_domain_traffic (backend_id, rule, domain, total_upload, total_download, total_connections, last_seen)- VALUES (@backendId, @rule, @domain, @upload, @download, @count, @timestamp)- ON CONFLICT(backend_id, rule, domain) DO UPDATE SET- total_upload = total_upload + @upload,- total_download = total_download + @download,- total_connections = total_connections + @count,- last_seen = @timestamp- `);-- for (const [key, data] of ruleDomainMap) {- ruleDomainStmt.run({- backendId,- rule: data.rule,- domain: data.domain,- upload: data.upload,- download: data.download,- count: data.count,- timestamp- });- }-- // Rule IP traffic- const ruleIPStmt = this.db.prepare(`- INSERT INTO rule_ip_traffic (backend_id, rule, ip, total_upload, total_download, total_connections, last_seen)- VALUES (@backendId, @rule, @ip, @upload, @download, @count, @timestamp)- ON CONFLICT(backend_id, rule, ip) DO UPDATE SET- total_upload = total_upload + @upload,- total_download = total_download + @download,- total_connections = total_connections + @count,- last_seen = @timestamp- `);-- for (const [key, data] of ruleIPMap) {- ruleIPStmt.run({- backendId,- rule: data.rule,- ip: data.ip,- upload: data.upload,- download: data.download,- count: data.count,- timestamp- });- }-- // UPSERT minute_stats- const minuteStmt = this.db.prepare(`- INSERT INTO minute_stats (backend_id, minute, upload, download, connections)- VALUES (@backendId, @minute, @upload, @download, @connections)- ON CONFLICT(backend_id, minute) DO UPDATE SET- upload = upload + @upload,- download = download + @download,- connections = connections + @connections- `);-- for (const [minute, data] of minuteMap) {- minuteStmt.run({- backendId,- minute,- upload: data.upload,- download: data.download,- connections: data.connections- });- }-- // UPSERT minute_dim_stats (range-query fact table)- const minuteDimStmt = this.db.prepare(`- INSERT INTO minute_dim_stats (backend_id, minute, domain, ip, source_ip, chain, rule, upload, download, connections)- VALUES (@backendId, @minute, @domain, @ip, @sourceIP, @chain, @rule, @upload, @download, @connections)- ON CONFLICT(backend_id, minute, domain, ip, source_ip, chain, rule) DO UPDATE SET- upload = upload + @upload,- download = download + @download,- connections = connections + @connections- `);-- for (const [, data] of minuteDimMap) {- minuteDimStmt.run({- backendId,- minute: data.minute,- domain: data.domain,- ip: data.ip,- sourceIP: data.sourceIP,- chain: data.chain,- rule: data.rule,- upload: data.upload,- download: data.download,- connections: data.connections,- });- }-- // UPSERT domain_proxy_stats- const domainProxyStmt = this.db.prepare(`- INSERT INTO domain_proxy_stats (backend_id, domain, chain, total_upload, total_download, total_connections, last_seen)- VALUES (@backendId, @domain, @chain, @upload, @download, @count, @timestamp)- ON CONFLICT(backend_id, domain, chain) DO UPDATE SET- total_upload = total_upload + @upload,- total_download = total_download + @download,- total_connections = total_connections + @count,- last_seen = @timestamp- `);-- for (const [key, data] of domainProxyMap) {- domainProxyStmt.run({- backendId,- domain: data.domain,- chain: data.chain,- upload: data.upload,- download: data.download,- count: data.count,- timestamp- });- }-- // UPSERT ip_proxy_stats (traffic only; domains are updated separately with dedupe)- const ipProxyStmt = this.db.prepare(`- INSERT INTO ip_proxy_stats (backend_id, ip, chain, total_upload, total_download, total_connections, last_seen)- VALUES (@backendId, @ip, @chain, @upload, @download, @count, @timestamp)- ON CONFLICT(backend_id, ip, chain) DO UPDATE SET- total_upload = total_upload + @upload,- total_download = total_download + @download,- total_connections = total_connections + @count,- last_seen = @timestamp- `);- const ipProxyDomainStmt = this.db.prepare(`- UPDATE ip_proxy_stats- SET domains = CASE- WHEN domains IS NULL OR domains = '' THEN @domain- WHEN LENGTH(domains) > 4000 THEN domains- WHEN INSTR(',' || domains || ',', ',' || @domain || ',') > 0 THEN domains- ELSE domains || ',' || @domain- END- WHERE backend_id = @backendId AND ip = @ip AND chain = @chain- `);-- for (const [key, data] of ipProxyMap) {- ipProxyStmt.run({- backendId,- ip: data.ip,- chain: data.chain,- upload: data.upload,- download: data.download,- count: data.count,- timestamp- });-- if (data.domains.size > 0) {- for (const domain of data.domains) {- ipProxyDomainStmt.run({- backendId,- ip: data.ip,- chain: data.chain,- domain- });- }- }- }- });- tx2();-- // Sub-transaction 3: Device tables- const tx3 = this.db.transaction(() => {- // Device stats- const deviceStmt = this.db.prepare(`- INSERT INTO device_stats (backend_id, source_ip, total_upload, total_download, total_connections, last_seen)- VALUES (@backendId, @sourceIP, @upload, @download, @count, @timestamp)- ON CONFLICT(backend_id, source_ip) DO UPDATE SET- total_upload = total_upload + @upload,- total_download = total_download + @download,- total_connections = total_connections + @count,- last_seen = @timestamp- `);-- for (const [key, data] of deviceMap) {- deviceStmt.run({- backendId,- sourceIP: data.sourceIP,- upload: data.upload,- download: data.download,- count: data.count,- timestamp- });- }-- // Device domain stats- const deviceDomainStmt = this.db.prepare(`- INSERT INTO device_domain_stats (backend_id, source_ip, domain, total_upload, total_download, total_connections, last_seen)- VALUES (@backendId, @sourceIP, @domain, @upload, @download, @count, @timestamp)- ON CONFLICT(backend_id, source_ip, domain) DO UPDATE SET- total_upload = total_upload + @upload,- total_download = total_download + @download,- total_connections = total_connections + @count,- last_seen = @timestamp- `);-- for (const [key, data] of deviceDomainMap) {- deviceDomainStmt.run({- backendId,- sourceIP: data.sourceIP,- domain: data.domain,- upload: data.upload,- download: data.download,- count: data.count,- timestamp- });- }-- // Device IP stats- const deviceIPStmt = this.db.prepare(`- INSERT INTO device_ip_stats (backend_id, source_ip, ip, total_upload, total_download, total_connections, last_seen)- VALUES (@backendId, @sourceIP, @ip, @upload, @download, @count, @timestamp)- ON CONFLICT(backend_id, source_ip, ip) DO UPDATE SET- total_upload = total_upload + @upload,- total_download = total_download + @download,- total_connections = total_connections + @count,- last_seen = @timestamp- `);-- for (const [key, data] of deviceIPMap) {- deviceIPStmt.run({- backendId,- sourceIP: data.sourceIP,- ip: data.ip,- upload: data.upload,- download: data.download,- count: data.count,- timestamp- });- }- });- tx3();- }-+ // ==================== Traffic Writer ====================+ updateTrafficStats(backendId: number, update: TrafficUpdate) { this.repos.trafficWriter.updateTrafficStats(backendId, update); }+ batchUpdateTrafficStats(backendId: number, updates: TrafficUpdate[]) { this.repos.trafficWriter.batchUpdateTrafficStats(backendId, updates); }++ // ==================== Domain ====================+ getDomainByName(backendId: number, domain: string) { return this.repos.domain.getDomainByName(backendId, domain); }+ getDomainStats(backendId: number, limit = 100, start?: string, end?: string) { return this.repos.domain.getDomainStats(backendId, limit, start, end); }+ getTopDomains(backendId: number, limit = 10, start?: string, end?: string) { return this.repos.domain.getTopDomains(backendId, limit, start, end); }+ getDomainStatsPaginated(backendId: number, opts: { offset?: number; limit?: number; sortBy?: string; sortOrder?: string; search?: string; start?: string; end?: string } = {}) { return this.repos.domain.getDomainStatsPaginated(backendId, opts); }+ getDomainIPDetails(backendId: number, domain: string, start?: string, end?: string, limit?: number, sourceIP?: string, sourceChain?: string) { return this.repos.ip.getDomainIPDetails(backendId, domain, start, end, limit, sourceIP, sourceChain); }+ getDomainProxyStats(backendId: number, domain: string, start?: string, end?: string, sourceIP?: string, sourceChain?: string) { return this.repos.domain.getDomainProxyStats(backendId, domain, start, end, sourceIP, sourceChain); }++ // ==================== IP ====================+ getIPStats(backendId: number, limit = 100, start?: string, end?: string) { return this.repos.ip.getIPStats(backendId, limit, start, end); }+ getIPStatsByIPs(backendId: number, ips: string[]) { return this.repos.ip.getIPStatsByIPs(backendId, ips); }+ getIPStatsPaginated(backendId: number, opts: { offset?: number; limit?: number; sortBy?: string; sortOrder?: string; search?: string; start?: string; end?: string } = {}) { return this.repos.ip.getIPStatsPaginated(backendId, opts); }+ getIPDomainDetails(backendId: number, ip: string, start?: string, end?: string, limit?: number, sourceIP?: string, sourceChain?: string) { return this.repos.ip.getIPDomainDetails(backendId, ip, start, end, limit, sourceIP, sourceChain); }+ getIPProxyStats(backendId: number, ip: string, start?: string, end?: string, sourceIP?: string, sourceChain?: string) { return this.repos.ip.getIPProxyStats(backendId, ip, start, end, sourceIP, sourceChain); }+ getTopIPs(backendId: number, limit = 10, start?: string, end?: string) { return this.getIPStats(backendId, limit, start, end); }+ updateASNInfo(ip: string, asn: string, org: string) { this.repos.ip.updateASNInfo(ip, asn, org); }+ getASNInfo(ips: string[]) { return this.repos.ip.getASNInfo(ips); }+ getASNInfoForIP(ip: string) { return this.repos.ip.getASNInfoForIP(ip); }+ getIPGeolocation(ip: string) { return this.repos.ip.getIPGeolocation(ip); }+ saveIPGeolocation(ip: string, geo: { country: string; country_name: string; city: string; asn: string; as_name: string; as_domain: string; continent: string; continent_name: string }) { this.repos.ip.saveIPGeolocation(ip, geo); }++ // ==================== Timeseries ====================+ getHourlyStats(backendId: number, hours = 24, start?: string, end?: string) { return this.repos.timeseries.getHourlyStats(backendId, hours, start, end); }+ getTodayTraffic(backendId: number) { return this.repos.timeseries.getTodayTraffic(backendId); }+ getTrafficInRange(backendId: number, start?: string, end?: string) { return this.repos.timeseries.getTrafficInRange(backendId, start, end); }+ getTrafficTrend(backendId: number, minutes?: number, start?: string, end?: string) { return this.repos.timeseries.getTrafficTrend(backendId, minutes, start, end); }+ getTrafficTrendAggregated(backendId: number, minutes?: number, bucketMinutes?: number, start?: string, end?: string) { return this.repos.timeseries.getTrafficTrendAggregated(backendId, minutes, bucketMinutes, start, end); }++ // ==================== Country ====================+ getCountryStats(backendId: number, limit?: number, start?: string, end?: string) { return this.repos.country.getCountryStats(backendId, limit, start, end); }+ updateCountryStats(backendId: number, country: string, countryName: string, continent: string, upload: number, download: number) { this.repos.country.updateCountryStats(backendId, country, countryName, continent, upload, download); }+ batchUpdateCountryStats(backendId: number, results: Array<{ country: string; countryName: string; continent: string; upload: number; download: number; timestampMs?: number }>) { this.repos.country.batchUpdateCountryStats(backendId, results); }++ // ==================== Device ====================+ getDevices(backendId: number, limit?: number, start?: string, end?: string) { return this.repos.device.getDevices(backendId, limit, start, end); }+ getDeviceDomains(backendId: number, sourceIP: string, limit?: number, start?: string, end?: string) { return this.repos.device.getDeviceDomains(backendId, sourceIP, limit, start, end); }+ getDeviceIPs(backendId: number, sourceIP: string, limit?: number, start?: string, end?: string) { return this.repos.device.getDeviceIPs(backendId, sourceIP, limit, start, end); }++ // ==================== Proxy ====================+ getProxyStats(backendId: number, start?: string, end?: string) { return this.repos.proxy.getProxyStats(backendId, start, end); }+ getProxyDomains(backendId: number, chain: string, limit?: number, start?: string, end?: string) { return this.repos.proxy.getProxyDomains(backendId, chain, limit, start, end); }+ getProxyIPs(backendId: number, chain: string, limit?: number, start?: string, end?: string) { return this.repos.proxy.getProxyIPs(backendId, chain, limit, start, end); }++ // ==================== Rule ====================+ getRuleStats(backendId: number, start?: string, end?: string) { return this.repos.rule.getRuleStats(backendId, start, end); }+ getRuleProxyMap(backendId: number) { return this.repos.rule.getRuleProxyMap(backendId); }+ getRuleDomains(backendId: number, rule: string, limit?: number, start?: string, end?: string) { return this.repos.rule.getRuleDomains(backendId, rule, limit, start, end); }+ getRuleIPs(backendId: number, rule: string, limit?: number, start?: string, end?: string) { return this.repos.rule.getRuleIPs(backendId, rule, limit, start, end); }+ getRuleDomainProxyStats(backendId: number, rule: string, domain: string, start?: string, end?: string) { return this.repos.rule.getRuleDomainProxyStats(backendId, rule, domain, start, end); }+ getRuleDomainIPDetails(backendId: number, rule: string, domain: string, start?: string, end?: string, limit?: number) { return this.repos.rule.getRuleDomainIPDetails(backendId, rule, domain, start, end, limit); }+ getRuleIPProxyStats(backendId: number, rule: string, ip: string, start?: string, end?: string) { return this.repos.rule.getRuleIPProxyStats(backendId, rule, ip, start, end); }+ getRuleIPDomainDetails(backendId: number, rule: string, ip: string, start?: string, end?: string, limit?: number) { return this.repos.rule.getRuleIPDomainDetails(backendId, rule, ip, start, end, limit); }+ getRuleChainFlow(backendId: number, rule: string, start?: string, end?: string) { return this.repos.rule.getRuleChainFlow(backendId, rule, start, end); }+ getAllRuleChainFlows(backendId: number, start?: string, end?: string) { return this.repos.rule.getAllRuleChainFlows(backendId, start, end); }++ // ==================== Private helpers (kept for getSummary) ====================private toMinuteKey(date: Date): string {return `${date.toISOString().slice(0, 16)}:00`;}- private toHourKey(date: Date): string {- return `${date.toISOString().slice(0, 13)}:00:00`;- }-private parseMinuteRange(start?: string,end?: string,): { startMinute: string; endMinute: string } | null {- if (!start || !end) {- return null;- }-+ if (!start || !end) return null;const startDate = new Date(start);const endDate = new Date(end);- if (- Number.isNaN(startDate.getTime()) ||- Number.isNaN(endDate.getTime()) ||- startDate > endDate- ) {- return null;- }-- return {- startMinute: this.toMinuteKey(startDate),- endMinute: this.toMinuteKey(endDate),- };- }-- private splitChainParts(chain: string): string[] {- return chain- .split(">")- .map((part) => part.trim())- .filter(Boolean);- }-- private normalizeFlowLabel(label: string): string {- return label- .normalize("NFKC")- .replace(/^[^\p{L}\p{N}]+/gu, "")- .replace(/\s+/g, " ")- .trim()- .toLowerCase();- }-- private findRuleIndexInChain(chainParts: string[], rule: string): number {- const exactIndex = chainParts.findIndex((part) => part === rule);- if (exactIndex !== -1) {- return exactIndex;- }-- const normalizedRule = this.normalizeFlowLabel(rule);- if (!normalizedRule) {- return -1;- }-- return chainParts.findIndex(- (part) => this.normalizeFlowLabel(part) === normalizedRule,- );- }-- private getChainFirstHop(chain: string): string {- const parts = this.splitChainParts(chain);- return parts[0] || chain;- }-- private aggregateProxyStatsByFirstHop(rows: ProxyStats[]): ProxyStats[] {- const merged = new Map<string, ProxyStats>();-- for (const row of rows) {- const hop = this.getChainFirstHop(row.chain || "") || "DIRECT";- const existing = merged.get(hop);- if (existing) {- existing.totalUpload += row.totalUpload;- existing.totalDownload += row.totalDownload;- existing.totalConnections += row.totalConnections;- if (row.lastSeen > existing.lastSeen) {- existing.lastSeen = row.lastSeen;- }- } else {- merged.set(hop, {- chain: hop,- totalUpload: row.totalUpload,- totalDownload: row.totalDownload,- totalConnections: row.totalConnections,- lastSeen: row.lastSeen,- });- }- }-- return Array.from(merged.values()).sort(- (a, b) => (b.totalDownload + b.totalUpload) - (a.totalDownload + a.totalUpload),- );- }-- private allocateByWeights(total: number, weights: number[]): number[] {- if (weights.length === 0) return [];- const sum = weights.reduce((acc, w) => acc + w, 0);- if (sum <= 0) {- const base = Math.floor(total / weights.length);- const result = new Array(weights.length).fill(base);- let remainder = total - base * weights.length;- for (let i = 0; i < result.length && remainder > 0; i++, remainder--) {- result[i] += 1;- }- return result;- }-- const raw = weights.map(w => (total * w) / sum);- const floored = raw.map(v => Math.floor(v));- let remainder = total - floored.reduce((acc, v) => acc + v, 0);- const order = raw- .map((v, i) => ({ i, frac: v - Math.floor(v) }))- .sort((a, b) => b.frac - a.frac);- for (let k = 0; k < order.length && remainder > 0; k++, remainder--) {- floored[order[k].i] += 1;- }- return floored;+ if (Number.isNaN(startDate.getTime()) || Number.isNaN(endDate.getTime()) || startDate > endDate) return null;+ return { startMinute: this.toMinuteKey(startDate), endMinute: this.toMinuteKey(endDate) };}- /**- * minute_dim_stats stores proxy hop in `chain` (e.g. DIRECT), which loses- * middle nodes. For ranged chain-flow queries we remap range traffic onto- * full chains from cumulative rule_chain_traffic to preserve topology.- */- private remapRangeRowsToFullChains(- rangeRows: Array<{- rule: string;- chain: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- }>,- baselineRows: Array<{- rule: string;- chain: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- }>,- ): Array<{- rule: string;- chain: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- }> {- if (rangeRows.length === 0) return [];- if (baselineRows.length === 0) return rangeRows;-- const baselineByRuleHop = new Map<string, typeof baselineRows>();- const baselineByNormalizedRuleHop = new Map<string, typeof baselineRows>();- for (const row of baselineRows) {- const hop = this.getChainFirstHop(row.chain);- const key = `${row.rule}|||${hop}`;- const list = baselineByRuleHop.get(key);- if (list) {- list.push(row);- } else {- baselineByRuleHop.set(key, [row]);- }-- const normalizedRule = this.normalizeFlowLabel(row.rule);- if (!normalizedRule) continue;- const normalizedKey = `${normalizedRule}|||${hop}`;- const normalizedList = baselineByNormalizedRuleHop.get(normalizedKey);- if (normalizedList) {- normalizedList.push(row);- } else {- baselineByNormalizedRuleHop.set(normalizedKey, [row]);- }- }-- const mapped: Array<{- rule: string;- chain: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- }> = [];-- for (const row of rangeRows) {- const parts = this.splitChainParts(row.chain);- const alreadyFull =- parts.length > 1 && this.findRuleIndexInChain(parts, row.rule) !== -1;- if (alreadyFull) {- mapped.push(row);- continue;- }-- const hop = this.getChainFirstHop(row.chain);- const normalizedRule = this.normalizeFlowLabel(row.rule);- const candidates =- baselineByRuleHop.get(`${row.rule}|||${hop}`) ||- (normalizedRule- ? baselineByNormalizedRuleHop.get(`${normalizedRule}|||${hop}`)- : undefined);- if (!candidates || candidates.length === 0) {- mapped.push(row);- continue;- }-- if (candidates.length === 1) {- mapped.push({- rule: row.rule,- chain: candidates[0].chain,- totalUpload: row.totalUpload,- totalDownload: row.totalDownload,- totalConnections: row.totalConnections,- });- continue;- }-- const weights = candidates.map(c => {- const traffic = c.totalUpload + c.totalDownload;- return traffic > 0 ? traffic : Math.max(1, c.totalConnections);- });- const uploadParts = this.allocateByWeights(row.totalUpload, weights);- const downloadParts = this.allocateByWeights(row.totalDownload, weights);- const connParts = this.allocateByWeights(row.totalConnections, weights);- for (let i = 0; i < candidates.length; i++) {- mapped.push({- rule: row.rule,- chain: candidates[i].chain,- totalUpload: uploadParts[i] || 0,- totalDownload: downloadParts[i] || 0,- totalConnections: connParts[i] || 0,- });- }- }-- return mapped;- }-- /**- * Build a normalized rule flow path in "rule -> ... -> proxy" order.- * Legacy cumulative rows often store full chain ending with rule, while- * minute_dim rows may only store proxy/group chain without rule.- */- private buildRuleFlowPath(rule: string, chain: string): string[] {- const chainParts = this.splitChainParts(chain);- if (chainParts.length === 0) {- return [];- }-- const ruleIndex = this.findRuleIndexInChain(chainParts, rule);- if (ruleIndex !== -1) {- // Full chain stored as proxy > ... > rule, reverse to rule > ... > proxy.- return chainParts.slice(0, ruleIndex + 1).reverse();- }-- // Fallback for mismatched labels or minute_dim rows:- // normalize direction to rule/group -> ... -> proxy.- const reversed = [...chainParts].reverse();- const normalizedRule = this.normalizeFlowLabel(rule);- const normalizedHead = this.normalizeFlowLabel(reversed[0] || "");- if (normalizedRule && normalizedRule === normalizedHead) {- return reversed;- }-- return [rule, ...reversed];- }-- private uniqueNonEmpty(values: string[]): string[] {- const seen = new Set<string>();- const out: string[] = [];- for (const raw of values) {- const v = (raw || "").trim();- if (!v || seen.has(v)) continue;- seen.add(v);- out.push(v);- }- return out;- }-- /**- * Some range queries aggregate from minute_dim_stats where historical rows may- * only contain the final proxy hop in `chain`. This method expands those- * short hops to full chains using rule_chain_traffic when possible.- */- private expandShortChainsForRules(+ // ==================== Summary (kept in db.ts - cross-table aggregation) ====================+ getSummary(backendId: number,- chains: string[],- rules: string[],- ): string[] {- const normalizedChains = this.uniqueNonEmpty(chains);- if (normalizedChains.length === 0) return [];-- const shortChains = normalizedChains.filter((c) => !c.includes(">"));- if (shortChains.length === 0) return normalizedChains;-- const normalizedRules = this.uniqueNonEmpty(rules);- const whereParts: string[] = [];- const params: Array<string | number> = [backendId];-- if (normalizedRules.length > 0) {- const rulePlaceholders = normalizedRules.map(() => "?").join(", ");- whereParts.push(`rule IN (${rulePlaceholders})`);- params.push(...normalizedRules);- }-- const chainMatchers: string[] = [];- for (const chain of shortChains) {- chainMatchers.push("(chain = ? OR chain LIKE ?)");- params.push(chain, `${chain} > %`);- }- whereParts.push(`(${chainMatchers.join(" OR ")})`);-- const whereClause = whereParts.length > 0 ? `AND ${whereParts.join(" AND ")}` : "";- const stmt = this.db.prepare(`- SELECT DISTINCT chain- FROM rule_chain_traffic- WHERE backend_id = ? ${whereClause}- LIMIT 500- `);-- const rows = stmt.all(...params) as Array<{ chain: string }>;- const expanded = this.uniqueNonEmpty(rows.map((r) => r.chain));-- if (expanded.length === 0) {- return normalizedChains;- }-- // Prefer expanded full chains, but keep already-full values from input too.- const fullInputChains = normalizedChains.filter((c) => c.includes(">"));- return this.uniqueNonEmpty([...expanded, ...fullInputChains]);- }-- // Get a specific domain by name- getDomainByName(backendId: number, domain: string): DomainStats | null {- const stmt = this.db.prepare(`- SELECT domain, total_upload as totalUpload, total_download as totalDownload,- total_connections as totalConnections, last_seen as lastSeen, ips, rules, chains- FROM domain_stats- WHERE backend_id = ? AND domain = ?- `);- const row = stmt.get(backendId, domain) as {- domain: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- lastSeen: string;- ips: string;- rules: string | null;- chains: string | null;- } | undefined;-- if (!row) return null;-- return {- ...row,- ips: row.ips ? row.ips.split(',') : [],- rules: row.rules ? row.rules.split(',') : [],- chains: row.chains ? row.chains.split(',') : [],- } as DomainStats;- }-- // Get all domain stats for a specific backend- getDomainStats(backendId: number, limit = 100, start?: string, end?: string): DomainStats[] {+ start?: string,+ end?: string,+ ): { totalConnections: number; totalUpload: number; totalDownload: number; uniqueDomains: number; uniqueIPs: number } {const range = this.parseMinuteRange(start, end);if (range) {- const stmt = this.db.prepare(`+ const trafficStmt = this.db.prepare(`SELECT- domain,- SUM(upload) as totalUpload,- SUM(download) as totalDownload,- SUM(connections) as totalConnections,- MAX(minute) as lastSeen,- '' as ips,- GROUP_CONCAT(DISTINCT rule) as rules,- GROUP_CONCAT(DISTINCT chain) as chains+ COALESCE(SUM(connections), 0) as connections,+ COALESCE(SUM(upload), 0) as upload,+ COALESCE(SUM(download), 0) as downloadFROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND domain != ''- GROUP BY domain- ORDER BY (SUM(upload) + SUM(download)) DESC- LIMIT ?+ WHERE backend_id = ? AND minute >= ? AND minute <= ?`);- const rows = stmt.all(- backendId,- range.startMinute,- range.endMinute,- limit,- ) as Array<{- domain: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- lastSeen: string;- ips: string | null;- rules: string | null;- chains: string | null;- }>;-- return rows.map(row => {- const rules = row.rules ? row.rules.split(',').filter(Boolean) : [];- const chains = row.chains ? row.chains.split(',').filter(Boolean) : [];- return {- ...row,- ips: row.ips ? row.ips.split(',').filter(Boolean) : [],- rules,- chains: this.expandShortChainsForRules(backendId, chains, rules),- };- }) as DomainStats[];+ const { connections, upload, download } = trafficStmt.get(backendId, range.startMinute, range.endMinute) as { connections: number; upload: number; download: number };+ const uniqueDomains = (this.db.prepare(`SELECT COUNT(DISTINCT domain) as count FROM minute_dim_stats WHERE backend_id = ? AND minute >= ? AND minute <= ? AND domain != ''`).get(backendId, range.startMinute, range.endMinute) as { count: number }).count;+ const uniqueIPs = (this.db.prepare(`SELECT COUNT(DISTINCT ip) as count FROM minute_dim_stats WHERE backend_id = ? AND minute >= ? AND minute <= ? AND ip != ''`).get(backendId, range.startMinute, range.endMinute) as { count: number }).count;+ return { totalConnections: connections, totalUpload: upload, totalDownload: download, uniqueDomains, uniqueIPs };}+ const { connections, upload, download } = this.db.prepare(`SELECT COALESCE(SUM(total_connections), 0) as connections, COALESCE(SUM(total_upload), 0) as upload, COALESCE(SUM(total_download), 0) as download FROM ip_stats WHERE backend_id = ?`).get(backendId) as { connections: number; upload: number; download: number };+ const uniqueDomains = (this.db.prepare(`SELECT COUNT(DISTINCT domain) as count FROM domain_stats WHERE backend_id = ?`).get(backendId) as { count: number }).count;+ const uniqueIPs = (this.db.prepare(`SELECT COUNT(DISTINCT ip) as count FROM ip_stats WHERE backend_id = ?`).get(backendId) as { count: number }).count;+ return { totalConnections: connections, totalUpload: upload, totalDownload: download, uniqueDomains, uniqueIPs };+ }- const stmt = this.db.prepare(`- SELECT domain, total_upload as totalUpload, total_download as totalDownload,- total_connections as totalConnections, last_seen as lastSeen, ips, rules, chains- FROM domain_stats- WHERE backend_id = ?- ORDER BY (total_upload + total_download) DESC- LIMIT ?- `);- const rows = stmt.all(backendId, limit) as Array<{- domain: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- lastSeen: string;- ips: string | null;- rules: string | null;- chains: string | null;- }>;-- return rows.map(row => {- const rules = row.rules ? row.rules.split(',').filter(Boolean) : [];- const chains = row.chains ? row.chains.split(',').filter(Boolean) : [];- return {- ...row,- ips: row.ips ? row.ips.split(',') : [],- rules,- chains: this.expandShortChainsForRules(backendId, chains, rules),- };- }) as DomainStats[];+ // Get recent connections for a specific backend+ // Returns empty array - connection_logs is no longer written to (replaced by aggregation tables)+ getRecentConnections(backendId: number, limit = 100): Connection[] {+ return [];}- // Get IP stats for specific IPs (used for domain IP details)- getIPStatsByIPs(backendId: number, ips: string[]): IPStats[] {- const filteredIps = ips.filter(ip => ip && ip.trim() !== '');- if (filteredIps.length === 0) return [];-- const placeholders = filteredIps.map(() => '?').join(',');- const stmt = this.db.prepare(`- SELECT- i.ip,- i.domains,- i.total_upload as totalUpload,- i.total_download as totalDownload,- i.total_connections as totalConnections,- i.last_seen as lastSeen,- COALESCE(i.asn, g.asn) as asn,- CASE- WHEN g.country IS NOT NULL THEN- json_array(- g.country,- COALESCE(g.country_name, g.country),- COALESCE(g.city, ''),- COALESCE(g.as_name, '')- )- WHEN i.geoip IS NOT NULL THEN- json(i.geoip)- ELSE- NULL- END as geoIP,- i.chains- FROM ip_stats i- LEFT JOIN geoip_cache g ON i.ip = g.ip- WHERE i.backend_id = ? AND i.ip IN (${placeholders})- ORDER BY (i.total_upload + i.total_download) DESC- `);- const rows = stmt.all(backendId, ...filteredIps) as Array<{- ip: string;- domains: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- lastSeen: string;- asn: string | null;- geoIP: string | null;- chains: string | null;- }>;-- return rows.map(row => ({- ...row,- domains: row.domains ? row.domains.split(',') : [],- geoIP: row.geoIP ? JSON.parse(row.geoIP).filter(Boolean) : undefined,- asn: row.asn || undefined,- chains: row.chains ? row.chains.split(',') : [],- })) as IPStats[];- }-- // Get IP details for a specific domain (supports optional time range)- getDomainIPDetails(- backendId: number,- domain: string,- start?: string,- end?: string,- limit = 100,- sourceIP?: string,- sourceChain?: string,- ): IPStats[] {- const range = this.parseMinuteRange(start, end);- if (range || sourceIP || sourceChain) {- const conditions = ["m.backend_id = ?", "m.domain = ?", "m.ip != ''"];- const params: Array<string | number> = [backendId, domain];- if (range) {- conditions.push("m.minute >= ?", "m.minute <= ?");- params.push(range.startMinute, range.endMinute);- }- if (sourceIP) {- conditions.push("m.source_ip = ?");- params.push(sourceIP);- }- if (sourceChain) {- conditions.push("(m.chain = ? OR m.chain LIKE ?)");- params.push(sourceChain, `${sourceChain} > %`);- }-- const stmt = this.db.prepare(`- SELECT- m.ip,- GROUP_CONCAT(DISTINCT CASE WHEN m.domain != '' THEN m.domain END) as domains,- SUM(m.upload) as totalUpload,- SUM(m.download) as totalDownload,- SUM(m.connections) as totalConnections,- MAX(m.minute) as lastSeen,- COALESCE(i.asn, g.asn) as asn,- CASE- WHEN g.country IS NOT NULL THEN- json_array(- g.country,- COALESCE(g.country_name, g.country),- COALESCE(g.city, ''),- COALESCE(g.as_name, '')- )- WHEN i.geoip IS NOT NULL THEN- json(i.geoip)- ELSE- NULL- END as geoIP,- GROUP_CONCAT(DISTINCT m.chain) as chains,- GROUP_CONCAT(DISTINCT m.rule) as rules- FROM minute_dim_stats m- LEFT JOIN ip_stats i ON m.backend_id = i.backend_id AND m.ip = i.ip- LEFT JOIN geoip_cache g ON m.ip = g.ip- WHERE ${conditions.join(" AND ")}- GROUP BY m.ip- ORDER BY (SUM(m.upload) + SUM(m.download)) DESC- LIMIT ?- `);- const rows = stmt.all(...params, limit) as Array<{- ip: string;- domains: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- lastSeen: string;- asn: string | null;- geoIP: string | null;- chains: string | null;- rules: string | null;- }>;-- return rows.map(row => {- const rules = row.rules ? row.rules.split(',').filter(Boolean) : [];- const chains = row.chains ? row.chains.split(',').filter(Boolean) : [];- return {- ...row,- domains: row.domains ? row.domains.split(',').filter(Boolean) : [],- geoIP: row.geoIP ? JSON.parse(row.geoIP).filter(Boolean) : undefined,- asn: row.asn || undefined,- chains: this.expandShortChainsForRules(backendId, chains, rules),- };- }) as IPStats[];- }-- const domainData = this.getDomainByName(backendId, domain);- if (!domainData || !domainData.ips || domainData.ips.length === 0) {- return [];- }- return this.getIPStatsByIPs(backendId, domainData.ips.slice(0, limit));- }-- // Get domain details for a specific IP (supports optional time range and sourceIP filter)- getIPDomainDetails(- backendId: number,- ip: string,- start?: string,- end?: string,- limit = 100,- sourceIP?: string,- sourceChain?: string,- ): DomainStats[] {- const range = this.parseMinuteRange(start, end);- const conditions = ["backend_id = ?", "ip = ?", "domain != ''"];- const params: Array<string | number> = [backendId, ip];-- if (range) {- conditions.push("minute >= ?", "minute <= ?");- params.push(range.startMinute, range.endMinute);- }- if (sourceIP) {- conditions.push("source_ip = ?");- params.push(sourceIP);- }- if (sourceChain) {- conditions.push("(chain = ? OR chain LIKE ?)");- params.push(sourceChain, `${sourceChain} > %`);- }-- const stmt = this.db.prepare(`- SELECT- domain,- GROUP_CONCAT(DISTINCT ip) as ips,- SUM(upload) as totalUpload,- SUM(download) as totalDownload,- SUM(connections) as totalConnections,- MAX(minute) as lastSeen,- GROUP_CONCAT(DISTINCT CASE WHEN rule != '' THEN rule END) as rules,- GROUP_CONCAT(DISTINCT chain) as chains- FROM minute_dim_stats- WHERE ${conditions.join(" AND ")}- GROUP BY domain- ORDER BY (SUM(upload) + SUM(download)) DESC- LIMIT ?- `);-- const rows = stmt.all(...params, limit) as Array<{- domain: string;- ips: string | null;- totalUpload: number;- totalDownload: number;- totalConnections: number;- lastSeen: string;- rules: string | null;- chains: string | null;- }>;-- return rows.map(row => {- const rules = row.rules ? row.rules.split(',').filter(Boolean) : [];- const chains = row.chains ? row.chains.split(',').filter(Boolean) : [];- return {- domain: row.domain,- ips: row.ips ? row.ips.split(',').filter(Boolean) : [],- totalUpload: row.totalUpload,- totalDownload: row.totalDownload,- totalConnections: row.totalConnections,- lastSeen: row.lastSeen,- rules,- chains: this.expandShortChainsForRules(backendId, chains, rules),- };- }) as DomainStats[];- }-- // Get all IP stats for a specific backend- getIPStats(backendId: number, limit = 100, start?: string, end?: string): IPStats[] {- const range = this.parseMinuteRange(start, end);- if (range) {- const stmt = this.db.prepare(`- SELECT- m.ip,- GROUP_CONCAT(DISTINCT CASE WHEN m.domain != '' THEN m.domain END) as domains,- SUM(m.upload) as totalUpload,- SUM(m.download) as totalDownload,- SUM(m.connections) as totalConnections,- MAX(m.minute) as lastSeen,- COALESCE(i.asn, g.asn) as asn,- CASE- WHEN g.country IS NOT NULL THEN- json_array(- g.country,- COALESCE(g.country_name, g.country),- COALESCE(g.city, ''),- COALESCE(g.as_name, '')- )- WHEN i.geoip IS NOT NULL THEN- json(i.geoip)- ELSE- NULL- END as geoIP,- GROUP_CONCAT(DISTINCT m.chain) as chains,- GROUP_CONCAT(DISTINCT m.rule) as rules- FROM minute_dim_stats m- LEFT JOIN ip_stats i ON m.backend_id = i.backend_id AND m.ip = i.ip- LEFT JOIN geoip_cache g ON m.ip = g.ip- WHERE m.backend_id = ? AND m.minute >= ? AND m.minute <= ? AND m.ip != ''- GROUP BY m.ip- ORDER BY (SUM(m.upload) + SUM(m.download)) DESC- LIMIT ?- `);- const rows = stmt.all(- backendId,- range.startMinute,- range.endMinute,- limit,- ) as Array<{- ip: string;- domains: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- lastSeen: string;- asn: string | null;- geoIP: string | null;- chains: string | null;- rules: string | null;- }>;-- return rows.map(row => {- const rules = row.rules ? row.rules.split(',').filter(Boolean) : [];- const chains = row.chains ? row.chains.split(',').filter(Boolean) : [];- return {- ...row,- domains: row.domains ? row.domains.split(',').filter(Boolean) : [],- geoIP: row.geoIP ? JSON.parse(row.geoIP).filter(Boolean) : undefined,- asn: row.asn || undefined,- chains: this.expandShortChainsForRules(backendId, chains, rules),- };- }) as IPStats[];- }-- const stmt = this.db.prepare(`- SELECT- i.ip,- i.domains,- i.total_upload as totalUpload,- i.total_download as totalDownload,- i.total_connections as totalConnections,- i.last_seen as lastSeen,- COALESCE(i.asn, g.asn) as asn,- CASE- WHEN g.country IS NOT NULL THEN- json_array(- g.country,- COALESCE(g.country_name, g.country),- COALESCE(g.city, ''),- COALESCE(g.as_name, '')- )- WHEN i.geoip IS NOT NULL THEN- json(i.geoip)- ELSE- NULL- END as geoIP,- i.chains- FROM ip_stats i- LEFT JOIN geoip_cache g ON i.ip = g.ip- WHERE i.backend_id = ? AND i.ip != ''- ORDER BY (i.total_upload + i.total_download) DESC- LIMIT ?- `);- const rows = stmt.all(backendId, limit) as Array<{- ip: string;- domains: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- lastSeen: string;- asn: string | null;- geoIP: string | null;- chains: string | null;- }>;-- return rows.map(row => ({- ...row,- domains: row.domains ? row.domains.split(',') : [],- geoIP: row.geoIP ? JSON.parse(row.geoIP).filter(Boolean) : undefined,- asn: row.asn || undefined,- chains: row.chains ? row.chains.split(',') : [],- })) as IPStats[];- }-- // Get hourly stats for a specific backend- getHourlyStats(backendId: number, hours = 24, start?: string, end?: string): HourlyStats[] {- const range = this.parseMinuteRange(start, end);- if (range) {- const stmt = this.db.prepare(`- SELECT- substr(minute, 1, 13) || ':00:00' as hour,- SUM(upload) as upload,- SUM(download) as download,- SUM(connections) as connections- FROM minute_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ?- GROUP BY substr(minute, 1, 13)- ORDER BY hour DESC- LIMIT ?- `);- return stmt.all(backendId, range.startMinute, range.endMinute, hours) as HourlyStats[];- }-- const stmt = this.db.prepare(`- SELECT hour, upload, download, connections- FROM hourly_stats- WHERE backend_id = ?- ORDER BY hour DESC- LIMIT ?- `);- return stmt.all(backendId, hours) as HourlyStats[];- }-- // Get today's traffic for a specific backend- getTodayTraffic(backendId: number): { upload: number; download: number } {- const today = new Date().toISOString().split('T')[0]; // YYYY-MM-DD- const stmt = this.db.prepare(`- SELECT COALESCE(SUM(upload), 0) as upload, COALESCE(SUM(download), 0) as download- FROM hourly_stats- WHERE backend_id = ? AND hour >= ?- `);- return stmt.get(backendId, today) as { upload: number; download: number };- }-- getTrafficInRange(- backendId: number,- start?: string,- end?: string,- ): { upload: number; download: number } {- const range = this.parseMinuteRange(start, end);- if (!range) {- return this.getTodayTraffic(backendId);- }-- const stmt = this.db.prepare(`- SELECT- COALESCE(SUM(upload), 0) as upload,- COALESCE(SUM(download), 0) as download- FROM minute_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ?- `);- return stmt.get(- backendId,- range.startMinute,- range.endMinute,- ) as { upload: number; download: number };- }-- // Get traffic trend for a specific backend (for time range selection)- getTrafficTrend(- backendId: number,- minutes = 30,- start?: string,- end?: string,- ): Array<{ time: string; upload: number; download: number }> {- const range = this.parseMinuteRange(start, end);- const cutoff = new Date(Date.now() - minutes * 60 * 1000);- const cutoffStr = cutoff.toISOString().slice(0, 16) + ':00';- const stmt = this.db.prepare(`- SELECT minute as time, upload, download- FROM minute_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ?- ORDER BY minute ASC- `);- return stmt.all(- backendId,- range?.startMinute || cutoffStr,- range?.endMinute || this.toMinuteKey(new Date()),- ) as Array<{ time: string; upload: number; download: number }>;- }-- // Get traffic trend aggregated by time buckets for chart display- getTrafficTrendAggregated(- backendId: number,- minutes = 30,- bucketMinutes = 1,- start?: string,- end?: string,- ): Array<{ time: string; upload: number; download: number }> {- const range = this.parseMinuteRange(start, end);- const cutoff = new Date(Date.now() - minutes * 60 * 1000);- const cutoffStr = cutoff.toISOString().slice(0, 16) + ':00';- const endMinute = range?.endMinute || this.toMinuteKey(new Date());-- if (bucketMinutes <= 1) {- // Direct query - each row is already 1-minute granularity- const stmt = this.db.prepare(`- SELECT minute as time, upload, download- FROM minute_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ?- ORDER BY minute ASC- `);- return stmt.all(- backendId,- range?.startMinute || cutoffStr,- endMinute,- ) as Array<{ time: string; upload: number; download: number }>;- }-- // Aggregate multiple minutes into larger buckets- const bucketExpr = `strftime('%Y-%m-%dT%H:%M:00', datetime((strftime('%s', datetime(minute)) / ${bucketMinutes * 60}) * ${bucketMinutes * 60}, 'unixepoch'))`;- const stmt = this.db.prepare(`- SELECT- ${bucketExpr} as time,- SUM(upload) as upload,- SUM(download) as download- FROM minute_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ?- GROUP BY ${bucketExpr}- ORDER BY time ASC- `);- return stmt.all(- backendId,- range?.startMinute || cutoffStr,- endMinute,- ) as Array<{ time: string; upload: number; download: number }>;- }-- // Get country stats for a specific backend- getCountryStats(- backendId: number,- limit = 50,- start?: string,- end?: string,- ): Array<{ country: string; countryName: string; continent: string; totalUpload: number; totalDownload: number; totalConnections: number }> {- const range = this.parseMinuteRange(start, end);- if (range) {- const minuteCountryStmt = this.db.prepare(`- SELECT- country,- MAX(country_name) as countryName,- MAX(continent) as continent,- SUM(upload) as totalUpload,- SUM(download) as totalDownload,- SUM(connections) as totalConnections- FROM minute_country_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ?- GROUP BY country- ORDER BY (SUM(upload) + SUM(download)) DESC- LIMIT ?- `);- const minuteCountryRows = minuteCountryStmt.all(- backendId,- range.startMinute,- range.endMinute,- limit,- ) as Array<{ country: string; countryName: string; continent: string; totalUpload: number; totalDownload: number; totalConnections: number }>;- if (minuteCountryRows.length > 0) {- return minuteCountryRows;- }-- // Fallback 1: derive country distribution from minute_dim_stats + geoip_cache- // This covers periods where minute_country_stats is missing but dimension facts exist.- const minuteDimFallbackStmt = this.db.prepare(`- SELECT- COALESCE(g.country, 'UNKNOWN') as country,- COALESCE(MAX(g.country_name), 'Unknown') as countryName,- COALESCE(MAX(g.continent), 'Unknown') as continent,- SUM(m.upload) as totalUpload,- SUM(m.download) as totalDownload,- SUM(m.connections) as totalConnections- FROM minute_dim_stats m- LEFT JOIN geoip_cache g ON m.ip = g.ip- WHERE m.backend_id = ? AND m.minute >= ? AND m.minute <= ? AND m.ip != ''- GROUP BY COALESCE(g.country, 'UNKNOWN')- ORDER BY (SUM(m.upload) + SUM(m.download)) DESC- LIMIT ?- `);- const minuteDimFallbackRows = minuteDimFallbackStmt.all(- backendId,- range.startMinute,- range.endMinute,- limit,- ) as Array<{ country: string; countryName: string; continent: string; totalUpload: number; totalDownload: number; totalConnections: number }>;- if (minuteDimFallbackRows.length > 0) {- return minuteDimFallbackRows;- }-- // Fallback 2: if only minute_stats exists for this range, keep totals visible as UNKNOWN country.- const totalStmt = this.db.prepare(`- SELECT- COALESCE(SUM(upload), 0) as upload,- COALESCE(SUM(download), 0) as download,- COALESCE(SUM(connections), 0) as connections- FROM minute_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ?- `);- const total = totalStmt.get(- backendId,- range.startMinute,- range.endMinute,- ) as { upload: number; download: number; connections: number };- if (total.upload > 0 || total.download > 0 || total.connections > 0) {- return [{- country: 'UNKNOWN',- countryName: 'Unknown',- continent: 'Unknown',- totalUpload: total.upload,- totalDownload: total.download,- totalConnections: total.connections,- }];- }-- return [];- }-- const stmt = this.db.prepare(`- SELECT country, country_name as countryName, continent,- total_upload as totalUpload, total_download as totalDownload,- total_connections as totalConnections- FROM country_stats- WHERE backend_id = ?- ORDER BY (total_upload + total_download) DESC- LIMIT ?- `);- return stmt.all(backendId, limit) as Array<{ country: string; countryName: string; continent: string; totalUpload: number; totalDownload: number; totalConnections: number }>;- }-- // Update country stats for a specific backend- updateCountryStats(backendId: number, country: string, countryName: string, continent: string, upload: number, download: number) {- const stmt = this.db.prepare(`- INSERT INTO country_stats (backend_id, country, country_name, continent, total_upload, total_download, total_connections, last_seen)- VALUES (?, ?, ?, ?, ?, ?, 1, CURRENT_TIMESTAMP)- ON CONFLICT(backend_id, country) DO UPDATE SET- total_upload = total_upload + ?,- total_download = total_download + ?,- total_connections = total_connections + 1,- last_seen = CURRENT_TIMESTAMP- `);- stmt.run(backendId, country, countryName, continent, upload, download, upload, download);- }-- // Batch update country stats - wraps all upserts in a single transaction- batchUpdateCountryStats(backendId: number, results: Array<{- country: string; countryName: string; continent: string;- upload: number; download: number;- timestampMs?: number;- }>): void {- if (results.length === 0) return;-- const cumulativeStmt = this.db.prepare(`- INSERT INTO country_stats (backend_id, country, country_name, continent, total_upload, total_download, total_connections, last_seen)- VALUES (?, ?, ?, ?, ?, ?, 1, CURRENT_TIMESTAMP)- ON CONFLICT(backend_id, country) DO UPDATE SET- total_upload = total_upload + ?,- total_download = total_download + ?,- total_connections = total_connections + 1,- last_seen = CURRENT_TIMESTAMP- `);-- const minuteStmt = this.db.prepare(`- INSERT INTO minute_country_stats (backend_id, minute, country, country_name, continent, upload, download, connections)- VALUES (@backendId, @minute, @country, @countryName, @continent, @upload, @download, @connections)- ON CONFLICT(backend_id, minute, country) DO UPDATE SET- upload = upload + @upload,- download = download + @download,- connections = connections + @connections- `);-- const tx = this.db.transaction(() => {- const minuteMap = new Map<string, {- minute: string;- country: string;- countryName: string;- continent: string;- upload: number;- download: number;- connections: number;- }>();-- for (const r of results) {- cumulativeStmt.run(backendId, r.country, r.countryName, r.continent, r.upload, r.download, r.upload, r.download);-- const minute = this.toMinuteKey(new Date(r.timestampMs ?? Date.now()));- const key = `${minute}:${r.country}`;- const existing = minuteMap.get(key);- if (existing) {- existing.upload += r.upload;- existing.download += r.download;- existing.connections++;- } else {- minuteMap.set(key, {- minute,- country: r.country,- countryName: r.countryName,- continent: r.continent,- upload: r.upload,- download: r.download,- connections: 1,- });- }- }-- for (const [, item] of minuteMap) {- minuteStmt.run({- backendId,- minute: item.minute,- country: item.country,- countryName: item.countryName,- continent: item.continent,- upload: item.upload,- download: item.download,- connections: item.connections,- });- }- });- tx();- }-- // Get device stats for a specific backend- getDevices(backendId: number, limit = 50, start?: string, end?: string): DeviceStats[] {- const range = this.parseMinuteRange(start, end);- if (range) {- const stmt = this.db.prepare(`- SELECT- source_ip as sourceIP,- SUM(upload) as totalUpload,- SUM(download) as totalDownload,- SUM(connections) as totalConnections,- MAX(minute) as lastSeen- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND source_ip != ''- GROUP BY source_ip- ORDER BY (SUM(upload) + SUM(download)) DESC- LIMIT ?- `);- return stmt.all(- backendId,- range.startMinute,- range.endMinute,- limit,- ) as DeviceStats[];- }-- const stmt = this.db.prepare(`- SELECT source_ip as sourceIP,- total_upload as totalUpload, total_download as totalDownload,- total_connections as totalConnections, last_seen as lastSeen- FROM device_stats- WHERE backend_id = ?- ORDER BY (total_upload + total_download) DESC- LIMIT ?- `);- return stmt.all(backendId, limit) as DeviceStats[];- }-- // Get domain breakdown for a specific device- getDeviceDomains(- backendId: number,- sourceIP: string,- limit = 5000,- start?: string,- end?: string,- ): DomainStats[] {- const range = this.parseMinuteRange(start, end);- if (range) {- const stmt = this.db.prepare(`- SELECT- domain,- SUM(upload) as totalUpload,- SUM(download) as totalDownload,- SUM(connections) as totalConnections,- MAX(minute) as lastSeen,- GROUP_CONCAT(DISTINCT ip) as ips,- GROUP_CONCAT(DISTINCT rule) as rules,- GROUP_CONCAT(DISTINCT chain) as chains- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND source_ip = ? AND domain != ''- GROUP BY domain- ORDER BY (SUM(upload) + SUM(download)) DESC- LIMIT ?- `);- const rows = stmt.all(- backendId,- range.startMinute,- range.endMinute,- sourceIP,- limit,- ) as Array<{- domain: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- lastSeen: string;- ips: string | null;- rules: string | null;- chains: string | null;- }>;- return rows.map(row => {- const rules = row.rules ? row.rules.split(',').filter(Boolean) : [];- const chains = row.chains ? row.chains.split(',').filter(Boolean) : [];- return {- ...row,- ips: row.ips ? row.ips.split(',').filter(Boolean) : [],- rules,- chains: this.expandShortChainsForRules(backendId, chains, rules),- };- }) as DomainStats[];- }-- const stmt = this.db.prepare(`- SELECT- d.domain,- d.total_upload as totalUpload,- d.total_download as totalDownload,- d.total_connections as totalConnections,- d.last_seen as lastSeen,- g.ips- FROM device_domain_stats d- LEFT JOIN domain_stats g ON d.domain = g.domain AND d.backend_id = g.backend_id- WHERE d.backend_id = ? AND d.source_ip = ?- ORDER BY (d.total_upload + d.total_download) DESC- LIMIT ?- `);- const result = stmt.all(backendId, sourceIP, limit) as any[];- return result.map(r => ({- ...r,- ips: r.ips ? r.ips.split(',') : [],- rules: [],- chains: []- }));- }-- // Get IP breakdown for a specific device- getDeviceIPs(- backendId: number,- sourceIP: string,- limit = 5000,- start?: string,- end?: string,- ): IPStats[] {- const range = this.parseMinuteRange(start, end);- if (range) {- const stmt = this.db.prepare(`- SELECT- m.ip,- SUM(m.upload) as totalUpload,- SUM(m.download) as totalDownload,- SUM(m.connections) as totalConnections,- MAX(m.minute) as lastSeen,- GROUP_CONCAT(DISTINCT CASE WHEN m.domain != '' THEN m.domain END) as domains,- COALESCE(i.asn, g.asn) as asn,- CASE- WHEN g.country IS NOT NULL THEN- json_array(- g.country,- COALESCE(g.country_name, g.country),- COALESCE(g.city, ''),- COALESCE(g.as_name, '')- )- WHEN i.geoip IS NOT NULL THEN- json(i.geoip)- ELSE- NULL- END as geoIP,- GROUP_CONCAT(DISTINCT m.chain) as chains,- GROUP_CONCAT(DISTINCT m.rule) as rules- FROM minute_dim_stats m- LEFT JOIN ip_stats i ON m.backend_id = i.backend_id AND m.ip = i.ip- LEFT JOIN geoip_cache g ON m.ip = g.ip- WHERE m.backend_id = ? AND m.minute >= ? AND m.minute <= ? AND m.source_ip = ? AND m.ip != ''- GROUP BY m.ip- ORDER BY (SUM(m.upload) + SUM(m.download)) DESC- LIMIT ?- `);- const rows = stmt.all(- backendId,- range.startMinute,- range.endMinute,- sourceIP,- limit,- ) as Array<{- ip: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- lastSeen: string;- domains: string | null;- asn: string | null;- geoIP: string | null;- chains: string | null;- rules: string | null;- }>;- return rows.map(row => {- const rules = row.rules ? row.rules.split(',').filter(Boolean) : [];- const chains = row.chains ? row.chains.split(',').filter(Boolean) : [];- return {- ...row,- domains: row.domains ? row.domains.split(',').filter(Boolean) : [],- geoIP: row.geoIP ? JSON.parse(row.geoIP).filter(Boolean) : undefined,- asn: row.asn || undefined,- chains: this.expandShortChainsForRules(backendId, chains, rules),- };- }) as IPStats[];- }-- const stmt = this.db.prepare(`- SELECT- d.ip,- d.total_upload as totalUpload,- d.total_download as totalDownload,- d.total_connections as totalConnections,- d.last_seen as lastSeen,- i.domains,- COALESCE(i.asn, g.asn) as asn,- CASE- WHEN g.country IS NOT NULL THEN- json_array(- g.country,- COALESCE(g.country_name, g.country),- COALESCE(g.city, ''),- COALESCE(g.as_name, '')- )- WHEN i.geoip IS NOT NULL THEN- json(i.geoip)- ELSE- NULL- END as geoIP- FROM device_ip_stats d- LEFT JOIN ip_stats i ON d.ip = i.ip AND d.backend_id = i.backend_id- LEFT JOIN geoip_cache g ON d.ip = g.ip- WHERE d.backend_id = ? AND d.source_ip = ?- ORDER BY (d.total_upload + d.total_download) DESC- LIMIT ?- `);- const result = stmt.all(backendId, sourceIP, limit) as any[];- return result.map(r => ({- ...r,- domains: r.domains ? r.domains.split(',') : [],- geoIP: r.geoIP ? JSON.parse(r.geoIP).filter(Boolean) : undefined,- asn: r.asn || undefined,- }));- }-- // Get top domains for a specific backend- getTopDomains(backendId: number, limit = 10, start?: string, end?: string): DomainStats[] {- return this.getDomainStats(backendId, limit, start, end);- }-- // Get top IPs for a specific backend- getTopIPs(backendId: number, limit = 10, start?: string, end?: string): IPStats[] {- return this.getIPStats(backendId, limit, start, end);- }-- // Get domain stats with server-side pagination, sorting and search- getDomainStatsPaginated(backendId: number, opts: {- offset?: number;- limit?: number;- sortBy?: string;- sortOrder?: string;- search?: string;- start?: string;- end?: string;- } = {}): { data: DomainStats[]; total: number } {- const offset = opts.offset ?? 0;- const limit = Math.min(opts.limit ?? 50, 200);- const sortOrder = opts.sortOrder === 'asc' ? 'ASC' : 'DESC';- const search = opts.search?.trim() || '';- const range = this.parseMinuteRange(opts.start, opts.end);-- const sortColumnMap: Record<string, string> = {- domain: 'domain',- totalDownload: 'total_download',- totalUpload: 'total_upload',- totalTraffic: '(total_upload + total_download)',- totalConnections: 'total_connections',- lastSeen: 'last_seen',- };- const sortColumn = sortColumnMap[opts.sortBy || 'totalDownload'] || 'total_download';-- if (range) {- const rangeSortColumnMap: Record<string, string> = {- domain: 'domain',- totalDownload: 'totalDownload',- totalUpload: 'totalUpload',- totalTraffic: '(totalUpload + totalDownload)',- totalConnections: 'totalConnections',- lastSeen: 'lastSeen',- };- const rangeSortColumn =- rangeSortColumnMap[opts.sortBy || 'totalDownload'] || 'totalDownload';-- const whereSearch = search ? 'AND domain LIKE ?' : '';- const baseParams: any[] = [backendId, range.startMinute, range.endMinute];- const searchParams: any[] = search ? [`%${search}%`] : [];-- const countStmt = this.db.prepare(`- SELECT COUNT(*) as total- FROM (- SELECT domain- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND domain != '' ${whereSearch}- GROUP BY domain- )- `);- const { total } = countStmt.get(- ...baseParams,- ...searchParams,- ) as { total: number };-- const dataStmt = this.db.prepare(`- SELECT- domain,- SUM(upload) as totalUpload,- SUM(download) as totalDownload,- SUM(connections) as totalConnections,- MAX(minute) as lastSeen,- GROUP_CONCAT(DISTINCT ip) as ips,- GROUP_CONCAT(DISTINCT rule) as rules,- GROUP_CONCAT(DISTINCT chain) as chains- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND domain != '' ${whereSearch}- GROUP BY domain- ORDER BY ${rangeSortColumn} ${sortOrder}- LIMIT ? OFFSET ?- `);- const rows = dataStmt.all(- ...baseParams,- ...searchParams,- limit,- offset,- ) as Array<{- domain: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- lastSeen: string;- ips: string | null;- rules: string | null;- chains: string | null;- }>;-- const data = rows.map(row => {- const rules = row.rules ? row.rules.split(',').filter(Boolean) : [];- const chains = row.chains ? row.chains.split(',').filter(Boolean) : [];- return {- ...row,- ips: row.ips ? row.ips.split(',').filter(Boolean) : [],- rules,- chains: this.expandShortChainsForRules(backendId, chains, rules),- };- }) as DomainStats[];-- return { data, total };- }-- const whereClause = search- ? 'WHERE backend_id = ? AND domain LIKE ?'- : 'WHERE backend_id = ?';- const params: any[] = search- ? [backendId, `%${search}%`]- : [backendId];-- const countStmt = this.db.prepare(- `SELECT COUNT(*) as total FROM domain_stats ${whereClause}`- );- const { total } = countStmt.get(...params) as { total: number };-- const dataStmt = this.db.prepare(`- SELECT domain, total_upload as totalUpload, total_download as totalDownload,- total_connections as totalConnections, last_seen as lastSeen, ips, rules, chains- FROM domain_stats- ${whereClause}- ORDER BY ${sortColumn} ${sortOrder}- LIMIT ? OFFSET ?- `);- const rows = dataStmt.all(...params, limit, offset) as Array<{- domain: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- lastSeen: string;- ips: string | null;- rules: string | null;- chains: string | null;- }>;-- const data = rows.map(row => {- const rules = row.rules ? row.rules.split(',').filter(Boolean) : [];- const chains = row.chains ? row.chains.split(',').filter(Boolean) : [];- return {- ...row,- ips: row.ips ? row.ips.split(',') : [],- rules,- chains: this.expandShortChainsForRules(backendId, chains, rules),- };- }) as DomainStats[];-- return { data, total };- }-- // Get IP stats with server-side pagination, sorting and search- getIPStatsPaginated(backendId: number, opts: {- offset?: number;- limit?: number;- sortBy?: string;- sortOrder?: string;- search?: string;- start?: string;- end?: string;- } = {}): { data: IPStats[]; total: number } {- const offset = opts.offset ?? 0;- const limit = Math.min(opts.limit ?? 50, 200);- const sortOrder = opts.sortOrder === 'asc' ? 'ASC' : 'DESC';- const search = opts.search?.trim() || '';- const range = this.parseMinuteRange(opts.start, opts.end);-- const sortColumnMap: Record<string, string> = {- ip: 'i.ip',- totalDownload: 'i.total_download',- totalUpload: 'i.total_upload',- totalTraffic: '(i.total_upload + i.total_download)',- totalConnections: 'i.total_connections',- lastSeen: 'i.last_seen',- };- const sortColumn = sortColumnMap[opts.sortBy || 'totalDownload'] || 'i.total_download';-- if (range) {- const rangeSortColumnMap: Record<string, string> = {- ip: 'agg.ip',- totalDownload: 'agg.totalDownload',- totalUpload: 'agg.totalUpload',- totalTraffic: '(agg.totalUpload + agg.totalDownload)',- totalConnections: 'agg.totalConnections',- lastSeen: 'agg.lastSeen',- };- const rangeSortColumn =- rangeSortColumnMap[opts.sortBy || 'totalDownload'] || 'agg.totalDownload';-- const whereSearch = search- ? "AND (ip LIKE ? OR domain LIKE ?)"- : "";- const baseParams: any[] = [backendId, range.startMinute, range.endMinute];- const searchParams: any[] = search- ? [`%${search}%`, `%${search}%`]- : [];-- const countStmt = this.db.prepare(`- SELECT COUNT(*) as total- FROM (- SELECT ip- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND ip != '' ${whereSearch}- GROUP BY ip- )- `);- const { total } = countStmt.get(- ...baseParams,- ...searchParams,- ) as { total: number };-- const dataStmt = this.db.prepare(`- WITH agg AS (- SELECT- ip,- GROUP_CONCAT(DISTINCT CASE WHEN domain != '' THEN domain END) as domains,- GROUP_CONCAT(DISTINCT rule) as rules,- SUM(upload) as totalUpload,- SUM(download) as totalDownload,- SUM(connections) as totalConnections,- MAX(minute) as lastSeen,- GROUP_CONCAT(DISTINCT chain) as chains- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND ip != '' ${whereSearch}- GROUP BY ip- )- SELECT- agg.ip,- agg.domains,- agg.rules,- agg.totalUpload,- agg.totalDownload,- agg.totalConnections,- agg.lastSeen,- COALESCE(i.asn, g.asn) as asn,- CASE- WHEN g.country IS NOT NULL THEN- json_array(- g.country,- COALESCE(g.country_name, g.country),- COALESCE(g.city, ''),- COALESCE(g.as_name, '')- )- WHEN i.geoip IS NOT NULL THEN- json(i.geoip)- ELSE- NULL- END as geoIP,- agg.chains- FROM agg- LEFT JOIN ip_stats i ON i.backend_id = ? AND i.ip = agg.ip- LEFT JOIN geoip_cache g ON g.ip = agg.ip- ORDER BY ${rangeSortColumn} ${sortOrder}- LIMIT ? OFFSET ?- `);- const rows = dataStmt.all(- ...baseParams,- ...searchParams,- backendId,- limit,- offset,- ) as Array<{- ip: string;- domains: string;- rules: string | null;- totalUpload: number;- totalDownload: number;- totalConnections: number;- lastSeen: string;- asn: string | null;- geoIP: string | null;- chains: string | null;- }>;-- const data = rows.map(row => {- const rules = row.rules ? row.rules.split(',').filter(Boolean) : [];- const chains = row.chains ? row.chains.split(',').filter(Boolean) : [];- return {- ip: row.ip,- domains: row.domains ? row.domains.split(',').filter(Boolean) : [],- totalUpload: row.totalUpload,- totalDownload: row.totalDownload,- totalConnections: row.totalConnections,- lastSeen: row.lastSeen,- geoIP: row.geoIP ? JSON.parse(row.geoIP).filter(Boolean) : undefined,- asn: row.asn || undefined,- chains: this.expandShortChainsForRules(backendId, chains, rules),- };- }) as IPStats[];-- return { data, total };- }-- const whereClause = search- ? "WHERE i.backend_id = ? AND i.ip != '' AND (i.ip LIKE ? OR i.domains LIKE ?)"- : "WHERE i.backend_id = ? AND i.ip != ''";- const params: any[] = search- ? [backendId, `%${search}%`, `%${search}%`]- : [backendId];-- const countStmt = this.db.prepare(- `SELECT COUNT(*) as total FROM ip_stats i ${whereClause}`- );- const { total } = countStmt.get(...params) as { total: number };-- const dataStmt = this.db.prepare(`- SELECT- i.ip,- i.domains,- i.total_upload as totalUpload,- i.total_download as totalDownload,- i.total_connections as totalConnections,- i.last_seen as lastSeen,- i.rules,- COALESCE(i.asn, g.asn) as asn,- CASE- WHEN g.country IS NOT NULL THEN- json_array(- g.country,- COALESCE(g.country_name, g.country),- COALESCE(g.city, ''),- COALESCE(g.as_name, '')- )- WHEN i.geoip IS NOT NULL THEN- json(i.geoip)- ELSE- NULL- END as geoIP,- i.chains- FROM ip_stats i- LEFT JOIN geoip_cache g ON i.ip = g.ip- ${whereClause}- ORDER BY ${sortColumn} ${sortOrder}- LIMIT ? OFFSET ?- `);- const rows = dataStmt.all(...params, limit, offset) as Array<{- ip: string;- domains: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- lastSeen: string;- rules: string | null;- asn: string | null;- geoIP: string | null;- chains: string | null;- }>;-- const data = rows.map(row => {- const rules = row.rules ? row.rules.split(',').filter(Boolean) : [];- const chains = row.chains ? row.chains.split(',').filter(Boolean) : [];- return {- ip: row.ip,- domains: row.domains ? row.domains.split(',') : [],- totalUpload: row.totalUpload,- totalDownload: row.totalDownload,- totalConnections: row.totalConnections,- lastSeen: row.lastSeen,- geoIP: row.geoIP ? JSON.parse(row.geoIP).filter(Boolean) : undefined,- asn: row.asn || undefined,- chains: this.expandShortChainsForRules(backendId, chains, rules),- };- }) as IPStats[];-- return { data, total };- }-- // Get proxy stats for a specific backend- getProxyStats(backendId: number, start?: string, end?: string): ProxyStats[] {- const range = this.parseMinuteRange(start, end);- if (range) {- const stmt = this.db.prepare(`- SELECT- chain,- SUM(upload) as totalUpload,- SUM(download) as totalDownload,- SUM(connections) as totalConnections,- MAX(minute) as lastSeen- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ?- GROUP BY chain- ORDER BY (SUM(upload) + SUM(download)) DESC- `);- const rows = stmt.all(- backendId,- range.startMinute,- range.endMinute,- ) as ProxyStats[];- return this.aggregateProxyStatsByFirstHop(rows);- }-- const stmt = this.db.prepare(`- SELECT chain, total_upload as totalUpload, total_download as totalDownload,- total_connections as totalConnections, last_seen as lastSeen- FROM proxy_stats- WHERE backend_id = ?- ORDER BY (total_upload + total_download) DESC- `);- const rows = stmt.all(backendId) as ProxyStats[];- return this.aggregateProxyStatsByFirstHop(rows);- }-- // Get rule stats for a specific backend- getRuleStats(backendId: number, start?: string, end?: string): RuleStats[] {- const range = this.parseMinuteRange(start, end);- if (range) {- const stmt = this.db.prepare(`- SELECT- rule,- MAX(chain) as finalProxy,- SUM(upload) as totalUpload,- SUM(download) as totalDownload,- SUM(connections) as totalConnections,- MAX(minute) as lastSeen- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ?- GROUP BY rule- ORDER BY (SUM(upload) + SUM(download)) DESC- `);- return stmt.all(- backendId,- range.startMinute,- range.endMinute,- ) as RuleStats[];- }-- const stmt = this.db.prepare(`- SELECT rule, final_proxy as finalProxy, total_upload as totalUpload, total_download as totalDownload,- total_connections as totalConnections, last_seen as lastSeen- FROM rule_stats- WHERE backend_id = ?- ORDER BY (total_upload + total_download) DESC- `);- return stmt.all(backendId) as RuleStats[];- }-- // Get rule-proxy mapping for a specific backend- getRuleProxyMap(backendId: number): Array<{ rule: string; proxies: string[] }> {- const stmt = this.db.prepare(`- SELECT rule, proxy FROM rule_proxy_map WHERE backend_id = ? ORDER BY rule, proxy- `);- const rows = stmt.all(backendId) as Array<{ rule: string; proxy: string }>;-- const map = new Map<string, string[]>();- for (const row of rows) {- if (!map.has(row.rule)) {- map.set(row.rule, []);- }- map.get(row.rule)!.push(row.proxy);- }-- return Array.from(map.entries()).map(([rule, proxies]) => ({ rule, proxies }));- }-- // Update ASN info for an IP- updateASNInfo(ip: string, asn: string, org: string) {- const stmt = this.db.prepare(`- INSERT INTO asn_cache (ip, asn, org, queried_at)- VALUES (?, ?, ?, CURRENT_TIMESTAMP)- ON CONFLICT(ip) DO UPDATE SET- asn = ?,- org = ?,- queried_at = CURRENT_TIMESTAMP- `);- stmt.run(ip, asn, org, asn, org);- }-- // Get ASN info for IPs- getASNInfo(ips: string[]): Array<{ ip: string; asn: string; org: string }> {- const stmt = this.db.prepare(`- SELECT ip, asn, org FROM asn_cache WHERE ip IN (${ips.map(() => '?').join(',')})- `);- return stmt.all(...ips) as Array<{ ip: string; asn: string; org: string }>;- }-- // Get ASN info for a single IP- getASNInfoForIP(ip: string): { ip: string; asn: string; org: string } | undefined {- const stmt = this.db.prepare(`SELECT ip, asn, org FROM asn_cache WHERE ip = ?`);- return stmt.get(ip) as { ip: string; asn: string; org: string } | undefined;- }-- // Get GeoIP info for an IP (from cache)- getIPGeolocation(ip: string): {- country: string;- country_name: string;- city: string;- asn: string;- as_name: string;- as_domain: string;- continent: string;- continent_name: string;- } | undefined {- const stmt = this.db.prepare(`- SELECT country, country_name, city, asn, as_name, as_domain, continent, continent_name- FROM geoip_cache- WHERE ip = ?- `);- return stmt.get(ip) as {- country: string;- country_name: string;- city: string;- asn: string;- as_name: string;- as_domain: string;- continent: string;- continent_name: string;- } | undefined;- }-- // Save GeoIP info to cache- saveIPGeolocation(ip: string, geo: {- country: string;- country_name: string;- city: string;- asn: string;- as_name: string;- as_domain: string;- continent: string;- continent_name: string;- }): void {- const stmt = this.db.prepare(`- INSERT INTO geoip_cache (ip, country, country_name, city, asn, as_name, as_domain, continent, continent_name, queried_at)- VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, CURRENT_TIMESTAMP)- ON CONFLICT(ip) DO UPDATE SET- country = ?,- country_name = ?,- city = ?,- asn = ?,- as_name = ?,- as_domain = ?,- continent = ?,- continent_name = ?,- queried_at = CURRENT_TIMESTAMP- `);- stmt.run(- ip, geo.country, geo.country_name, geo.city, geo.asn, geo.as_name, geo.as_domain, geo.continent, geo.continent_name,- geo.country, geo.country_name, geo.city, geo.asn, geo.as_name, geo.as_domain, geo.continent, geo.continent_name- );- }-- // Get summary stats for a specific backend- getSummary(- backendId: number,- start?: string,- end?: string,- ): { totalConnections: number; totalUpload: number; totalDownload: number; uniqueDomains: number; uniqueIPs: number } {- const range = this.parseMinuteRange(start, end);- if (range) {- const trafficStmt = this.db.prepare(`- SELECT- COALESCE(SUM(connections), 0) as connections,- COALESCE(SUM(upload), 0) as upload,- COALESCE(SUM(download), 0) as download- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ?- `);- const { connections, upload, download } = trafficStmt.get(- backendId,- range.startMinute,- range.endMinute,- ) as {- connections: number;- upload: number;- download: number;- };-- const domainsStmt = this.db.prepare(`- SELECT COUNT(DISTINCT domain) as count- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND domain != ''- `);- const uniqueDomains = (domainsStmt.get(- backendId,- range.startMinute,- range.endMinute,- ) as { count: number }).count;-- const ipsStmt = this.db.prepare(`- SELECT COUNT(DISTINCT ip) as count- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND ip != ''- `);- const uniqueIPs = (ipsStmt.get(- backendId,- range.startMinute,- range.endMinute,- ) as { count: number }).count;-- return {- totalConnections: connections,- totalUpload: upload,- totalDownload: download,- uniqueDomains,- uniqueIPs,- };- }-- // Calculate totals from ip_stats to include traffic with unknown domains- const trafficStmt = this.db.prepare(`- SELECT- COALESCE(SUM(total_connections), 0) as connections,- COALESCE(SUM(total_upload), 0) as upload,- COALESCE(SUM(total_download), 0) as download- FROM ip_stats- WHERE backend_id = ?- `);- const { connections, upload, download } = trafficStmt.get(backendId) as {- connections: number;- upload: number;- download: number;- };-- const domainsStmt = this.db.prepare(`- SELECT COUNT(DISTINCT domain) as count FROM domain_stats WHERE backend_id = ?- `);- const uniqueDomains = (domainsStmt.get(backendId) as { count: number }).count;-- const ipsStmt = this.db.prepare(`- SELECT COUNT(DISTINCT ip) as count FROM ip_stats WHERE backend_id = ?- `);- const uniqueIPs = (ipsStmt.get(backendId) as { count: number }).count;-- return {- totalConnections: connections,- totalUpload: upload,- totalDownload: download,- uniqueDomains,- uniqueIPs- };- }-- // Get per-proxy traffic breakdown for a specific domain- getDomainProxyStats(- backendId: number,- domain: string,- start?: string,- end?: string,- sourceIP?: string,- sourceChain?: string,- ): ProxyTrafficStats[] {- const range = this.parseMinuteRange(start, end);- if (range || sourceIP || sourceChain) {- const conditions = ["backend_id = ?", "domain = ?"];- const params: Array<string | number> = [backendId, domain];- if (range) {- conditions.push("minute >= ?", "minute <= ?");- params.push(range.startMinute, range.endMinute);- }- if (sourceIP) {- conditions.push("source_ip = ?");- params.push(sourceIP);- }- if (sourceChain) {- conditions.push("(chain = ? OR chain LIKE ?)");- params.push(sourceChain, `${sourceChain} > %`);- }-- const stmt = this.db.prepare(`- SELECT- chain,- SUM(upload) as totalUpload,- SUM(download) as totalDownload,- SUM(connections) as totalConnections- FROM minute_dim_stats- WHERE ${conditions.join(" AND ")}- GROUP BY chain- ORDER BY (SUM(upload) + SUM(download)) DESC- `);- return stmt.all(...params) as ProxyTrafficStats[];- }-- const stmt = this.db.prepare(`- SELECT chain,- total_upload as totalUpload,- total_download as totalDownload,- total_connections as totalConnections- FROM domain_proxy_stats- WHERE backend_id = ? AND domain = ?- ORDER BY (total_upload + total_download) DESC- `);- return stmt.all(backendId, domain) as ProxyTrafficStats[];- }-- // Get per-proxy traffic breakdown for a specific IP- getIPProxyStats(- backendId: number,- ip: string,- start?: string,- end?: string,- sourceIP?: string,- sourceChain?: string,- ): ProxyTrafficStats[] {- const range = this.parseMinuteRange(start, end);- if (range || sourceIP || sourceChain) {- const conditions = ["backend_id = ?", "ip = ?"];- const params: Array<string | number> = [backendId, ip];- if (range) {- conditions.push("minute >= ?", "minute <= ?");- params.push(range.startMinute, range.endMinute);- }- if (sourceIP) {- conditions.push("source_ip = ?");- params.push(sourceIP);- }- if (sourceChain) {- conditions.push("(chain = ? OR chain LIKE ?)");- params.push(sourceChain, `${sourceChain} > %`);- }-- const stmt = this.db.prepare(`- SELECT- chain,- SUM(upload) as totalUpload,- SUM(download) as totalDownload,- SUM(connections) as totalConnections- FROM minute_dim_stats- WHERE ${conditions.join(" AND ")}- GROUP BY chain- ORDER BY (SUM(upload) + SUM(download)) DESC- `);- return stmt.all(...params) as ProxyTrafficStats[];- }-- const stmt = this.db.prepare(`- SELECT chain,- total_upload as totalUpload,- total_download as totalDownload,- total_connections as totalConnections- FROM ip_proxy_stats- WHERE backend_id = ? AND ip = ?- ORDER BY (total_upload + total_download) DESC- `);- return stmt.all(backendId, ip) as ProxyTrafficStats[];- }-- // Get domains for a specific proxy/node- getProxyDomains(- backendId: number,- chain: string,- limit = 50,- start?: string,- end?: string,- ): DomainStats[] {- const range = this.parseMinuteRange(start, end);- if (range) {- const stmt = this.db.prepare(`- SELECT- domain,- SUM(upload) as totalUpload,- SUM(download) as totalDownload,- SUM(connections) as totalConnections,- MAX(minute) as lastSeen,- GROUP_CONCAT(DISTINCT ip) as ips- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND (chain = ? OR chain LIKE ?) AND domain != ''- GROUP BY domain- ORDER BY (SUM(upload) + SUM(download)) DESC- LIMIT ?- `);- const rows = stmt.all(- backendId,- range.startMinute,- range.endMinute,- chain,- `${chain} > %`,- limit,- ) as Array<{- domain: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- lastSeen: string;- ips: string | null;- }>;-- return rows.map(row => ({- ...row,- ips: row.ips ? row.ips.split(',').filter(Boolean) : [],- rules: [],- chains: [chain],- })) as DomainStats[];- }-- const stmt = this.db.prepare(`- SELECT- dps.domain,- dps.total_upload as totalUpload,- dps.total_download as totalDownload,- dps.total_connections as totalConnections,- dps.last_seen as lastSeen,- ds.ips- FROM domain_proxy_stats dps- LEFT JOIN domain_stats ds ON dps.backend_id = ds.backend_id AND dps.domain = ds.domain- WHERE dps.backend_id = ? AND (dps.chain = ? OR dps.chain LIKE ?)- ORDER BY (dps.total_upload + dps.total_download) DESC- LIMIT ?- `);- const rows = stmt.all(backendId, chain, `${chain} > %`, limit) as Array<{- domain: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- lastSeen: string;- ips: string | null;- }>;-- return rows.map(row => ({- ...row,- ips: row.ips ? row.ips.split(',').filter(Boolean) : [],- rules: [],- chains: [chain],- })) as DomainStats[];- }-- // Get IPs for a specific proxy/node- getProxyIPs(- backendId: number,- chain: string,- limit = 50,- start?: string,- end?: string,- ): IPStats[] {- const range = this.parseMinuteRange(start, end);- if (range) {- const stmt = this.db.prepare(`- SELECT- m.ip,- SUM(m.upload) as totalUpload,- SUM(m.download) as totalDownload,- SUM(m.connections) as totalConnections,- MAX(m.minute) as lastSeen,- GROUP_CONCAT(DISTINCT CASE WHEN m.domain != '' THEN m.domain END) as domains,- COALESCE(i.asn, g.asn) as asn,- CASE- WHEN g.country IS NOT NULL THEN- json_array(- g.country,- COALESCE(g.country_name, g.country),- COALESCE(g.city, ''),- COALESCE(g.as_name, '')- )- WHEN i.geoip IS NOT NULL THEN- json(i.geoip)- ELSE- NULL- END as geoIP- FROM minute_dim_stats m- LEFT JOIN ip_stats i ON m.backend_id = i.backend_id AND m.ip = i.ip- LEFT JOIN geoip_cache g ON m.ip = g.ip- WHERE m.backend_id = ? AND m.minute >= ? AND m.minute <= ? AND (m.chain = ? OR m.chain LIKE ?) AND m.ip != ''- GROUP BY m.ip- ORDER BY (SUM(m.upload) + SUM(m.download)) DESC- LIMIT ?- `);- const rows = stmt.all(- backendId,- range.startMinute,- range.endMinute,- chain,- `${chain} > %`,- limit,- ) as Array<{- ip: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- lastSeen: string;- domains: string | null;- asn: string | null;- geoIP: string | null;- }>;-- return rows.map(row => ({- ...row,- domains: row.domains ? row.domains.split(',').filter(Boolean) : [],- chains: [chain],- asn: row.asn || undefined,- geoIP: row.geoIP ? JSON.parse(row.geoIP).filter(Boolean) : undefined,- })) as IPStats[];- }-- const stmt = this.db.prepare(`- SELECT- ips.ip,- ips.total_upload as totalUpload,- ips.total_download as totalDownload,- ips.total_connections as totalConnections,- ips.last_seen as lastSeen,- ips.domains- FROM ip_proxy_stats ips- WHERE ips.backend_id = ? AND (ips.chain = ? OR ips.chain LIKE ?) AND ips.ip != ''- ORDER BY (ips.total_upload + ips.total_download) DESC- LIMIT ?- `);- const rows = stmt.all(backendId, chain, `${chain} > %`, limit) as Array<{- ip: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- lastSeen: string;- domains: string | null;- }>;-- // Get geoIP info for each IP- const ipList = rows.map(r => r.ip).filter(ip => ip && ip.trim() !== '');- if (ipList.length === 0) {- return [];- }-- const placeholders = ipList.map(() => '?').join(',');- const geoStmt = this.db.prepare(`- SELECT- ip,- CASE- WHEN country IS NOT NULL THEN- json_array(- country,- COALESCE(country_name, country),- COALESCE(city, ''),- COALESCE(as_name, '')- )- ELSE- NULL- END as geoIP- FROM geoip_cache- WHERE ip IN (${placeholders})- `);- const geoRows = geoStmt.all(...ipList) as Array<{- ip: string;- geoIP: string | null;- }>;-- const geoMap = new Map(geoRows.map(r => [r.ip, r.geoIP]));-- return rows.map(row => ({- ...row,- domains: row.domains ? row.domains.split(',').filter(Boolean) : [],- chains: [chain],- geoIP: geoMap.get(row.ip) ? JSON.parse(geoMap.get(row.ip)!).filter(Boolean) : undefined,- })) as IPStats[];- }-- // Get domains for a specific rule- getRuleDomains(- backendId: number,- rule: string,- limit = 50,- start?: string,- end?: string,- ): DomainStats[] {- const range = this.parseMinuteRange(start, end);- if (range) {- const stmt = this.db.prepare(`- SELECT- domain,- SUM(upload) as totalUpload,- SUM(download) as totalDownload,- SUM(connections) as totalConnections,- MAX(minute) as lastSeen,- GROUP_CONCAT(DISTINCT ip) as ips,- GROUP_CONCAT(DISTINCT chain) as chains- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND rule = ? AND domain != ''- GROUP BY domain- ORDER BY (SUM(upload) + SUM(download)) DESC- LIMIT ?- `);- const rows = stmt.all(- backendId,- range.startMinute,- range.endMinute,- rule,- limit,- ) as Array<{- domain: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- lastSeen: string;- ips: string | null;- chains: string | null;- }>;-- return rows.map(row => {- const chains = row.chains ? row.chains.split(',').filter(Boolean) : [];- return {- domain: row.domain,- totalUpload: row.totalUpload,- totalDownload: row.totalDownload,- totalConnections: row.totalConnections,- lastSeen: row.lastSeen,- ips: row.ips ? row.ips.split(',').filter(Boolean) : [],- chains: this.expandShortChainsForRules(backendId, chains, [rule]),- rules: [rule],- };- }) as DomainStats[];- }-- // Query rule_domain_traffic for accurate all-time per-rule domain traffic- const stmt = this.db.prepare(`- SELECT- rdt.domain,- rdt.total_upload as totalUpload,- rdt.total_download as totalDownload,- rdt.total_connections as totalConnections,- rdt.last_seen as lastSeen,- ds.ips,- ds.chains- FROM rule_domain_traffic rdt- LEFT JOIN domain_stats ds ON rdt.backend_id = ds.backend_id AND rdt.domain = ds.domain- WHERE rdt.backend_id = ? AND rdt.rule = ?- ORDER BY (rdt.total_upload + rdt.total_download) DESC- LIMIT ?- `);-- const rows = stmt.all(backendId, rule, limit) as Array<{- domain: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- lastSeen: string;- ips: string | null;- chains: string | null;- }>;-- return rows.map(row => ({- domain: row.domain,- totalUpload: row.totalUpload,- totalDownload: row.totalDownload,- totalConnections: row.totalConnections,- lastSeen: row.lastSeen,- ips: row.ips ? row.ips.split(',').filter(Boolean) : [],- chains: this.expandShortChainsForRules(- backendId,- row.chains ? row.chains.split(',').filter(Boolean) : [],- [rule],- ),- rules: [rule],- })) as DomainStats[];- }-- // Get IPs for a specific rule- getRuleIPs(- backendId: number,- rule: string,- limit = 50,- start?: string,- end?: string,- ): IPStats[] {- const range = this.parseMinuteRange(start, end);- if (range) {- const stmt = this.db.prepare(`- SELECT- m.ip,- SUM(m.upload) as totalUpload,- SUM(m.download) as totalDownload,- SUM(m.connections) as totalConnections,- MAX(m.minute) as lastSeen,- GROUP_CONCAT(DISTINCT CASE WHEN m.domain != '' THEN m.domain END) as domains,- GROUP_CONCAT(DISTINCT m.chain) as chains,- COALESCE(i.asn, g.asn) as asn,- CASE- WHEN g.country IS NOT NULL THEN- json_array(- g.country,- COALESCE(g.country_name, g.country),- COALESCE(g.city, ''),- COALESCE(g.as_name, '')- )- WHEN i.geoip IS NOT NULL THEN- json(i.geoip)- ELSE- NULL- END as geoIPData- FROM minute_dim_stats m- LEFT JOIN ip_stats i ON m.backend_id = i.backend_id AND m.ip = i.ip- LEFT JOIN geoip_cache g ON m.ip = g.ip- WHERE m.backend_id = ? AND m.minute >= ? AND m.minute <= ? AND m.rule = ? AND m.ip != ''- GROUP BY m.ip- ORDER BY (SUM(m.upload) + SUM(m.download)) DESC- LIMIT ?- `);-- const rows = stmt.all(- backendId,- range.startMinute,- range.endMinute,- rule,- limit,- ) as Array<{- ip: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- lastSeen: string;- domains: string | null;- chains: string | null;- asn: string | null;- geoIPData: string | null;- }>;-- return rows.map(row => {- const chains = row.chains ? row.chains.split(',').filter(Boolean) : [];- return {- ip: row.ip,- totalUpload: row.totalUpload,- totalDownload: row.totalDownload,- totalConnections: row.totalConnections,- lastSeen: row.lastSeen,- domains: row.domains ? row.domains.split(',').filter(Boolean) : [],- chains: this.expandShortChainsForRules(backendId, chains, [rule]),- asn: row.asn || undefined,- geoIP: row.geoIPData ? JSON.parse(row.geoIPData).filter(Boolean) : undefined,- };- }) as IPStats[];- }-- // Query rule_ip_traffic for accurate all-time per-rule IP traffic- const stmt = this.db.prepare(`- SELECT- rit.ip,- rit.total_upload as totalUpload,- rit.total_download as totalDownload,- rit.total_connections as totalConnections,- rit.last_seen as lastSeen,- i.domains,- i.chains,- COALESCE(i.asn, g.asn) as asn,- CASE- WHEN g.country IS NOT NULL THEN- json_array(- g.country,- COALESCE(g.country_name, g.country),- COALESCE(g.city, ''),- COALESCE(g.as_name, '')- )- ELSE- NULL- END as geoIPData- FROM rule_ip_traffic rit- LEFT JOIN ip_stats i ON rit.backend_id = i.backend_id AND rit.ip = i.ip- LEFT JOIN geoip_cache g ON rit.ip = g.ip- WHERE rit.backend_id = ? AND rit.rule = ?- ORDER BY (rit.total_upload + rit.total_download) DESC- LIMIT ?- `);-- const rows = stmt.all(backendId, rule, limit) as Array<{- ip: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- lastSeen: string;- domains: string | null;- chains: string | null;- asn: string | null;- geoIPData: string | null;- }>;-- return rows.map(row => ({- ip: row.ip,- totalUpload: row.totalUpload,- totalDownload: row.totalDownload,- totalConnections: row.totalConnections,- lastSeen: row.lastSeen,- domains: row.domains ? row.domains.split(',').filter(Boolean) : [],- chains: this.expandShortChainsForRules(- backendId,- row.chains ? row.chains.split(',').filter(Boolean) : [],- [rule],- ),- asn: row.asn || undefined,- geoIP: row.geoIPData ? JSON.parse(row.geoIPData).filter(Boolean) : undefined,- })) as IPStats[];- }-- // Get per-proxy traffic breakdown for a specific domain under a specific rule- getRuleDomainProxyStats(- backendId: number,- rule: string,- domain: string,- start?: string,- end?: string,- ): ProxyTrafficStats[] {- const range = this.parseMinuteRange(start, end);- if (range) {- const stmt = this.db.prepare(`- SELECT- chain,- SUM(upload) as totalUpload,- SUM(download) as totalDownload,- SUM(connections) as totalConnections- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND rule = ? AND domain = ?- GROUP BY chain- ORDER BY (SUM(upload) + SUM(download)) DESC- `);- return stmt.all(- backendId,- range.startMinute,- range.endMinute,- rule,- domain,- ) as ProxyTrafficStats[];- }-- return [];- }-- // Get IP details for a specific domain under a specific rule- getRuleDomainIPDetails(- backendId: number,- rule: string,- domain: string,- start?: string,- end?: string,- limit = 100,- ): IPStats[] {- const range = this.parseMinuteRange(start, end);- const conditions = ["m.backend_id = ?", "m.rule = ?", "m.domain = ?", "m.ip != ''"];- const params: Array<string | number> = [backendId, rule, domain];-- if (range) {- conditions.push("m.minute >= ?", "m.minute <= ?");- params.push(range.startMinute, range.endMinute);- }-- const stmt = this.db.prepare(`- SELECT- m.ip,- GROUP_CONCAT(DISTINCT CASE WHEN m.domain != '' THEN m.domain END) as domains,- SUM(m.upload) as totalUpload,- SUM(m.download) as totalDownload,- SUM(m.connections) as totalConnections,- MAX(m.minute) as lastSeen,- COALESCE(i.asn, g.asn) as asn,- CASE- WHEN g.country IS NOT NULL THEN- json_array(- g.country,- COALESCE(g.country_name, g.country),- COALESCE(g.city, ''),- COALESCE(g.as_name, '')- )- WHEN i.geoip IS NOT NULL THEN- json(i.geoip)- ELSE- NULL- END as geoIP,- GROUP_CONCAT(DISTINCT m.chain) as chains- FROM minute_dim_stats m- LEFT JOIN ip_stats i ON m.backend_id = i.backend_id AND m.ip = i.ip- LEFT JOIN geoip_cache g ON m.ip = g.ip- WHERE ${conditions.join(" AND ")}- GROUP BY m.ip- ORDER BY (SUM(m.upload) + SUM(m.download)) DESC- LIMIT ?- `);-- const rows = stmt.all(...params, limit) as Array<{- ip: string;- domains: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- lastSeen: string;- asn: string | null;- geoIP: string | null;- chains: string | null;- }>;-- return rows.map(row => {- const chains = row.chains ? row.chains.split(',').filter(Boolean) : [];- return {- ...row,- domains: row.domains ? row.domains.split(',').filter(Boolean) : [],- geoIP: row.geoIP ? JSON.parse(row.geoIP).filter(Boolean) : undefined,- asn: row.asn || undefined,- chains: this.expandShortChainsForRules(backendId, chains, [rule]),- };- }) as IPStats[];- }-- // Get per-proxy traffic breakdown for a specific IP under a specific rule- getRuleIPProxyStats(- backendId: number,- rule: string,- ip: string,- start?: string,- end?: string,- ): ProxyTrafficStats[] {- const range = this.parseMinuteRange(start, end);- if (range) {- const stmt = this.db.prepare(`- SELECT- chain,- SUM(upload) as totalUpload,- SUM(download) as totalDownload,- SUM(connections) as totalConnections- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND rule = ? AND ip = ?- GROUP BY chain- ORDER BY (SUM(upload) + SUM(download)) DESC- `);- return stmt.all(- backendId,- range.startMinute,- range.endMinute,- rule,- ip,- ) as ProxyTrafficStats[];- }-- return [];- }-- // Get domain details for a specific IP under a specific rule- getRuleIPDomainDetails(- backendId: number,- rule: string,- ip: string,- start?: string,- end?: string,- limit = 100,- ): DomainStats[] {- const range = this.parseMinuteRange(start, end);- const conditions = ["backend_id = ?", "rule = ?", "ip = ?", "domain != ''"];- const params: Array<string | number> = [backendId, rule, ip];-- if (range) {- conditions.push("minute >= ?", "minute <= ?");- params.push(range.startMinute, range.endMinute);- }-- const stmt = this.db.prepare(`- SELECT- domain,- GROUP_CONCAT(DISTINCT ip) as ips,- SUM(upload) as totalUpload,- SUM(download) as totalDownload,- SUM(connections) as totalConnections,- MAX(minute) as lastSeen,- GROUP_CONCAT(DISTINCT CASE WHEN rule != '' THEN rule END) as rules,- GROUP_CONCAT(DISTINCT chain) as chains- FROM minute_dim_stats- WHERE ${conditions.join(" AND ")}- GROUP BY domain- ORDER BY (SUM(upload) + SUM(download)) DESC- LIMIT ?- `);-- const rows = stmt.all(...params, limit) as Array<{- domain: string;- ips: string | null;- totalUpload: number;- totalDownload: number;- totalConnections: number;- lastSeen: string;- rules: string | null;- chains: string | null;- }>;-- return rows.map(row => {- const rules = row.rules ? row.rules.split(',').filter(Boolean) : [];- const chains = row.chains ? row.chains.split(',').filter(Boolean) : [];- return {- domain: row.domain,- ips: row.ips ? row.ips.split(',').filter(Boolean) : [],- totalUpload: row.totalUpload,- totalDownload: row.totalDownload,- totalConnections: row.totalConnections,- lastSeen: row.lastSeen,- rules,- chains: this.expandShortChainsForRules(backendId, chains, rules),- };- }) as DomainStats[];- }-- // Get rule chain flow for visualization- getRuleChainFlow(- backendId: number,- rule: string,- start?: string,- end?: string,- ): { nodes: Array<{ name: string; totalUpload: number; totalDownload: number; totalConnections: number }>; links: Array<{ source: number; target: number }> } {- const range = this.parseMinuteRange(start, end);- let rows: Array<{- chain: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- }>;-- if (range) {- const stmt = this.db.prepare(`- SELECT- chain,- SUM(upload) as totalUpload,- SUM(download) as totalDownload,- SUM(connections) as totalConnections- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND rule = ? AND chain != ''- GROUP BY chain- `);- rows = stmt.all(- backendId,- range.startMinute,- range.endMinute,- rule,- ) as Array<{- chain: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- }>;-- const baselineStmt = this.db.prepare(`- SELECT- rule,- chain,- total_upload as totalUpload,- total_download as totalDownload,- total_connections as totalConnections- FROM rule_chain_traffic- WHERE backend_id = ? AND rule = ?- `);- const baselineRows = baselineStmt.all(backendId, rule) as Array<{- rule: string;- chain: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- }>;- const remapped = this.remapRangeRowsToFullChains(- rows.map(r => ({- rule,- chain: r.chain,- totalUpload: r.totalUpload,- totalDownload: r.totalDownload,- totalConnections: r.totalConnections,- })),- baselineRows,- );- rows = remapped.map(r => ({- chain: r.chain,- totalUpload: r.totalUpload,- totalDownload: r.totalDownload,- totalConnections: r.totalConnections,- }));- } else {- // No time range: use cumulative table for fast query.- const stmt = this.db.prepare(`- SELECT chain, total_upload as totalUpload, total_download as totalDownload, total_connections as totalConnections- FROM rule_chain_traffic- WHERE backend_id = ? AND rule = ?- `);- rows = stmt.all(backendId, rule) as Array<{- chain: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- }>;- }-- const nodeMap = new Map<string, { name: string; totalUpload: number; totalDownload: number; totalConnections: number }>();- const linkSet = new Set<string>();-- for (const row of rows) {- const flowPath = this.buildRuleFlowPath(rule, row.chain);- if (flowPath.length < 2) continue;-- for (let i = 0; i < flowPath.length; i++) {- const nodeName = flowPath[i];- if (!nodeMap.has(nodeName)) {- nodeMap.set(nodeName, { name: nodeName, totalUpload: 0, totalDownload: 0, totalConnections: 0 });- }- const node = nodeMap.get(nodeName)!;- node.totalUpload += row.totalUpload;- node.totalDownload += row.totalDownload;- node.totalConnections += row.totalConnections;- }-- for (let i = 0; i < flowPath.length - 1; i++) {- linkSet.add(`${flowPath[i]}|${flowPath[i + 1]}`);- }- }-- const nodes = Array.from(nodeMap.values());- const nodeIndexMap = new Map(nodes.map((n, i) => [n.name, i]));-- const links = Array.from(linkSet).map(linkStr => {- const [sourceName, targetName] = linkStr.split('|');- return { source: nodeIndexMap.get(sourceName)!, target: nodeIndexMap.get(targetName)! };- });-- return { nodes, links };- }-- // Get all rule chain flows merged into a unified DAG- getAllRuleChainFlows(- backendId: number,- start?: string,- end?: string,- ): {- nodes: Array<{ name: string; layer: number; nodeType: 'rule' | 'group' | 'proxy'; totalUpload: number; totalDownload: number; totalConnections: number; rules: string[] }>;- links: Array<{ source: number; target: number; rules: string[] }>;- rulePaths: Record<string, { nodeIndices: number[]; linkIndices: number[] }>;- maxLayer: number;- } {- const range = this.parseMinuteRange(start, end);- let rows: Array<{- rule: string;- chain: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- }>;-- if (range) {- const stmt = this.db.prepare(`- SELECT- rule,- chain,- SUM(upload) as totalUpload,- SUM(download) as totalDownload,- SUM(connections) as totalConnections- FROM minute_dim_stats- WHERE backend_id = ? AND minute >= ? AND minute <= ? AND rule != '' AND chain != ''- GROUP BY rule, chain- ORDER BY rule, chain- `);- rows = stmt.all(- backendId,- range.startMinute,- range.endMinute,- ) as Array<{- rule: string;- chain: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- }>;-- const baselineStmt = this.db.prepare(`- SELECT- rule,- chain,- total_upload as totalUpload,- total_download as totalDownload,- total_connections as totalConnections- FROM rule_chain_traffic- WHERE backend_id = ?- `);- const baselineRows = baselineStmt.all(backendId) as Array<{- rule: string;- chain: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- }>;- rows = this.remapRangeRowsToFullChains(rows, baselineRows);- } else {- // No time range: use cumulative table for fast query.- const stmt = this.db.prepare(`- SELECT rule, chain, total_upload as totalUpload, total_download as totalDownload, total_connections as totalConnections- FROM rule_chain_traffic- WHERE backend_id = ?- ORDER BY rule, chain- `);- rows = stmt.all(backendId) as Array<{- rule: string;- chain: string;- totalUpload: number;- totalDownload: number;- totalConnections: number;- }>;- }-- // nodeMap: name -> { stats + rules set + max layer }- const nodeMap = new Map<string, {- totalUpload: number; totalDownload: number; totalConnections: number;- rules: Set<string>; layer: number;- }>();- // linkMap: "source|target" -> rules set- const linkMap = new Map<string, Set<string>>();- // rulePaths: rule -> Set of node names, Set of link keys- const rulePathNodes = new Map<string, Set<string>>();- const rulePathLinks = new Map<string, Set<string>>();-- for (const row of rows) {- const rule = row.rule;- if (!rulePathNodes.has(rule)) {- rulePathNodes.set(rule, new Set());- rulePathLinks.set(rule, new Set());- }-- const flowPath = this.buildRuleFlowPath(rule, row.chain);- if (flowPath.length < 2) continue;-- for (let i = 0; i < flowPath.length; i++) {- const nodeName = flowPath[i];- if (!nodeMap.has(nodeName)) {- nodeMap.set(nodeName, {- totalUpload: 0, totalDownload: 0, totalConnections: 0,- rules: new Set(), layer: i,- });- }- const node = nodeMap.get(nodeName)!;- node.totalUpload += row.totalUpload;- node.totalDownload += row.totalDownload;- node.totalConnections += row.totalConnections;- node.rules.add(rule);- node.layer = Math.max(node.layer, i);-- rulePathNodes.get(rule)!.add(nodeName);- }-- for (let i = 0; i < flowPath.length - 1; i++) {- const linkKey = `${flowPath[i]}|${flowPath[i + 1]}`;- if (!linkMap.has(linkKey)) {- linkMap.set(linkKey, new Set());- }- linkMap.get(linkKey)!.add(rule);- rulePathLinks.get(rule)!.add(linkKey);- }- }-- // Determine node types and fix layer assignments- const nodeEntries = Array.from(nodeMap.entries());- const nodeTypeMap = new Map<string, 'rule' | 'group' | 'proxy'>();- let computedMaxLayer = 0;-- for (const [name, data] of nodeEntries) {- const hasOutgoing = Array.from(linkMap.keys()).some(k => k.startsWith(name + '|'));- const hasIncoming = Array.from(linkMap.keys()).some(k => k.endsWith('|' + name));-- if (!hasIncoming) {- nodeTypeMap.set(name, 'rule');- data.layer = 0;- } else if (!hasOutgoing) {- nodeTypeMap.set(name, 'proxy');- } else {- nodeTypeMap.set(name, 'group');- }- computedMaxLayer = Math.max(computedMaxLayer, data.layer);- }-- // Force all proxy nodes to the rightmost column- for (const [name, data] of nodeEntries) {- if (nodeTypeMap.get(name) === 'proxy') {- data.layer = computedMaxLayer;- }- }-- // Stable ordering is important to avoid unnecessary ReactFlow relayout/flicker.- const nodeTypeOrder = (type: 'rule' | 'group' | 'proxy'): number => {- if (type === 'rule') return 0;- if (type === 'group') return 1;- return 2;- };- const sortedNodeEntries = [...nodeEntries].sort(([nameA, dataA], [nameB, dataB]) => {- if (dataA.layer !== dataB.layer) return dataA.layer - dataB.layer;- const typeDiff = nodeTypeOrder(nodeTypeMap.get(nameA)!) - nodeTypeOrder(nodeTypeMap.get(nameB)!);- if (typeDiff !== 0) return typeDiff;- return nameA.localeCompare(nameB);- });-- // Convert to arrays- const nodes = sortedNodeEntries.map(([name, data]) => ({- name,- layer: data.layer,- nodeType: nodeTypeMap.get(name)!,- totalUpload: data.totalUpload,- totalDownload: data.totalDownload,- totalConnections: data.totalConnections,- rules: Array.from(data.rules),- }));-- const nodeIndexMap = new Map(nodes.map((n, i) => [n.name, i]));-- const links = Array.from(linkMap.entries())- .sort(([keyA], [keyB]) => keyA.localeCompare(keyB))- .map(([key, rules]) => {- const [sourceName, targetName] = key.split('|');- return {- source: nodeIndexMap.get(sourceName)!,- target: nodeIndexMap.get(targetName)!,- rules: Array.from(rules),- };- });-- // Build rulePaths with index references- const rulePaths: Record<string, { nodeIndices: number[]; linkIndices: number[] }> = {};- for (const [rule, nodeNames] of rulePathNodes) {- const nodeIndices = Array.from(nodeNames).map(n => nodeIndexMap.get(n)!).filter(i => i !== undefined);- const linkIndices: number[] = [];- const linkKeys = rulePathLinks.get(rule)!;- links.forEach((link, idx) => {- const sourceName = nodes[link.source].name;- const targetName = nodes[link.target].name;- if (linkKeys.has(`${sourceName}|${targetName}`)) {- linkIndices.push(idx);- }- });- rulePaths[rule] = { nodeIndices, linkIndices };- }-- const maxLayer = nodes.reduce((max, n) => Math.max(max, n.layer), 0);-- return { nodes, links, rulePaths, maxLayer };- }-- // Get recent connections for a specific backend- // Returns empty array - connection_logs is no longer written to (replaced by aggregation tables)- getRecentConnections(backendId: number, limit = 100): Connection[] {- return [];- }-- // Clean old data for a specific backend (or all backends if backendId is null)- // days=0 means clear all data (no time limit)- cleanupOldData(backendId: number | null, days: number): { deletedConnections: number; deletedLogs: number; deletedDomains: number; deletedIPs: number; deletedProxies: number; deletedRules: number } {- let deletedConnections = 0;- let deletedLogs = 0;- let deletedDomains = 0;- let deletedIPs = 0;- let deletedProxies = 0;- let deletedRules = 0;-- if (backendId !== null) {- // Clean specific backend- if (days === 0) {- // Clear all data for this backend- const minuteResult = this.db.prepare(`DELETE FROM minute_stats WHERE backend_id = ?`).run(backendId);- deletedConnections = minuteResult.changes;- deletedLogs = minuteResult.changes;- this.db.prepare(`DELETE FROM minute_dim_stats WHERE backend_id = ?`).run(backendId);- this.db.prepare(`DELETE FROM minute_country_stats WHERE backend_id = ?`).run(backendId);- // Also clean old connection_logs if any remain from before migration- this.db.prepare(`DELETE FROM connection_logs WHERE backend_id = ?`).run(backendId);-- // Clear stats tables- const domainsStmt = this.db.prepare(`DELETE FROM domain_stats WHERE backend_id = ?`);- deletedDomains = domainsStmt.run(backendId).changes;-- const ipsStmt = this.db.prepare(`DELETE FROM ip_stats WHERE backend_id = ?`);- deletedIPs = ipsStmt.run(backendId).changes;-- const proxiesStmt = this.db.prepare(`DELETE FROM proxy_stats WHERE backend_id = ?`);- deletedProxies = proxiesStmt.run(backendId).changes;-- const rulesStmt = this.db.prepare(`DELETE FROM rule_stats WHERE backend_id = ?`);- deletedRules = rulesStmt.run(backendId).changes;-- // Clear country stats (Regions)- this.db.prepare(`DELETE FROM country_stats WHERE backend_id = ?`).run(backendId);-- // Clear rule to proxy mapping- this.db.prepare(`DELETE FROM rule_proxy_map WHERE backend_id = ?`).run(backendId);-- // Clear rule cross-reference tables- this.db.prepare(`DELETE FROM rule_chain_traffic WHERE backend_id = ?`).run(backendId);- this.db.prepare(`DELETE FROM rule_domain_traffic WHERE backend_id = ?`).run(backendId);- this.db.prepare(`DELETE FROM rule_ip_traffic WHERE backend_id = ?`).run(backendId);-- // Clear new aggregation tables- this.db.prepare(`DELETE FROM domain_proxy_stats WHERE backend_id = ?`).run(backendId);- this.db.prepare(`DELETE FROM ip_proxy_stats WHERE backend_id = ?`).run(backendId);-- // Clear device stats- this.db.prepare(`DELETE FROM device_stats WHERE backend_id = ?`).run(backendId);- this.db.prepare(`DELETE FROM device_domain_stats WHERE backend_id = ?`).run(backendId);- this.db.prepare(`DELETE FROM device_ip_stats WHERE backend_id = ?`).run(backendId);-- // Clear hourly stats (used for today traffic)- this.db.prepare(`DELETE FROM hourly_stats WHERE backend_id = ?`).run(backendId);- } else {- const cutoff = new Date(Date.now() - days * 24 * 60 * 60 * 1000);- const minuteCutoff = cutoff.toISOString().slice(0, 16) + ':00';- const minuteResult = this.db.prepare(`DELETE FROM minute_stats WHERE backend_id = ? AND minute < ?`).run(backendId, minuteCutoff);- deletedConnections = minuteResult.changes;- deletedLogs = minuteResult.changes;- this.db.prepare(`DELETE FROM minute_dim_stats WHERE backend_id = ? AND minute < ?`).run(backendId, minuteCutoff);- this.db.prepare(`DELETE FROM minute_country_stats WHERE backend_id = ? AND minute < ?`).run(backendId, minuteCutoff);- // Also clean old connection_logs if any remain- this.db.prepare(`DELETE FROM connection_logs WHERE backend_id = ? AND timestamp < ?`).run(backendId, cutoff.toISOString());- // domain_proxy_stats and ip_proxy_stats are permanent aggregation tables (no time-based cleanup)- }- } else {- // Clean all backends- if (days === 0) {- // Clear all data- const minuteResult = this.db.prepare(`DELETE FROM minute_stats`).run();- deletedConnections = minuteResult.changes;- deletedLogs = minuteResult.changes;- this.db.prepare(`DELETE FROM minute_dim_stats`).run();- this.db.prepare(`DELETE FROM minute_country_stats`).run();- // Also clean old connection_logs if any remain- this.db.prepare(`DELETE FROM connection_logs`).run();-- // Clear all stats tables- const domainsStmt = this.db.prepare(`DELETE FROM domain_stats`);- deletedDomains = domainsStmt.run().changes;-- const ipsStmt = this.db.prepare(`DELETE FROM ip_stats`);- deletedIPs = ipsStmt.run().changes;-- const proxiesStmt = this.db.prepare(`DELETE FROM proxy_stats`);- deletedProxies = proxiesStmt.run().changes;-- const rulesStmt = this.db.prepare(`DELETE FROM rule_stats`);- deletedRules = rulesStmt.run().changes;-- // Clear country stats (Regions)- this.db.prepare(`DELETE FROM country_stats`).run();-- // Clear rule to proxy mapping- this.db.prepare(`DELETE FROM rule_proxy_map`).run();-- // Clear rule cross-reference tables- this.db.prepare(`DELETE FROM rule_chain_traffic`).run();- this.db.prepare(`DELETE FROM rule_domain_traffic`).run();- this.db.prepare(`DELETE FROM rule_ip_traffic`).run();-- // Clear new aggregation tables- this.db.prepare(`DELETE FROM domain_proxy_stats`).run();- this.db.prepare(`DELETE FROM ip_proxy_stats`).run();-- // Clear device stats- this.db.prepare(`DELETE FROM device_stats`).run();- this.db.prepare(`DELETE FROM device_domain_stats`).run();- this.db.prepare(`DELETE FROM device_ip_stats`).run();-- // Clear hourly stats (used for today traffic)- this.db.prepare(`DELETE FROM hourly_stats`).run();- } else {- const cutoff = new Date(Date.now() - days * 24 * 60 * 60 * 1000);- const minuteCutoff = cutoff.toISOString().slice(0, 16) + ':00';- const minuteResult = this.db.prepare(`DELETE FROM minute_stats WHERE minute < ?`).run(minuteCutoff);- deletedConnections = minuteResult.changes;- deletedLogs = minuteResult.changes;- this.db.prepare(`DELETE FROM minute_dim_stats WHERE minute < ?`).run(minuteCutoff);- this.db.prepare(`DELETE FROM minute_country_stats WHERE minute < ?`).run(minuteCutoff);- // Also clean old connection_logs if any remain- this.db.prepare(`DELETE FROM connection_logs WHERE timestamp < ?`).run(cutoff.toISOString());- // domain_proxy_stats and ip_proxy_stats are permanent aggregation tables (no time-based cleanup)- }- }-- // Vacuum database to reclaim space after clearing all data- if (days === 0) {- this.vacuum();- }-- return { deletedConnections, deletedLogs, deletedDomains, deletedIPs, deletedProxies, deletedRules };- }-- // Clean old ASN cache entries- cleanupASNCache(days: number): number {- const cutoff = new Date(Date.now() - days * 24 * 60 * 60 * 1000).toISOString();- const stmt = this.db.prepare(`DELETE FROM asn_cache WHERE queried_at < ?`);- const result = stmt.run(cutoff);- return result.changes;- }-- // Get database file size in bytes- getDatabaseSize(): number {- try {- const stats = fs.statSync(this.dbPath);- return stats.size;- } catch {- return 0;- }- }-- // Get minute stats count for a specific backend- getConnectionLogsCount(backendId: number): number {- const stmt = this.db.prepare(`- SELECT COUNT(*) as count FROM minute_stats WHERE backend_id = ?- `);- const result = stmt.get(backendId) as { count: number };- return result.count;- }-- // Get total minute stats count (all backends)- getTotalConnectionLogsCount(): number {- const stmt = this.db.prepare(`SELECT COUNT(*) as count FROM minute_stats`);- const result = stmt.get() as { count: number };- return result.count;- }-- // Retention Configuration Methods-- // Get retention configuration- getRetentionConfig(): DatabaseRetentionConfig {- const connectionLogsDays = this.db.prepare(`- SELECT value FROM app_config WHERE key = 'retention.connection_logs_days'- `).get() as { value: string } | undefined;-- const hourlyStatsDays = this.db.prepare(`- SELECT value FROM app_config WHERE key = 'retention.hourly_stats_days'- `).get() as { value: string } | undefined;-- const autoCleanup = this.db.prepare(`- SELECT value FROM app_config WHERE key = 'retention.auto_cleanup'- `).get() as { value: string } | undefined;-- return {- connectionLogsDays: parseInt(connectionLogsDays?.value || '7', 10),- hourlyStatsDays: parseInt(hourlyStatsDays?.value || '30', 10),- autoCleanup: autoCleanup?.value === '1',- };- }-- // Update retention configuration- updateRetentionConfig(updates: {- connectionLogsDays?: number;- hourlyStatsDays?: number;- autoCleanup?: boolean;- }): DatabaseRetentionConfig {- if (updates.connectionLogsDays !== undefined) {- this.db.prepare(`- INSERT INTO app_config (key, value) VALUES ('retention.connection_logs_days', ?)- ON CONFLICT(key) DO UPDATE SET value = excluded.value, updated_at = CURRENT_TIMESTAMP- `).run(updates.connectionLogsDays.toString());- }-- if (updates.hourlyStatsDays !== undefined) {- this.db.prepare(`- INSERT INTO app_config (key, value) VALUES ('retention.hourly_stats_days', ?)- ON CONFLICT(key) DO UPDATE SET value = excluded.value, updated_at = CURRENT_TIMESTAMP- `).run(updates.hourlyStatsDays.toString());- }-- if (updates.autoCleanup !== undefined) {- this.db.prepare(`- INSERT INTO app_config (key, value) VALUES ('retention.auto_cleanup', ?)- ON CONFLICT(key) DO UPDATE SET value = excluded.value, updated_at = CURRENT_TIMESTAMP- `).run(updates.autoCleanup ? '1' : '0');- }-- return this.getRetentionConfig();- }-- // Backend Management Methods-- // Create a new backend configuration- createBackend(backend: { name: string; url: string; token?: string }): number {- const stmt = this.db.prepare(`- INSERT INTO backend_configs (name, url, token, enabled, is_active, listening)- VALUES (?, ?, ?, 1, 0, 1)- `);- const result = stmt.run(backend.name, backend.url, backend.token || '');- return Number(result.lastInsertRowid);- }-- // Get all backend configurations- getAllBackends(): BackendConfig[] {- const stmt = this.db.prepare(`- SELECT id, name, url, token, enabled, is_active, listening, created_at, updated_at- FROM backend_configs- ORDER BY created_at ASC- `);- return stmt.all() as BackendConfig[];- }-- // Get a backend by ID- getBackend(id: number): BackendConfig | undefined {- const stmt = this.db.prepare(`- SELECT id, name, url, token, enabled, is_active, listening, created_at, updated_at- FROM backend_configs- WHERE id = ?- `);- return stmt.get(id) as BackendConfig | undefined;- }-- // Get the currently active backend- getActiveBackend(): BackendConfig | undefined {- const stmt = this.db.prepare(`- SELECT id, name, url, token, enabled, is_active, listening, created_at, updated_at- FROM backend_configs- WHERE is_active = 1- LIMIT 1- `);- return stmt.get() as BackendConfig | undefined;- }-- // Get all backends that should be listening (collecting data)- getListeningBackends(): BackendConfig[] {- const stmt = this.db.prepare(`- SELECT id, name, url, token, enabled, is_active, listening, created_at, updated_at- FROM backend_configs- WHERE listening = 1 AND enabled = 1- `);- return stmt.all() as BackendConfig[];- }-- // Update a backend configuration- updateBackend(id: number, updates: Partial<Omit<BackendConfig, 'id' | 'created_at'>>): void {- const sets: string[] = [];- const values: (string | number | boolean)[] = [];-- if (updates.name !== undefined) {- sets.push('name = ?');- values.push(updates.name);- }- if (updates.url !== undefined) {- sets.push('url = ?');- values.push(updates.url);- }- if (updates.token !== undefined) {- sets.push('token = ?');- values.push(updates.token);- }- if (updates.enabled !== undefined) {- sets.push('enabled = ?');- values.push(updates.enabled ? 1 : 0);- }- if (updates.listening !== undefined) {- sets.push('listening = ?');- values.push(updates.listening ? 1 : 0);- }- if (updates.is_active !== undefined) {- sets.push('is_active = ?');- values.push(updates.is_active ? 1 : 0);- }-- if (sets.length === 0) return;-- sets.push('updated_at = CURRENT_TIMESTAMP');- values.push(id);-- const stmt = this.db.prepare(`- UPDATE backend_configs- SET ${sets.join(', ')}- WHERE id = ?- `);- stmt.run(...values);- }-- // Set a backend as active (for display) - unsets all others- setActiveBackend(id: number): void {- this.db.exec('BEGIN TRANSACTION');- try {- // Unset all backends as active- this.db.prepare(`UPDATE backend_configs SET is_active = 0`).run();- // Set the specified backend as active- this.db.prepare(`UPDATE backend_configs SET is_active = 1, updated_at = CURRENT_TIMESTAMP WHERE id = ?`).run(id);- this.db.exec('COMMIT');- } catch (error) {- this.db.exec('ROLLBACK');- throw error;- }- }-- // Set listening state for a backend (controls data collection)- setBackendListening(id: number, listening: boolean): void {- const stmt = this.db.prepare(`- UPDATE backend_configs- SET listening = ?, updated_at = CURRENT_TIMESTAMP- WHERE id = ?- `);- stmt.run(listening ? 1 : 0, id);- }-- // Delete a backend and all its associated data- deleteBackend(id: number): void {- // Due to ON DELETE CASCADE, all associated stats will be deleted automatically- const stmt = this.db.prepare(`DELETE FROM backend_configs WHERE id = ?`);- stmt.run(id);- }-- // Delete all data for a specific backend- deleteBackendData(id: number): void {- this.db.exec('BEGIN TRANSACTION');- try {- this.db.prepare(`DELETE FROM domain_stats WHERE backend_id = ?`).run(id);- this.db.prepare(`DELETE FROM ip_stats WHERE backend_id = ?`).run(id);- this.db.prepare(`DELETE FROM proxy_stats WHERE backend_id = ?`).run(id);- this.db.prepare(`DELETE FROM rule_stats WHERE backend_id = ?`).run(id);- this.db.prepare(`DELETE FROM rule_proxy_map WHERE backend_id = ?`).run(id);- this.db.prepare(`DELETE FROM rule_chain_traffic WHERE backend_id = ?`).run(id);- this.db.prepare(`DELETE FROM rule_domain_traffic WHERE backend_id = ?`).run(id);- this.db.prepare(`DELETE FROM rule_ip_traffic WHERE backend_id = ?`).run(id);- this.db.prepare(`DELETE FROM country_stats WHERE backend_id = ?`).run(id);- this.db.prepare(`DELETE FROM hourly_stats WHERE backend_id = ?`).run(id);- this.db.prepare(`DELETE FROM connection_logs WHERE backend_id = ?`).run(id);- this.db.prepare(`DELETE FROM minute_stats WHERE backend_id = ?`).run(id);- this.db.prepare(`DELETE FROM minute_dim_stats WHERE backend_id = ?`).run(id);- this.db.prepare(`DELETE FROM minute_country_stats WHERE backend_id = ?`).run(id);- this.db.prepare(`DELETE FROM domain_proxy_stats WHERE backend_id = ?`).run(id);- this.db.prepare(`DELETE FROM ip_proxy_stats WHERE backend_id = ?`).run(id);- this.db.prepare(`DELETE FROM device_stats WHERE backend_id = ?`).run(id);- this.db.prepare(`DELETE FROM device_domain_stats WHERE backend_id = ?`).run(id);- this.db.prepare(`DELETE FROM device_ip_stats WHERE backend_id = ?`).run(id);- this.db.exec('COMMIT');- } catch (error) {- this.db.exec('ROLLBACK');- throw error;- }- }-- // Get total stats across all backends- getGlobalSummary(): { totalConnections: number; totalUpload: number; totalDownload: number; uniqueDomains: number; uniqueIPs: number; backendCount: number } {- // Calculate totals from ip_stats to include traffic with unknown domains- const trafficStmt = this.db.prepare(`- SELECT- COALESCE(SUM(total_connections), 0) as connections,- COALESCE(SUM(total_upload), 0) as upload,- COALESCE(SUM(total_download), 0) as download- FROM ip_stats- `);- const { connections, upload, download } = trafficStmt.get() as {- connections: number;- upload: number;- download: number;- };-- const domainsStmt = this.db.prepare(`- SELECT COUNT(DISTINCT domain) as count FROM domain_stats- `);- const uniqueDomains = (domainsStmt.get() as { count: number }).count;-- const ipsStmt = this.db.prepare(`- SELECT COUNT(DISTINCT ip) as count FROM ip_stats- `);- const uniqueIPs = (ipsStmt.get() as { count: number }).count;-- const backendStmt = this.db.prepare(`- SELECT COUNT(*) as count FROM backend_configs- `);- const backendCount = (backendStmt.get() as { count: number }).count;-- return {- totalConnections: connections,- totalUpload: upload,- totalDownload: download,- uniqueDomains,- uniqueIPs,- backendCount- };- }-- vacuum(): void {- this.db.exec('VACUUM');- }-- // Cleanup methods for data retention- deleteOldMinuteStats(cutoff: string): number {- const minuteCutoff = cutoff.slice(0, 16) + ':00';- const stmt = this.db.prepare(`- DELETE FROM minute_stats WHERE minute < ?- `);- this.db.prepare(`DELETE FROM minute_dim_stats WHERE minute < ?`).run(minuteCutoff);- this.db.prepare(`DELETE FROM minute_country_stats WHERE minute < ?`).run(minuteCutoff);- // Also clean old connection_logs if any remain from before migration- this.db.prepare(`DELETE FROM connection_logs WHERE timestamp < ?`).run(cutoff);- return stmt.run(minuteCutoff).changes;- }-- // Keep old method name as alias for backward compatibility- deleteOldConnectionLogs(cutoff: string): number {- return this.deleteOldMinuteStats(cutoff);- }-- deleteOldHourlyStats(cutoff: string): number {- const stmt = this.db.prepare(`- DELETE FROM hourly_stats WHERE hour < ?- `);- return stmt.run(cutoff).changes;- }-- getCleanupStats(): {- connectionLogsCount: number;- hourlyStatsCount: number;- oldestConnectionLog: string | null;- oldestHourlyStat: string | null;- } {- const logsCountStmt = this.db.prepare('SELECT COUNT(*) as count FROM minute_stats');- const hourlyCountStmt = this.db.prepare('SELECT COUNT(*) as count FROM hourly_stats');- const oldestLogStmt = this.db.prepare('SELECT MIN(minute) as ts FROM minute_stats');- const oldestHourlyStmt = this.db.prepare('SELECT MIN(hour) as hr FROM hourly_stats');-- return {- connectionLogsCount: (logsCountStmt.get() as { count: number }).count,- hourlyStatsCount: (hourlyCountStmt.get() as { count: number }).count,- oldestConnectionLog: (oldestLogStmt.get() as { ts: string | null })?.ts || null,- oldestHourlyStat: (oldestHourlyStmt.get() as { hr: string | null })?.hr || null,- };- }-- // Auth Configuration Methods-- // Get auth configuration- getAuthConfig(): { enabled: boolean; tokenHash: string | null; updatedAt: string } {- const enabledStmt = this.db.prepare(`- SELECT value, updated_at FROM auth_config WHERE key = 'enabled'- `);- const tokenStmt = this.db.prepare(`- SELECT value, updated_at FROM auth_config WHERE key = 'token_hash'- `);-- const enabledRow = enabledStmt.get() as { value: string; updated_at: string } | undefined;- const tokenRow = tokenStmt.get() as { value: string; updated_at: string } | undefined;-- return {- enabled: enabledRow?.value === '1',- tokenHash: tokenRow?.value || null,- updatedAt: tokenRow?.updated_at || enabledRow?.updated_at || new Date().toISOString(),- };- }-- // Update auth configuration- updateAuthConfig(updates: { enabled?: boolean; tokenHash?: string | null }): void {- if (updates.enabled !== undefined) {- this.db.prepare(`- INSERT INTO auth_config (key, value) VALUES ('enabled', ?)- ON CONFLICT(key) DO UPDATE SET value = excluded.value, updated_at = CURRENT_TIMESTAMP- `).run(updates.enabled ? '1' : '0');- }-- if (updates.tokenHash !== undefined) {- this.db.prepare(`- INSERT INTO auth_config (key, value) VALUES ('token_hash', ?)- ON CONFLICT(key) DO UPDATE SET value = excluded.value, updated_at = CURRENT_TIMESTAMP- `).run(updates.tokenHash || '');- }- }+ // ==================== Delegated to repositories ====================++ // Config+ cleanupOldData(backendId: number | null, days: number) { return this.repos.config.cleanupOldData(backendId, days); }+ cleanupASNCache(days: number) { return this.repos.config.cleanupASNCache(days); }+ getDatabaseSize() { return this.repos.config.getDatabaseSize(); }+ getConnectionLogsCount(backendId: number) { return this.repos.config.getConnectionLogsCount(backendId); }+ getTotalConnectionLogsCount() { return this.repos.config.getTotalConnectionLogsCount(); }+ getRetentionConfig() { return this.repos.config.getRetentionConfig(); }+ updateRetentionConfig(updates: { connectionLogsDays?: number; hourlyStatsDays?: number; autoCleanup?: boolean }) { return this.repos.config.updateRetentionConfig(updates); }+ vacuum() { this.repos.config.vacuum(); }+ deleteOldMinuteStats(cutoff: string) { return this.repos.config.deleteOldMinuteStats(cutoff); }+ deleteOldConnectionLogs(cutoff: string) { return this.repos.config.deleteOldConnectionLogs(cutoff); }+ deleteOldHourlyStats(cutoff: string) { return this.repos.config.deleteOldHourlyStats(cutoff); }+ getCleanupStats() { return this.repos.config.getCleanupStats(); }++ // Backend+ createBackend(backend: { name: string; url: string; token?: string; type?: 'clash' | 'surge' }) { return this.repos.backend.createBackend(backend); }+ getAllBackends() { return this.repos.backend.getAllBackends(); }+ getBackend(id: number) { return this.repos.backend.getBackend(id); }+ getActiveBackend() { return this.repos.backend.getActiveBackend(); }+ getListeningBackends() { return this.repos.backend.getListeningBackends(); }+ updateBackend(id: number, updates: Partial<Omit<BackendConfig, 'id' | 'created_at'>>) { this.repos.backend.updateBackend(id, updates); }+ setActiveBackend(id: number) { this.repos.backend.setActiveBackend(id); }+ setBackendListening(id: number, listening: boolean) { this.repos.backend.setBackendListening(id, listening); }+ deleteBackend(id: number) { this.repos.backend.deleteBackend(id); }+ deleteBackendData(id: number) { this.repos.backend.deleteBackendData(id); }+ getGlobalSummary() { return this.repos.backend.getGlobalSummary(); }++ // Auth+ getAuthConfig() { return this.repos.auth.getAuthConfig(); }+ updateAuthConfig(updates: { enabled?: boolean; tokenHash?: string | null }) { this.repos.auth.updateAuthConfig(updates); }++ // Surge+ getSurgePolicyCache(backendId: number) { return this.repos.surge.getSurgePolicyCache(backendId); }+ updateSurgePolicyCache(backendId: number, policies: Array<{ policyGroup: string; selectedPolicy: string | null; policyType?: string; allPolicies?: string[] }>) { this.repos.surge.updateSurgePolicyCache(backendId, policies); }+ getSurgePolicyCacheLastUpdate(backendId: number) { return this.repos.surge.getSurgePolicyCacheLastUpdate(backendId); }+ clearSurgePolicyCache(backendId: number) { this.repos.surge.clearSurgePolicyCache(backendId); }close() {this.db.close();diff --git a/apps/collector/src/index.ts b/apps/collector/src/index.tsindex 3a70353..2c048ea 100644--- a/apps/collector/src/index.ts+++ b/apps/collector/src/index.ts@@ -13,8 +13,10 @@ config();import { StatsDatabase, BackendConfig } from './db.js';import { createCollector, GatewayCollector } from './collector.js';+import { createSurgeCollector, SurgeCollector } from './surge-collector.js';import { StatsWebSocketServer } from './websocket.js';import { realtimeStore } from './realtime.js';+import { SurgePolicySyncService } from './modules/surge/surge-policy-sync.js';let wsServer: StatsWebSocketServer;@@ -25,12 +27,13 @@ const COLLECTOR_WS_PORT = parseInt(process.env.COLLECTOR_WS_PORT || '3002');const API_PORT = parseInt(process.env.API_PORT || '3001');const DB_PATH = process.env.DB_PATH || path.join(process.cwd(), 'stats.db');-// Map of backend connections: backendId -> GatewayCollector-const collectors = new Map<number, GatewayCollector>();+// Map of backend connections: backendId -> GatewayCollector | SurgeCollector+const collectors = new Map<number, GatewayCollector | SurgeCollector>();let db: StatsDatabase;let apiServer: APIServer;let geoService: GeoIPService;+let policySyncService: SurgePolicySyncService;// Track last known backend configs to detect changeslet lastBackendConfigs: Map<number, BackendConfig> = new Map();@@ -50,9 +53,12 @@ async function main() {wsServer = new StatsWebSocketServer(COLLECTOR_WS_PORT, db);wsServer.start();+ // Initialize policy sync service+ policySyncService = new SurgePolicySyncService(db);+// Initialize API serverconsole.log('[Main] Starting API server on port', API_PORT);- apiServer = new APIServer(API_PORT, db, realtimeStore);+ apiServer = new APIServer(API_PORT, db, realtimeStore, policySyncService);apiServer.start();// Start backend management loop@@ -108,6 +114,7 @@ async function manageBackends() {const needsRestart = existingCollector && lastConfig && (lastConfig.url !== backend.url ||lastConfig.token !== backend.token ||+ lastConfig.type !== backend.type ||lastConfig.listening !== backend.listening ||lastConfig.enabled !== backend.enabled);@@ -152,29 +159,50 @@ function startCollector(backend: BackendConfig) {return;}- console.log(`[Collector] Starting collector for backend "${backend.name}" (ID: ${backend.id}) at ${backend.url}`);+ console.log(`[Collector] Starting ${backend.type || 'clash'} collector for backend "${backend.name}" (ID: ${backend.id}) at ${backend.url}`);++ if (backend.type === 'surge') {+ // Start policy sync service for Surge+ const baseUrl = backend.url.replace(/\/$/, '');+ policySyncService.startSync(backend.id, baseUrl, backend.token || undefined);++ // Create and start Surge collector (REST API polling)+ const collector = createSurgeCollector(+ db,+ backend.url,+ backend.token || undefined,+ geoService,+ () => {+ // Broadcast stats update via WebSocket when new data arrives+ wsServer.broadcastStats(backend.id);+ },+ backend.id // Pass backend ID for data isolation+ );++ collectors.set(backend.id, collector);+ collector.start();+ } else {+ // Create and start Clash collector (WebSocket)+ let wsUrl = backend.url.replace(/^http:\/\//, 'ws://').replace(/^https:\/\//, 'wss://');+ if (!wsUrl.endsWith('/connections')) {+ wsUrl = `${wsUrl}/connections`;+ }- // Convert HTTP URL to WebSocket URL- let wsUrl = backend.url.replace(/^http:\/\//, 'ws://').replace(/^https:\/\//, 'wss://');- if (!wsUrl.endsWith('/connections')) {- wsUrl = `${wsUrl}/connections`;+ const collector = createCollector(+ db,+ wsUrl,+ backend.token || undefined,+ geoService,+ () => {+ // Broadcast stats update via WebSocket when new data arrives+ wsServer.broadcastStats(backend.id);+ },+ backend.id // Pass backend ID for data isolation+ );++ collectors.set(backend.id, collector);+ collector.connect();}-- // Create and start collector- const collector = createCollector(- db,- wsUrl,- backend.token || undefined,- geoService,- () => {- // Broadcast stats update via WebSocket when new data arrives- wsServer.broadcastStats(backend.id);- },- backend.id // Pass backend ID for data isolation- );-- collectors.set(backend.id, collector);- collector.connect();}// Stop a collector for a specific backend@@ -182,19 +210,31 @@ function stopCollector(backendId: number) {const collector = collectors.get(backendId);if (collector) {console.log(`[Collector] Stopping collector for backend ID: ${backendId}`);- collector.disconnect();+ if (collector instanceof GatewayCollector) {+ collector.disconnect();+ } else {+ collector.stop();+ }collectors.delete(backendId);}++ // Also stop policy sync for this backend+ policySyncService.stopSync(backendId);}// Graceful shutdownfunction shutdown() {console.log('[Main] Shutting down...');- // Stop all collectors+ // Stop all collectors and policy syncfor (const [id, collector] of collectors) {console.log(`[Main] Disconnecting collector for backend ID: ${id}`);- collector.disconnect();+ if (collector instanceof GatewayCollector) {+ collector.disconnect();+ } else {+ collector.stop();+ }+ policySyncService.stopSync(id);}collectors.clear();diff --git a/apps/collector/src/modules/backend/backend.controller.ts b/apps/collector/src/modules/backend/backend.controller.tsindex b8b9f5e..527d92b 100644--- a/apps/collector/src/modules/backend/backend.controller.ts+++ b/apps/collector/src/modules/backend/backend.controller.ts@@ -49,14 +49,14 @@ const backendController: FastifyPluginAsync = async (fastify: FastifyInstance):return reply.status(403).send({ error: 'Forbidden' });}- const { name, url, token } = request.body;+ const { name, url, token, type } = request.body;if (!name || !url) {return reply.status(400).send({ error: 'Name and URL are required' });}try {- const result = service.createBackend({ name, url, token });+ const result = service.createBackend({ name, url, token, type });return result;} catch (error: any) {if (error.message?.includes('UNIQUE constraint failed')) {diff --git a/apps/collector/src/modules/backend/backend.service.ts b/apps/collector/src/modules/backend/backend.service.tsindex ff14ac8..4c49c4a 100644--- a/apps/collector/src/modules/backend/backend.service.ts+++ b/apps/collector/src/modules/backend/backend.service.ts@@ -1,5 +1,6 @@/*** Backend Service - Business logic for backend management+ * Includes automatic health checking for upstream gateways*/import type { StatsDatabase } from '../../db.js';@@ -9,6 +10,7 @@ import type {CreateBackendInput,UpdateBackendInput,BackendResponse,+ BackendHealthInfo,TestConnectionInput,TestConnectionResult,CreateBackendResult,@@ -33,6 +35,10 @@ function maskUrl(url: string): string {}export class BackendService {+ private healthStatus = new Map<number, BackendHealthInfo>();+ private healthCheckInterval: NodeJS.Timeout | null = null;+ private readonly HEALTH_CHECK_INTERVAL_MS = 30000; // 30 seconds+constructor(private db: StatsDatabase,private realtimeStore: RealtimeStore,@@ -40,21 +46,111 @@ export class BackendService {) {}/**- * Get all backends (with token hidden)+ * Start automatic health checks for all listening backends+ */+ startHealthChecks(): void {+ if (this.healthCheckInterval) return;++ console.log('[BackendService] Starting automatic health checks');++ // Run initial check+ this.runHealthChecks();++ // Schedule periodic checks+ this.healthCheckInterval = setInterval(() => {+ this.runHealthChecks();+ }, this.HEALTH_CHECK_INTERVAL_MS);+ }++ /**+ * Stop automatic health checks+ */+ stopHealthChecks(): void {+ if (this.healthCheckInterval) {+ clearInterval(this.healthCheckInterval);+ this.healthCheckInterval = null;+ console.log('[BackendService] Stopped automatic health checks');+ }+ }++ /**+ * Get health status for a specific backend+ */+ getHealthStatus(backendId: number): BackendHealthInfo | undefined {+ return this.healthStatus.get(backendId);+ }++ /**+ * Run health checks for all listening backends+ */+ private async runHealthChecks(): Promise<void> {+ const backends = this.db.getListeningBackends();++ for (const backend of backends) {+ try {+ const startTime = Date.now();+ const result = await this.testConnection({+ url: backend.url,+ token: backend.token,+ type: backend.type,+ });+ const latency = Date.now() - startTime;++ const health: BackendHealthInfo = {+ status: result.success ? 'healthy' : 'unhealthy',+ lastChecked: Date.now(),+ message: result.message,+ latency: result.success ? latency : undefined,+ };++ // Only log when status changes or on failure+ const prevHealth = this.healthStatus.get(backend.id);+ if (!result.success || prevHealth?.status !== health.status) {+ console.log(`[BackendService] Health check for ${backend.name}: ${health.status}${result.message ? ` - ${result.message}` : ''}`);+ }++ this.healthStatus.set(backend.id, health);+ } catch (error) {+ const health: BackendHealthInfo = {+ status: 'unhealthy',+ lastChecked: Date.now(),+ message: error instanceof Error ? error.message : 'Health check failed',+ };+ this.healthStatus.set(backend.id, health);+ console.warn(`[BackendService] Health check error for ${backend.name}:`, error);+ }+ }+ }++ /**+ * Attach health status to backend response+ */+ private attachHealthStatus(backend: BackendResponse): BackendResponse {+ const health = this.healthStatus.get(backend.id);+ if (health) {+ return { ...backend, health };+ }+ return backend;+ }++ /**+ * Get all backends (with token hidden and health status attached)*/getAllBackends(): BackendResponse[] {const backends = this.db.getAllBackends();const isShowcase = this.authService.isShowcaseMode();- return backends.map(({ token, ...rest }) => ({- ...rest,- hasToken: !!token,- url: isShowcase ? maskUrl(rest.url) : rest.url,- }));+ return backends.map(({ token, ...rest }) =>+ this.attachHealthStatus({+ ...rest,+ hasToken: !!token,+ url: isShowcase ? maskUrl(rest.url) : rest.url,+ })+ );}/**- * Get active backend+ * Get active backend (with health status attached)*/getActiveBackend(): BackendResponse | { error: string } {const backend = this.db.getActiveBackend();@@ -64,25 +160,27 @@ export class BackendService {const { token, ...rest } = backend;const isShowcase = this.authService.isShowcaseMode();- return {+ return this.attachHealthStatus({...rest,hasToken: !!token,url: isShowcase ? maskUrl(rest.url) : rest.url,- };+ });}/**- * Get listening backends+ * Get listening backends (with health status attached)*/getListeningBackends(): BackendResponse[] {const backends = this.db.getListeningBackends();const isShowcase = this.authService.isShowcaseMode();- return backends.map(({ token, ...rest }) => ({- ...rest,- hasToken: !!token,- url: isShowcase ? maskUrl(rest.url) : rest.url,- }));+ return backends.map(({ token, ...rest }) =>+ this.attachHealthStatus({+ ...rest,+ hasToken: !!token,+ url: isShowcase ? maskUrl(rest.url) : rest.url,+ })+ );}/**@@ -105,13 +203,13 @@ export class BackendService {* Create a new backend*/createBackend(input: CreateBackendInput): CreateBackendResult {- const { name, url, token } = input;+ const { name, url, token, type = 'clash' } = input;// Check if this is the first backendconst existingBackends = this.db.getAllBackends();const isFirstBackend = existingBackends.length === 0;- const id = this.db.createBackend({ name, url, token });+ const id = this.db.createBackend({ name, url, token, type });// If this is the first backend, automatically set it as activeif (isFirstBackend) {@@ -176,6 +274,7 @@ export class BackendService {return this.testConnection({url: backend.url,token: backend.token,+ type: backend.type,});}@@ -183,8 +282,19 @@ export class BackendService {* Test connection to a backend*/async testConnection(input: TestConnectionInput): Promise<TestConnectionResult> {- const { url, token } = input;+ const { url, token, type = 'clash' } = input;++ if (type === 'surge') {+ return this.testSurgeConnection(url, token);+ }+ return this.testClashConnection(url, token);+ }++ /**+ * Test Clash WebSocket connection+ */+ private async testClashConnection(url: string, token?: string): Promise<TestConnectionResult> {try {const wsUrl = url.replace('http://', 'ws://').replace('https://', 'wss://');const fullUrl = wsUrl.includes('/connections') ? wsUrl : `${wsUrl}/connections`;@@ -194,7 +304,6 @@ export class BackendService {headers['Authorization'] = `Bearer ${token}`;}- // Try to establish WebSocket connectionconst WebSocket = (await import('ws')).default;return new Promise((resolve) => {@@ -235,4 +344,51 @@ export class BackendService {return { success: false, message };}}++ /**+ * Test Surge HTTP REST API connection+ * Uses /v1/environment endpoint for lightweight health check+ */+ private async testSurgeConnection(url: string, token?: string): Promise<TestConnectionResult> {+ try {+ const baseUrl = url.replace(/\/$/, '');+ // Use /v1/environment for health check (lightweight, always available)+ const testUrl = `${baseUrl}/v1/environment`;++ const headers: Record<string, string> = {+ Accept: 'application/json',+ };++ if (token) {+ headers['x-key'] = token;+ }++ const response = await fetch(testUrl, {+ headers,+ signal: AbortSignal.timeout(5000),+ });++ if (!response.ok) {+ if (response.status === 401 || response.status === 403) {+ return { success: false, message: 'Authentication failed - check your API key' };+ }+ return { success: false, message: `HTTP ${response.status}: ${response.statusText}` };+ }++ const data = await response.json() as Record<string, unknown>;+ if (data && typeof data === 'object' && data.deviceName) {+ return { success: true, message: `Connected to Surge (${String(data.deviceName)})` };+ }++ return { success: false, message: 'Invalid response format from Surge API' };+ } catch (error: unknown) {+ if (error instanceof Error) {+ if (error.name === 'AbortError') {+ return { success: false, message: 'Connection timeout - check if Surge HTTP API is enabled' };+ }+ return { success: false, message: error.message };+ }+ return { success: false, message: 'Connection failed' };+ }+ }}diff --git a/apps/collector/src/modules/backend/backend.types.ts b/apps/collector/src/modules/backend/backend.types.tsindex 2d7ac78..d76c626 100644--- a/apps/collector/src/modules/backend/backend.types.ts+++ b/apps/collector/src/modules/backend/backend.types.ts@@ -7,6 +7,7 @@ export interface BackendConfig {name: string;url: string;token: string;+ type: 'clash' | 'surge';enabled: boolean;is_active: boolean;listening: boolean;@@ -21,31 +22,43 @@ export interface CreateBackendInput {name: string;url: string;token?: string;+ type?: 'clash' | 'surge';}export interface UpdateBackendInput {name?: string;url?: string;token?: string;+ type?: 'clash' | 'surge';enabled?: boolean;listening?: boolean;}+export interface BackendHealthInfo {+ status: 'healthy' | 'unhealthy' | 'unknown';+ lastChecked: number;+ message?: string;+ latency?: number;+}+export interface BackendResponse {id: number;name: string;url: string;+ type: 'clash' | 'surge';enabled: boolean;is_active: boolean;listening: boolean;created_at: string;updated_at: string;hasToken: boolean;+ health?: BackendHealthInfo;}export interface TestConnectionInput {url: string;token?: string;+ type?: 'clash' | 'surge';}export interface TestConnectionResult {diff --git a/apps/collector/src/modules/surge/surge-policy-sync.ts b/apps/collector/src/modules/surge/surge-policy-sync.tsnew file mode 100644index 0000000..071d7aa--- /dev/null+++ b/apps/collector/src/modules/surge/surge-policy-sync.ts@@ -0,0 +1,177 @@+/**+ * Surge Policy Sync Service+ *+ * Manages background synchronization of Surge policy group selections.+ * Caches policy data in database to avoid frequent API calls.+ */++import type { StatsDatabase } from '../../db.js';++interface SurgePolicyDetail {+ policyGroup: string;+ selectedPolicy: string | null;+ policyType: string;+ allPolicies: string[];+}++export class SurgePolicySyncService {+ private syncTimers = new Map<number, NodeJS.Timeout>();+ private readonly SYNC_INTERVAL = 10 * 60 * 1000; // 10 minutes default+ private readonly SYNC_INTERVAL_MS = parseInt(process.env.SURGE_POLICY_SYNC_INTERVAL_MS || '600000');++ constructor(private db: StatsDatabase) {}++ /**+ * Start periodic sync for a backend+ */+ startSync(backendId: number, baseUrl: string, token?: string): void {+ this.stopSync(backendId);++ const interval = this.SYNC_INTERVAL_MS || this.SYNC_INTERVAL;++ // Execute immediately+ this.syncNow(backendId, baseUrl, token).catch(err => {+ console.error(`[SurgePolicySync:${backendId}] Initial sync failed:`, err.message);+ });++ // Set up periodic sync+ const timer = setInterval(() => {+ this.syncNow(backendId, baseUrl, token).catch(err => {+ console.error(`[SurgePolicySync:${backendId}] Scheduled sync failed:`, err.message);+ });+ }, interval);++ this.syncTimers.set(backendId, timer);+ console.log(`[SurgePolicySync:${backendId}] Started, interval: ${interval}ms`);+ }++ /**+ * Stop periodic sync for a backend+ */+ stopSync(backendId: number): void {+ const timer = this.syncTimers.get(backendId);+ if (timer) {+ clearInterval(timer);+ this.syncTimers.delete(backendId);+ console.log(`[SurgePolicySync:${backendId}] Stopped`);+ }+ }++ /**+ * Perform immediate sync+ */+ async syncNow(+ backendId: number,+ baseUrl: string,+ token?: string+ ): Promise<{ success: boolean; updated: number; message: string }> {+ const headers: Record<string, string> = {+ 'Accept': 'application/json',+ };+ if (token) {+ headers['x-key'] = token;+ }++ try {+ // 1. Get policy groups list+ const res = await fetch(`${baseUrl}/v1/policies`, {+ headers,+ signal: AbortSignal.timeout(10000)+ });++ if (!res.ok) {+ throw new Error(`Failed to fetch policies: ${res.status}`);+ }++ const data = await res.json() as {+ proxies: string[];+ 'policy-groups': string[];+ };++ const policyGroups = data['policy-groups'] || [];++ // 2. Fetch details for each policy group (with concurrency limit)+ const details: SurgePolicyDetail[] = [];+ const CONCURRENCY = 3;++ for (let i = 0; i < policyGroups.length; i += CONCURRENCY) {+ const batch = policyGroups.slice(i, i + CONCURRENCY);+ const results = await Promise.allSettled(+ batch.map(name => this.fetchPolicyDetail(baseUrl, name, headers))+ );++ for (const result of results) {+ if (result.status === 'fulfilled' && result.value) {+ details.push(result.value);+ }+ }+ }++ // 3. Update database cache+ if (details.length > 0) {+ this.db.updateSurgePolicyCache(backendId, details);+ }++ console.log(`[SurgePolicySync:${backendId}] Synced ${details.length}/${policyGroups.length} policies`);++ return {+ success: true,+ updated: details.length,+ message: `Updated ${details.length} policies`,+ };+ } catch (error) {+ const message = error instanceof Error ? error.message : 'Sync failed';+ console.error(`[SurgePolicySync:${backendId}] ${message}`);+ return { success: false, updated: 0, message };+ }+ }++ private async fetchPolicyDetail(+ baseUrl: string,+ groupName: string,+ headers: Record<string, string>+ ): Promise<SurgePolicyDetail | null> {+ try {+ // Surge uses /v1/policy_groups/select?group_name=xxx endpoint+ const res = await fetch(+ `${baseUrl}/v1/policy_groups/select?group_name=${encodeURIComponent(groupName)}`,+ { headers, signal: AbortSignal.timeout(5000) }+ );++ if (!res.ok) return null;++ const detail = await res.json() as {+ policy?: string;+ type?: string;+ policies?: string[];+ };++ return {+ policyGroup: groupName,+ selectedPolicy: detail.policy || null,+ policyType: detail.type || 'Select',+ allPolicies: detail.policies || [],+ };+ } catch {+ return null;+ }+ }++ /**+ * Get cache status for a backend+ */+ getCacheStatus(backendId: number): {+ cached: boolean;+ lastUpdate: string | null;+ policyCount: number;+ } {+ const lastUpdate = this.db.getSurgePolicyCacheLastUpdate(backendId);+ const policies = this.db.getSurgePolicyCache(backendId);++ return {+ cached: policies.length > 0,+ lastUpdate,+ policyCount: policies.length,+ };+ }+}diff --git a/apps/collector/src/surge-collector.ts b/apps/collector/src/surge-collector.tsnew file mode 100644index 0000000..e9ac44b--- /dev/null+++ b/apps/collector/src/surge-collector.ts@@ -0,0 +1,943 @@+import { isIPv4, isIPv6 } from "net";+import type { StatsDatabase } from "./db.js";+import { GeoIPService } from "./geo-service.js";+import { realtimeStore } from "./realtime.js";+import type { SurgeRequest, SurgeRequestsData } from "@neko-master/shared";+import { calculateBackoffDelay } from "./utils.js";++// Debug configuration+const DEBUG_SURGE = process.env.DEBUG_SURGE === "true";+const STALE_CONNECTION_TIMEOUT = 5 * 60 * 1000; // 5 minutes+const CLEANUP_INTERVAL = 2 * 60 * 1000; // 2 minutes++interface SurgeCollectorOptions {+ url: string;+ token?: string;+ pollInterval?: number;+ onData?: (data: SurgeRequestsData) => void;+ onError?: (error: Error) => void;+}++interface TrafficUpdate {+ domain: string;+ ip: string;+ chain: string;+ chains: string[];+ rule: string;+ rulePayload: string;+ upload: number;+ download: number;+ sourceIP?: string;+ timestampMs?: number;+}++interface GeoIPResult {+ ip: string;+ geo: {+ country: string;+ country_name: string;+ continent: string;+ } | null;+ upload: number;+ download: number;+ timestampMs?: number;+}++function toMinuteKey(timestampMs?: number): string {+ const date = new Date(timestampMs ?? Date.now()).toISOString();+ return `${date.slice(0, 16)}:00`;+}++export class SurgeCollector {+ private url: string;+ private token?: string;+ private pollInterval: number;+ private onData?: (data: SurgeRequestsData) => void;+ private onError?: (error: Error) => void;+ private pollTimer: NodeJS.Timeout | null = null;+ private isClosing = false;+ private backendId: number;+ private consecutiveErrors = 0;+ private readonly MAX_RETRY_ATTEMPTS = 5;+ private readonly MAX_RETRY_DELAY = 60000; // 1 minute+ private readonly BASE_RETRY_DELAY = 2000; // 2 seconds++ constructor(backendId: number, options: SurgeCollectorOptions) {+ this.backendId = backendId;+ this.url = options.url;+ this.token = options.token;+ this.pollInterval = options.pollInterval || 2000;+ this.onData = options.onData;+ this.onError = options.onError;+ }++ start() {+ if (this.isClosing) return;++ console.log(+ `[SurgeCollector:${this.backendId}] Starting polling ${this.url}...`+ );++ this.poll();+ }++ private async poll() {+ if (this.isClosing) return;++ try {+ const data = await this.fetchWithRetry();+ if (data) {+ this.onData?.(data);+ // Reset error counter on success+ if (this.consecutiveErrors > 0) {+ console.log(`[SurgeCollector:${this.backendId}] Recovered after ${this.consecutiveErrors} errors`);+ this.consecutiveErrors = 0;+ }+ }+ } catch (err) {+ const error = err instanceof Error ? err : new Error(String(err));+ this.consecutiveErrors++;+ console.error(+ `[SurgeCollector:${this.backendId}] Poll error (${this.consecutiveErrors}/${this.MAX_RETRY_ATTEMPTS}):`,+ error.message+ );+ this.onError?.(error);+ }++ if (!this.isClosing) {+ // Calculate next poll delay with backoff if errors occurred+ const delay = this.consecutiveErrors > 0+ ? calculateBackoffDelay(+ this.consecutiveErrors - 1,+ this.BASE_RETRY_DELAY,+ this.MAX_RETRY_DELAY+ )+ : this.pollInterval;++ this.pollTimer = setTimeout(() => this.poll(), delay);+ }+ }++ private async fetchWithRetry(): Promise<SurgeRequestsData | null> {+ let lastError: Error | undefined;++ for (let attempt = 0; attempt < this.MAX_RETRY_ATTEMPTS; attempt++) {+ try {+ return await this.fetchRequests();+ } catch (err) {+ lastError = err instanceof Error ? err : new Error(String(err));++ // Don't retry on 4xx errors (client errors)+ if (lastError.message.includes('HTTP 4')) {+ throw lastError;+ }++ // Last attempt failed+ if (attempt === this.MAX_RETRY_ATTEMPTS - 1) {+ break;+ }++ // Wait before retry+ const delay = calculateBackoffDelay(attempt, this.BASE_RETRY_DELAY, this.MAX_RETRY_DELAY);+ console.log(`[SurgeCollector:${this.backendId}] Retrying in ${delay}ms (attempt ${attempt + 2}/${this.MAX_RETRY_ATTEMPTS})`);+ await new Promise(resolve => setTimeout(resolve, delay));+ }+ }++ throw lastError;+ }++ private async fetchRequests(): Promise<SurgeRequestsData | null> {+ const url = this.url.endsWith("/v1/requests/recent")+ ? this.url+ : `${this.url}/v1/requests/recent`;++ const headers: Record<string, string> = {+ Accept: "application/json",+ };++ if (this.token) {+ headers["x-key"] = this.token;+ }++ const response = await fetch(url, {+ headers,+ signal: AbortSignal.timeout(10000),+ });++ if (!response.ok) {+ throw new Error(`HTTP ${response.status}: ${response.statusText}`);+ }++ return response.json() as Promise<SurgeRequestsData>;+ }++ stop() {+ this.isClosing = true;+ if (this.pollTimer) {+ clearTimeout(this.pollTimer);+ this.pollTimer = null;+ }+ console.log(`[SurgeCollector:${this.backendId}] Stopped`);+ }+}++// Batch buffer for traffic updates+class BatchBuffer {+ private buffer: Map<string, TrafficUpdate> = new Map();+ private geoQueue: GeoIPResult[] = [];+ private lastLogTime = 0;+ private logCounter = 0;++ add(backendId: number, update: TrafficUpdate) {+ const minuteKey = toMinuteKey(update.timestampMs);+ const fullChain = update.chains.join(" > ");+ const key = [+ backendId,+ minuteKey,+ update.domain,+ update.ip,+ update.chain,+ fullChain,+ update.rule,+ update.rulePayload,+ update.sourceIP || "",+ ].join(":");+ const existing = this.buffer.get(key);++ if (existing) {+ existing.upload += update.upload;+ existing.download += update.download;+ if ((update.timestampMs ?? 0) > (existing.timestampMs ?? 0)) {+ existing.timestampMs = update.timestampMs;+ }+ } else {+ this.buffer.set(key, { ...update });+ }+ }++ addGeoResult(result: GeoIPResult) {+ this.geoQueue.push(result);+ }++ size(): number {+ return this.buffer.size;+ }++ hasPending(): boolean {+ return this.buffer.size > 0 || this.geoQueue.length > 0;+ }++ flush(+ db: StatsDatabase,+ _geoService: GeoIPService | undefined,+ backendId: number+ ): {+ domains: number;+ rules: number;+ trafficOk: boolean;+ countryOk: boolean;+ hasUpdates: boolean;+ } {+ const updates = Array.from(this.buffer.values());+ const geoResults = [...this.geoQueue];++ // Calculate unique domains and rules for logging+ const domains = new Set<string>();+ const rules = new Set<string>();++ for (const update of updates) {+ if (update.domain) domains.add(update.domain);+ const initialRule =+ update.chains.length > 0+ ? update.chains[update.chains.length - 1]+ : "DIRECT";+ rules.add(initialRule);+ }++ let trafficOk = true;+ let countryOk = true;++ if (updates.length > 0) {+ try {+ db.batchUpdateTrafficStats(backendId, updates);+ } catch (err) {+ trafficOk = false;+ console.error(`[SurgeCollector:${backendId}] Batch write failed:`, err);+ }+ }++ if (trafficOk) {+ this.buffer.clear();+ }++ if (geoResults.length > 0) {+ try {+ const countryUpdates = geoResults+ .filter(+ (r): r is GeoIPResult & { geo: NonNullable<GeoIPResult["geo"]> } =>+ r.geo !== null+ )+ .map((r) => ({+ country: r.geo.country,+ countryName: r.geo.country_name,+ continent: r.geo.continent,+ upload: r.upload,+ download: r.download,+ timestampMs: r.timestampMs,+ }));+ db.batchUpdateCountryStats(backendId, countryUpdates);+ } catch (err) {+ countryOk = false;+ console.error(+ `[SurgeCollector:${backendId}] Country batch write failed:`,+ err+ );+ }+ }++ if (countryOk) {+ this.geoQueue = [];+ }++ return {+ domains: domains.size,+ rules: rules.size,+ trafficOk,+ countryOk,+ hasUpdates: updates.length > 0 || geoResults.length > 0,+ };+ }++ shouldLog(): boolean {+ const now = Date.now();+ if (now - this.lastLogTime > 10000) {+ this.lastLogTime = now;+ return true;+ }+ return false;+ }++ incrementLogCounter(): number {+ return ++this.logCounter;+ }+}++// Track request state with their accumulated traffic+interface TrackedRequest {+ id: string;+ domain: string;+ ip: string;+ chains: string[];+ rule: string;+ rulePayload: string;+ lastUpload: number;+ lastDownload: number;+ totalUpload: number;+ totalDownload: number;+ sourceIP?: string;+ firstSeen: number;+ lastSeen: number;+ completed: boolean;+ disconnected: boolean;+ lastStatus?: string;+ // Track if initial values have been processed+ initialProcessed: boolean;+}++// Recently completed request IDs to prevent double counting+// Surge may return completed requests in subsequent /v1/requests/recent calls+interface CompletedRequestInfo {+ id: string;+ finalUpload: number;+ finalDownload: number;+ completedAt: number;+}++/**+ * Extract policy decision path from Surge notes+ * Format: "Policy decision path: rule -> group1 -> group2 -> finalProxy"+ * Returns array: [finalProxy, ..., group2, group1, rule] (reversed for Clash format)+ */+function extractPolicyPathFromNotes(notes?: string[]): string[] {+ if (!notes || notes.length === 0) return [];++ for (const note of notes) {+ const match = note.match(/\[Rule\] Policy decision path: (.+)/);+ if (match) {+ const path = match[1].split(" -> ").map(s => s.trim()).filter(Boolean);+ if (path.length >= 2) {+ // Reverse to match Clash format: [finalProxy, ..., group2, group1, rule]+ return path.reverse();+ }+ }+ }+ return [];+}++/**+ * Convert Surge request to chains array+ * Surge: policyName is the final proxy+ * From notes: "rule -> group1 -> group2 -> finalProxy"+ * Clash format: chains[0] = first proxy, chains[last] = rule name+ */+function convertSurgeChains(+ policyName: string,+ originalPolicyName: string,+ notes?: string[]+): string[] {+ // Try to extract full path from notes+ const pathFromNotes = extractPolicyPathFromNotes(notes);+ if (pathFromNotes.length >= 2) {+ return pathFromNotes;+ }++ // Fallback: use policyName and originalPolicyName+ const chains: string[] = [];+ if (policyName) {+ chains.push(policyName);+ }+ if (originalPolicyName && originalPolicyName !== policyName) {+ chains.push(originalPolicyName);+ }+ if (chains.length === 0) {+ chains.push("DIRECT");+ }+ return chains;+}++export function createSurgeCollector(+ db: StatsDatabase,+ url: string,+ token?: string,+ geoService?: GeoIPService,+ onTrafficUpdate?: () => void,+ backendId?: number+) {+ const id = backendId || 0;+ const activeRequests = new Map<string, TrackedRequest>();+ const batchBuffer = new BatchBuffer();++ // Track recently completed requests to prevent double counting+ // Key: request ID, Value: completion timestamp+ const recentlyCompleted = new Map<string, CompletedRequestInfo>();+ const COMPLETED_REQUEST_TTL = 5 * 60 * 1000; // Keep for 5 minutes++ let lastBroadcastTime = 0;+ const broadcastThrottleMs = 500;+ let flushInterval: NodeJS.Timeout | null = null;+ let cleanupInterval: NodeJS.Timeout | null = null;+ const FLUSH_INTERVAL_MS = parseInt(process.env.FLUSH_INTERVAL_MS || "30000");+ const FLUSH_MAX_BUFFER_SIZE = parseInt(+ process.env.FLUSH_MAX_BUFFER_SIZE || "5000"+ );+ let isFlushing = false;+ let newConnectionsCount = 0;+ let completedConnectionsCount = 0;++ // Clean up stale connections that haven't been updated for a while+ const cleanupStaleConnections = () => {+ const now = Date.now();+ let cleaned = 0;+ for (const [reqId, req] of activeRequests) {+ if (now - req.lastSeen > STALE_CONNECTION_TIMEOUT) {+ activeRequests.delete(reqId);+ cleaned++;+ }+ }++ // Also clean up old completed request records+ let completedCleaned = 0;+ for (const [reqId, info] of recentlyCompleted) {+ if (now - info.completedAt > COMPLETED_REQUEST_TTL) {+ recentlyCompleted.delete(reqId);+ completedCleaned++;+ }+ }++ if ((cleaned > 0 || completedCleaned > 0) && DEBUG_SURGE) {+ console.log(`[SurgeCollector:${id}] Cleaned up ${cleaned} stale connections, ${completedCleaned} completed records`);+ }+ };++ const flushBatch = () => {+ if (isFlushing || !batchBuffer.hasPending()) {+ return;+ }++ isFlushing = true;+ try {+ const stats = batchBuffer.flush(db, geoService, id);++ if (stats.trafficOk) {+ realtimeStore.clearTraffic(id);+ }+ if (stats.countryOk) {+ realtimeStore.clearCountries(id);+ }++ if (batchBuffer.shouldLog() && (stats.domains > 0 || stats.rules > 0)) {+ console.log(+ `[SurgeCollector:${id}] Active: ${activeRequests.size}, New: ${newConnectionsCount}, Completed: ${completedConnectionsCount}, Domains: ${stats.domains}, Rules: ${stats.rules}`+ );+ // Reset counters+ newConnectionsCount = 0;+ completedConnectionsCount = 0;+ }+ } finally {+ isFlushing = false;+ }+ };++ flushInterval = setInterval(() => {+ flushBatch();+ }, FLUSH_INTERVAL_MS);++ // Start cleanup interval for stale connections+ cleanupInterval = setInterval(() => {+ cleanupStaleConnections();+ }, CLEANUP_INTERVAL);++ const collector = new SurgeCollector(id, {+ url,+ token,+ onData: (data) => {+ if (!data) {+ console.warn(`[SurgeCollector:${id}] Received null/undefined data`);+ return;+ }++ if (!data.requests) {+ return;+ }++ if (!Array.isArray(data.requests)) {+ console.warn(+ `[SurgeCollector:${id}] Invalid requests format: ${typeof data.requests}`+ );+ return;+ }++ const now = Date.now();+ const currentIds = new Set(+ data.requests.map((r) => r?.id).filter(Boolean)+ );+ let hasNewTraffic = false;+ const geoBatchByIp = new Map<+ string,+ { upload: number; download: number; connections: number }+ >();++ // Track connections that are completing in this poll+ const completingRequests = new Set<string>();++ for (const req of data.requests) {+ if (!req || typeof req !== "object") {+ continue;+ }++ if (!req.id) {+ continue;+ }++ const remoteHost = req.remoteHost || "";+ // remoteAddress format: "1.2.3.4 (Proxy)" or "1.2.3.4"+ //+ // IMPORTANT: Surge's DNS resolution happens at the proxy server, not locally.+ // When using a proxy:+ // - remoteHost = target domain+ // - remoteAddress = proxy node's IP (NOT the target's resolved IP)+ //+ // Surge does NOT provide the real landing IP in its API, because:+ // 1. Surge sends the domain to the proxy server+ // 2. Proxy server performs DNS resolution+ // 3. Surge only knows the proxy node's IP+ //+ // Therefore, for proxy traffic, we can only show the proxy node IP, not the real landing IP.+ const remoteAddress = req.remoteAddress+ ? req.remoteAddress.split(" ")[0].trim()+ : "";+ const hostWithoutPort = extractHost(remoteHost);+ const domain = isDomain(remoteHost) ? hostWithoutPort : "";+ // Use remoteHost if it's an IP (direct connection), otherwise use remoteAddress (proxy node IP)+ const ip = isIP(remoteHost)+ ? hostWithoutPort+ : isIP(remoteAddress)+ ? remoteAddress+ : "";+ const sourceIP = req.localAddress || "";++ // For Surge:+ // - policyName = final proxy (e.g., "๐บ๐ธ US-SJC-IEPL")+ // - originalPolicyName = usually same as policyName+ // - req.rule = rule type (e.g., "FINAL", "RULE-SET")+ // - notes contains full path: "rule -> group1 -> group2 -> finalProxy"+ //+ // We extract the full policy path from notes to get:+ // chains = [finalProxy, ..., group2, group1, rule]+ // rule = first element of path (the actual rule name like "๐ ๆผ็ฝไน้ฑผ")+ const chains = convertSurgeChains(+ req.policyName,+ req.originalPolicyName,+ req.notes+ );++ // rule = last element of chains (the rule name like "๐ ๆผ็ฝไน้ฑผ")+ // rulePayload = req.rule for reference (like "FINAL")+ const rule = chains.length > 0 ? chains[chains.length - 1] : (req.originalPolicyName || 'Match');+ const rulePayload = req.rule || '';++ const isCompleted = req.completed === true;+ const isDisconnected = req.disconnected === true;+ const isFailed = req.failed === true;++ // Check if this request was recently completed (prevent double counting)+ const completedInfo = recentlyCompleted.get(req.id);+ if (completedInfo) {+ // This request was already processed and completed+ // Only process if it has new traffic (shouldn't happen for completed requests)+ const currentUpload = req.outBytes || 0;+ const currentDownload = req.inBytes || 0;++ if (currentUpload <= completedInfo.finalUpload && currentDownload <= completedInfo.finalDownload) {+ // No new traffic, skip this request entirely+ if (DEBUG_SURGE) {+ console.log(`[SurgeCollector:${id}] Skipping completed request: ${req.id}`);+ }+ continue;+ }+ // If somehow there's new traffic, remove from completed and re-process+ recentlyCompleted.delete(req.id);+ }++ const existing = activeRequests.get(req.id);+ const currentUpload = req.outBytes || 0;+ const currentDownload = req.inBytes || 0;++ if (!existing) {+ // New connection - record initial state AND initial traffic+ // IMPORTANT: We now record initial traffic immediately to prevent data loss+ // for short-lived connections that disappear before the next poll.+ // The recentlyCompleted map prevents double counting if the same request reappears.+ activeRequests.set(req.id, {+ id: req.id,+ domain,+ ip,+ chains,+ rule,+ rulePayload,+ lastUpload: currentUpload,+ lastDownload: currentDownload,+ totalUpload: currentUpload, // Record initial traffic+ totalDownload: currentDownload,+ sourceIP,+ firstSeen: now,+ lastSeen: now,+ completed: isCompleted,+ disconnected: isDisconnected,+ lastStatus: req.status,+ initialProcessed: true, // Mark as processed+ });++ newConnectionsCount++;++ if (DEBUG_SURGE) {+ console.log(`[SurgeCollector:${id}] New connection: ${req.id}, domain: ${domain || 'N/A'}, ip: ${ip || 'N/A'}, initial: โ${currentUpload} โ${currentDownload}`);+ }++ // Record initial traffic if non-zero+ if (currentUpload > 0 || currentDownload > 0) {+ batchBuffer.add(id, {+ domain,+ ip,+ chain: chains[0] || "DIRECT",+ chains,+ rule: rule || "Match",+ rulePayload: rulePayload || "",+ upload: currentUpload,+ download: currentDownload,+ sourceIP,+ timestampMs: req.time || now,+ });+ realtimeStore.recordTraffic(+ id,+ {+ domain,+ ip,+ sourceIP,+ chains,+ rule: rule || "Match",+ rulePayload: rulePayload || "",+ upload: currentUpload,+ download: currentDownload,+ },+ 1,+ now+ );++ // Queue GeoIP lookup for initial traffic+ if (geoService && ip) {+ const existingGeo = geoBatchByIp.get(ip) || {+ upload: 0,+ download: 0,+ connections: 0,+ };+ existingGeo.upload += currentUpload;+ existingGeo.download += currentDownload;+ existingGeo.connections += 1;+ geoBatchByIp.set(ip, existingGeo);+ }++ hasNewTraffic = true;+ }+ } else {+ // Existing connection - calculate delta+ let uploadDelta = 0;+ let downloadDelta = 0;++ // Detect counter reset (connection restart/reuse)+ if (currentUpload < existing.lastUpload || currentDownload < existing.lastDownload) {+ // Counter was reset - treat current value as new traffic+ uploadDelta = currentUpload;+ downloadDelta = currentDownload;+ if (DEBUG_SURGE) {+ console.log(`[SurgeCollector:${id}] Counter reset detected: ${req.id}`);+ }+ } else {+ // Normal delta calculation+ uploadDelta = currentUpload - existing.lastUpload;+ downloadDelta = currentDownload - existing.lastDownload;+ }++ // Update tracking state+ existing.lastUpload = currentUpload;+ existing.lastDownload = currentDownload;+ existing.lastSeen = now;+ existing.lastStatus = req.status;+ existing.initialProcessed = true;++ // Check if connection is completing+ const wasCompleted = existing.completed;+ if ((isCompleted || isDisconnected || isFailed) && !existing.completed) {+ existing.completed = true;+ completingRequests.add(req.id);+ completedConnectionsCount++;++ if (DEBUG_SURGE) {+ console.log(`[SurgeCollector:${id}] Connection completing: ${req.id}, total: โ${existing.totalUpload + uploadDelta} โ${existing.totalDownload + downloadDelta}`);+ }+ }++ // Only record if there's actual new traffic+ if (uploadDelta > 0 || downloadDelta > 0) {+ existing.totalUpload += uploadDelta;+ existing.totalDownload += downloadDelta;++ batchBuffer.add(id, {+ domain: existing.domain,+ ip: existing.ip,+ chain: existing.chains[0] || "DIRECT",+ chains: existing.chains,+ rule: existing.rule || "Match",+ rulePayload: existing.rulePayload || "",+ upload: uploadDelta,+ download: downloadDelta,+ sourceIP: existing.sourceIP,+ timestampMs: req.time || now,+ });+ realtimeStore.recordTraffic(+ id,+ {+ domain: existing.domain,+ ip: existing.ip,+ sourceIP: existing.sourceIP,+ chains: existing.chains,+ rule: existing.rule || "Match",+ rulePayload: existing.rulePayload || "",+ upload: uploadDelta,+ download: downloadDelta,+ },+ 1,+ now+ );++ if (geoService && existing.ip) {+ const existingGeo = geoBatchByIp.get(existing.ip) || {+ upload: 0,+ download: 0,+ connections: 0,+ };+ existingGeo.upload += uploadDelta;+ existingGeo.download += downloadDelta;+ existingGeo.connections += 1;+ geoBatchByIp.set(existing.ip, existingGeo);+ }++ hasNewTraffic = true;+ }++ // If connection just completed, record it to prevent double counting+ if (existing.completed && !wasCompleted) {+ recentlyCompleted.set(req.id, {+ id: req.id,+ finalUpload: existing.totalUpload,+ finalDownload: existing.totalDownload,+ completedAt: now,+ });+ }+ }+ }++ // Clean up completed/disappeared connections+ for (const [reqId, trackedReq] of activeRequests) {+ if (!currentIds.has(reqId)) {+ // Connection disappeared from Surge's recent list+ if (DEBUG_SURGE) {+ console.log(`[SurgeCollector:${id}] Connection disappeared: ${reqId}, lasted ${Date.now() - trackedReq.firstSeen}ms, total: โ${trackedReq.totalUpload} โ${trackedReq.totalDownload}`);+ }++ // Record completed connection to prevent double counting if it reappears+ if (!recentlyCompleted.has(reqId)) {+ recentlyCompleted.set(reqId, {+ id: reqId,+ finalUpload: trackedReq.totalUpload,+ finalDownload: trackedReq.totalDownload,+ completedAt: now,+ });+ }++ activeRequests.delete(reqId);+ }+ }++ if (geoService && geoBatchByIp.size > 0) {+ for (const [ip, traffic] of geoBatchByIp) {+ geoService+ .getGeoLocation(ip)+ .then((geo) => {+ if (geo) {+ batchBuffer.addGeoResult({+ ip,+ geo,+ upload: traffic.upload,+ download: traffic.download,+ timestampMs: now,+ });+ realtimeStore.recordCountryTraffic(+ id,+ geo,+ traffic.upload,+ traffic.download,+ traffic.connections,+ now+ );+ }+ })+ .catch(() => {+ // Silently fail for GeoIP errors+ });+ }+ }++ if (batchBuffer.size() >= FLUSH_MAX_BUFFER_SIZE) {+ flushBatch();+ }++ if (+ hasNewTraffic &&+ onTrafficUpdate &&+ now - lastBroadcastTime > broadcastThrottleMs+ ) {+ lastBroadcastTime = now;+ onTrafficUpdate();+ }+ },+ onError: (err) => {+ console.error(`[SurgeCollector:${id}] Error:`, err);+ },+ });++ const originalStop = collector.stop.bind(collector);+ collector.stop = () => {+ if (flushInterval) {+ clearInterval(flushInterval);+ flushInterval = null;+ flushBatch();+ }+ if (cleanupInterval) {+ clearInterval(cleanupInterval);+ cleanupInterval = null;+ }+ originalStop();+ };++ // Add debug stats method+ (collector as any).getDebugStats = () => {+ const now = Date.now();+ const connections = Array.from(activeRequests.values()).map(r => ({+ id: r.id,+ domain: r.domain,+ ip: r.ip,+ duration: now - r.firstSeen,+ lastActivity: now - r.lastSeen,+ totalUpload: r.totalUpload,+ totalDownload: r.totalDownload,+ completed: r.completed,+ status: r.lastStatus,+ }));++ return {+ activeConnections: activeRequests.size,+ newConnectionsThisPoll: newConnectionsCount,+ completedConnectionsThisPoll: completedConnectionsCount,+ bufferSize: batchBuffer.size(),+ connections: DEBUG_SURGE ? connections : undefined,+ };+ };++ return collector;+}++/**+ * Extract host without port from a host:port string+ */+function extractHost(hostWithPort: string): string {+ if (!hostWithPort) return "";+ // Handle IPv6 addresses like [2001:db8::1]:443+ if (hostWithPort.startsWith("[")) {+ const closingBracket = hostWithPort.indexOf("]");+ if (closingBracket !== -1) {+ return hostWithPort.slice(1, closingBracket);+ }+ }+ // Handle IPv4:port or domain:port+ const lastColon = hostWithPort.lastIndexOf(":");+ if (lastColon !== -1) {+ // Check if it's an IPv6 without brackets (multiple colons)+ // IPv6 has multiple colons, while IPv4:port or domain:port has only one+ const colonCount = hostWithPort.split(":").length - 1;+ if (colonCount > 1) {+ // Multiple colons, likely IPv6 without port+ return hostWithPort;+ }+ return hostWithPort.slice(0, lastColon);+ }+ return hostWithPort;+}++function isDomain(host: string): boolean {+ if (!host) return false;+ // Remove port if present+ const hostWithoutPort = extractHost(host);+ if (isIP(hostWithoutPort)) return false;+ return /^[a-zA-Z0-9]([a-zA-Z0-9\-]{0,61}[a-zA-Z0-9])?(\.[a-zA-Z0-9]([a-zA-Z0-9\-]{0,61}[a-zA-Z0-9])?)*$/.test(+ hostWithoutPort+ );+}++function isIP(host: string): boolean {+ if (!host) return false;+ // Remove port if present+ const hostWithoutPort = extractHost(host);+ return isIPv4(hostWithoutPort) || isIPv6(hostWithoutPort);+}diff --git a/apps/collector/src/utils.ts b/apps/collector/src/utils.tsnew file mode 100644index 0000000..9a21cd9--- /dev/null+++ b/apps/collector/src/utils.ts@@ -0,0 +1,11 @@+/**+ * ๆๆฐ้้ฟ่ฎก็ฎๅจ+ */+export function calculateBackoffDelay(+ attempt: number,+ baseDelay: number,+ maxDelay: number+): number {+ const delay = baseDelay * Math.pow(2, attempt);+ return Math.min(delay, maxDelay);+}diff --git a/apps/web/app/[locale]/dashboard/components/header/index.tsx b/apps/web/app/[locale]/dashboard/components/header/index.tsxindex d1e2c0d..a8bfce2 100644--- a/apps/web/app/[locale]/dashboard/components/header/index.tsx+++ b/apps/web/app/[locale]/dashboard/components/header/index.tsx@@ -44,11 +44,19 @@ import { cn } from "@/lib/utils";import type { TimeRange } from "@/lib/api";import type { BackendStatus, TimePreset } from "@/lib/types/dashboard";+interface BackendHealth {+ status: 'healthy' | 'unhealthy' | 'unknown';+ lastChecked: number;+ message?: string;+ latency?: number;+}+interface Backend {id: number;name: string;is_active: boolean;listening: boolean;+ health?: BackendHealth;}import { useAuth } from "@/lib/auth";@@ -147,7 +155,7 @@ export function Header({</div>{/* Backend Selector */}- {backends.length > 0 && (+ {listeningBackends.length > 0 && (<DropdownMenu><DropdownMenuTrigger asChild><Button@@ -165,41 +173,60 @@ export function Header({<DropdownMenuContent align="start" className="w-56"><DropdownMenuLabel>{backendT("backendsTab")}</DropdownMenuLabel><DropdownMenuSeparator />- {backends.map((backend) => (- <DropdownMenuItem- key={backend.id}- onClick={() => onSwitchBackend(backend.id)}- className="flex items-center justify-between"- >- <span+ {listeningBackends.map((backend) => {+ const healthStatus = backend.health?.status;+ const isUnhealthy = healthStatus === 'unhealthy';++ return (+ <DropdownMenuItem+ key={backend.id}+ onClick={() => onSwitchBackend(backend.id)}className={cn(- "truncate",- backend.is_active && "font-medium"+ "flex items-center justify-between",+ isUnhealthy && "text-red-600")}>- {backend.name}- </span>- <div className="flex items-center gap-1">- {!!backend.is_active && (- <Badge- variant="default"- className="text-[10px] h-5"- >- {backendT("displaying")}- </Badge>- )}- {!!backend.listening && !backend.is_active && (- <Badge- variant="secondary"- className="text-[10px] h-5 gap-1"+ <div className="flex items-center gap-2">+ <span+ className={cn(+ "truncate",+ backend.is_active && "font-medium"+ )}>- <Radio className="w-2 h-2" />- {backendT("collecting")}- </Badge>- )}- </div>- </DropdownMenuItem>- ))}+ {backend.name}+ </span>+ {isUnhealthy && (+ <AlertTriangle className="w-3 h-3 text-red-500" />+ )}+ </div>+ <div className="flex items-center gap-1">+ {!!backend.is_active && (+ <Badge+ variant="default"+ className={cn(+ "text-[10px] h-5",+ isUnhealthy && "bg-red-500 hover:bg-red-600"+ )}+ >+ {isUnhealthy ? backendT("unhealthy") : backendT("displaying")}+ </Badge>+ )}+ {!!backend.listening && !backend.is_active && (+ <Badge+ variant="secondary"+ className={cn(+ "text-[10px] h-5 gap-1",+ isUnhealthy && "bg-red-100 text-red-600 border-red-200"+ )}+ >+ <Radio className={cn("w-2 h-2", isUnhealthy && "text-red-500")} />+ {isUnhealthy ? backendT("unhealthy") : backendT("collecting")}+ </Badge>+ )}+ </div>+ </DropdownMenuItem>+ );+ })}<DropdownMenuSeparator /><DropdownMenuItem onClick={onOpenBackendDialog}><Settings className="w-4 h-4 mr-2" />@@ -212,16 +239,55 @@ export function Header({{/* Listening Indicators */}{listeningBackends.length > 0 && (<div className="hidden md:flex items-center gap-1">- {listeningBackends.slice(0, 3).map((backend) => (- <Badge- key={backend.id}- variant="outline"- className="text-[10px] h-5 gap-1 px-1.5 border-green-500/30 text-green-600"- >- <Radio className="w-2 h-2" />- {backend.name}- </Badge>- ))}+ {listeningBackends.slice(0, 3).map((backend) => {+ // Determine color based on health status+ const healthStatus = backend.health?.status;+ const badgeClasses = {+ healthy: "border-green-500/30 text-green-600 bg-green-50/50",+ unhealthy: "border-red-500/30 text-red-600 bg-red-50/50",+ unknown: "border-gray-500/30 text-gray-500 bg-gray-50/50",+ };+ const iconClasses = {+ healthy: "text-green-500",+ unhealthy: "text-red-500",+ unknown: "text-gray-400",+ };+ const status = healthStatus || 'unknown';+ const tooltipText = backend.health?.message ||+ (status === 'healthy' ? 'Connected' : status === 'unhealthy' ? 'Connection failed' : 'Checking...');++ return (+ <TooltipProvider key={backend.id} delayDuration={100}>+ <Tooltip>+ <TooltipTrigger asChild>+ <Badge+ variant="outline"+ className={cn(+ "text-[10px] h-5 gap-1 px-1.5 cursor-pointer",+ badgeClasses[status]+ )}+ >+ <Radio className={cn("w-2 h-2", iconClasses[status])} />+ {backend.name}+ </Badge>+ </TooltipTrigger>+ <TooltipContent side="bottom" className="text-xs">+ <div className="flex flex-col gap-0.5">+ <span className="font-medium">{backend.name}</span>+ <span className={cn(+ "text-[10px]",+ status === 'healthy' ? "text-green-500" :+ status === 'unhealthy' ? "text-red-500" : "text-gray-400"+ )}>+ {tooltipText}+ {backend.health?.latency && ` (${backend.health.latency}ms)`}+ </span>+ </div>+ </TooltipContent>+ </Tooltip>+ </TooltipProvider>+ );+ })}{listeningBackends.length > 3 && (<Badge variant="outline" className="text-[10px] h-5 px-1.5">+{listeningBackends.length - 3}diff --git a/apps/web/app/[locale]/dashboard/hooks/use-dashboard.ts b/apps/web/app/[locale]/dashboard/hooks/use-dashboard.tsindex d90cf43..2ed2f60 100644--- a/apps/web/app/[locale]/dashboard/hooks/use-dashboard.ts+++ b/apps/web/app/[locale]/dashboard/hooks/use-dashboard.ts@@ -23,6 +23,7 @@ import {import { useStableTimeRange } from "@/lib/hooks/use-stable-time-range";import { useStatsWebSocket } from "@/lib/websocket";import { useRequireAuth } from "@/lib/auth";+import { toast } from "sonner";import type {StatsSummary,CountryStats,@@ -279,11 +280,23 @@ export function useDashboard(): UseDashboardReturn {await api.setActiveBackend(backendId);await backendsQuery.refetch();await refreshNow(true);++ // Check backend health after switching+ const switchedBackend = backends.find(b => b.id === backendId);+ if (switchedBackend?.health?.status === 'unhealthy') {+ toast.warning(+ dashboardT("backendUnhealthyTitle"),+ {+ description: switchedBackend.health.message || dashboardT("backendUnhealthyDesc"),+ duration: 5000,+ }+ );+ }} catch (err) {console.error("Failed to switch backend:", err);}},- [backendsQuery.refetch, refreshNow]+ [backendsQuery.refetch, refreshNow, backends, backendT]);const handleBackendChange = useCallback(async () => {diff --git a/apps/web/components/features/backend/backend-config-dialog.tsx b/apps/web/components/features/backend/backend-config-dialog.tsxindex 41e9206..88fbb77 100644--- a/apps/web/components/features/backend/backend-config-dialog.tsx+++ b/apps/web/components/features/backend/backend-config-dialog.tsx@@ -31,6 +31,12 @@ import { Button } from "@/components/ui/button";import { Input } from "@/components/ui/input";import { Switch } from "@/components/ui/switch";import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card";+import {+ Tooltip,+ TooltipContent,+ TooltipProvider,+ TooltipTrigger,+} from "@/components/ui/tooltip";import {AlertDialog,AlertDialogAction,@@ -53,6 +59,7 @@ import { cn, formatBytes, formatNumber } from "@/lib/utils";import { api } from "@/lib/api";import { toast } from "sonner";import { BackendVerifyAnimation } from "@/components/features/backend/backend-verify-animation";+import { BackendListSkeleton } from "@/components/ui/insight-skeleton";import { useSettings, FaviconProvider, getFaviconUrl } from "@/lib/settings";import { Label } from "@/components/ui/label";import { RadioGroup, RadioGroupItem } from "@/components/ui/radio-group";@@ -235,6 +242,13 @@ function FaviconProviderPreview({);}+interface BackendHealth {+ status: 'healthy' | 'unhealthy' | 'unknown';+ lastChecked: number;+ message?: string;+ latency?: number;+}+interface Backend {id: number;name: string;@@ -242,10 +256,12 @@ interface Backend {host: string;port: number;token: string;+ type?: 'clash' | 'surge';enabled: boolean;is_active: boolean;listening: boolean;hasToken?: boolean;+ health?: BackendHealth;created_at: string;}@@ -309,13 +325,10 @@ export function BackendConfigDialog({const commonT = useTranslations("common");const [backends, setBackends] = useState<Backend[]>([]);+ const [backendsLoading, setBackendsLoading] = useState(false);const [loading, setLoading] = useState(false);const [editingId, setEditingId] = useState<number | null>(null);const [testingId, setTestingId] = useState<number | null>(null);- const [testResult, setTestResult] = useState<{- success: boolean;- message: string;- } | null>(null);const [activeTab, setActiveTab] = useState<"backends" | "database" | "preferences" | "security">("backends");@@ -380,6 +393,7 @@ export function BackendConfigDialog({name: string;url: string;token: string;+ type: 'clash' | 'surge';} | null>(null);const [formData, setFormData] = useState({@@ -388,6 +402,7 @@ export function BackendConfigDialog({port: "9090",ssl: false,token: "",+ type: "clash" as 'clash' | 'surge',});const [editFormData, setEditFormData] = useState({@@ -396,6 +411,7 @@ export function BackendConfigDialog({port: "9090",ssl: false,token: "",+ type: "clash" as 'clash' | 'surge',});const [showAddForm, setShowAddForm] = useState(false);@@ -409,16 +425,19 @@ export function BackendConfigDialog({}, [open]);const loadBackends = async () => {+ setBackendsLoading(true);try {const data = await api.getBackends();// Parse URL to host/port for displayconst parsedData: Backend[] = data.map((b) => {const parsed = parseUrl(b.url);- return { ...b, host: parsed.host, port: parseInt(parsed.port) || 9090 };+ return { ...b, type: b.type || 'clash', host: parsed.host, port: parseInt(parsed.port) || 9090 };});setBackends(parsedData);} catch (error) {console.error("Failed to load backends:", error);+ } finally {+ setBackendsLoading(false);}};@@ -513,14 +532,14 @@ export function BackendConfigDialog({const url = buildUrl(formData.host, formData.port, formData.ssl);// Show verification animation immediately- setPendingBackend({ name: formData.name, url, token: formData.token });+ setPendingBackend({ name: formData.name, url, token: formData.token, type: formData.type });setVerifyPhase("pending");setVerifyMessage("");setShowVerifyAnimation(true);// Perform verificationtry {- const testResult = await api.testBackend(url, formData.token);+ const testResult = await api.testBackend(url, formData.token, formData.type);if (testResult.success) {setVerifyPhase("success");@@ -546,6 +565,7 @@ export function BackendConfigDialog({name: pendingBackend.name,url: pendingBackend.url,token: pendingBackend.token,+ type: pendingBackend.type,});setFormData({@@ -554,6 +574,7 @@ export function BackendConfigDialog({port: "9090",ssl: false,token: "",+ type: "clash",});setShowAddForm(false);setShowVerifyAnimation(false);@@ -563,11 +584,7 @@ export function BackendConfigDialog({// Show success message for first backendif (result.isActive) {- setTestResult({- success: true,- message: t("firstBackendAutoActive"),- });- setTimeout(() => setTestResult(null), 3000);+ toast.success(t("firstBackendAutoActive"));}if (isFirstTime && onConfigComplete) {@@ -599,6 +616,7 @@ export function BackendConfigDialog({name: editFormData.name,url,token: editFormData.token || undefined,+ type: editFormData.type,});setEditingId(null);setEditFormData({@@ -607,6 +625,7 @@ export function BackendConfigDialog({port: "9090",ssl: false,token: "",+ type: "clash",});await loadBackends();await onBackendChange?.();@@ -701,15 +720,15 @@ export function BackendConfigDialog({const handleTest = async (backend: Backend) => {setTestingId(backend.id);- setTestResult(null);try {const result = await api.testBackendById(backend.id);- setTestResult(result);+ if (result.success) {+ toast.success(result.message);+ } else {+ toast.error(result.message);+ }} catch (error: any) {- setTestResult({- success: false,- message: error.message || "Test failed",- });+ toast.error(error.message || "Test failed");} finally {setTestingId(null);}@@ -834,6 +853,7 @@ export function BackendConfigDialog({port: String(backend.port || 9090),ssl: backend.url.startsWith("https"),token: "",+ type: backend.type || "clash",});setShowAddForm(false);};@@ -846,6 +866,7 @@ export function BackendConfigDialog({port: "9090",ssl: false,token: "",+ type: "clash",});};@@ -905,6 +926,17 @@ export function BackendConfigDialog({{activeTab === "backends" ? (// Backends Tab<div className="space-y-3">+ {backendsLoading ? (+ <BackendListSkeleton count={3} />+ ) : backends.length === 0 && !showAddForm ? (+ // Empty state+ <div className="text-center py-8 text-muted-foreground">+ <Server className="w-12 h-12 mx-auto mb-3 opacity-50" />+ <p className="text-sm">{t("noBackends")}</p>+ <p className="text-xs mt-1 opacity-70">{t("addBackendHint")}</p>+ </div>+ ) : (+ <>{backends.map((backend) => (<divkey={backend.id}@@ -935,6 +967,23 @@ export function BackendConfigDialog({className="h-9 mt-1"/></div>+ <div>+ <label className="text-xs font-medium">+ {t("type")}+ </label>+ <select+ value={editFormData.type}+ onChange={(e) =>+ setEditFormData({ ...editFormData, type: e.target.value as 'clash' | 'surge' })+ }+ className="h-9 mt-1 w-full rounded-md border border-input bg-background px-3 py-1 text-sm shadow-sm transition-colors focus-visible:outline-none focus-visible:ring-1 focus-visible:ring-ring"+ >+ <option value="clash">Clash / Mihomo</option>+ <option value="surge">Surge</option>+ </select>+ </div>+ </div>+ <div className="grid grid-cols-2 gap-3"><div><label className="text-xs font-medium">{t("host")}@@ -951,8 +1000,6 @@ export function BackendConfigDialog({className="h-9 mt-1"/></div>- </div>- <div className="grid grid-cols-3 gap-3"><div><label className="text-xs font-medium">{t("port")}@@ -969,18 +1016,18 @@ export function BackendConfigDialog({className="h-9 mt-1"/></div>- <div className="col-span-2 flex items-center gap-2 pt-5">- <Switch- checked={editFormData.ssl}- onCheckedChange={(checked) =>- setEditFormData({- ...editFormData,- ssl: checked,- })- }- />- <label className="text-sm">{t("useSsl")}</label>- </div>+ </div>+ <div className="flex items-center gap-2">+ <Switch+ checked={editFormData.ssl}+ onCheckedChange={(checked) =>+ setEditFormData({+ ...editFormData,+ ssl: checked,+ })+ }+ />+ <label className="text-sm">{t("useSsl")}</label></div><div><label className="text-xs font-medium">@@ -1029,6 +1076,17 @@ export function BackendConfigDialog({<span className="font-medium text-base">{backend.name}</span>+ {/* Backend Type Icon */}+ <div+ className="w-4 h-4 rounded-sm bg-white/90 flex items-center justify-center p-0.5"+ title={backend.type === 'surge' ? 'Surge' : 'Clash / Mihomo'}+ >+ <img+ src={backend.type === 'surge' ? '/icons/icon-surge.png' : '/icons/icon-clash.png'}+ alt={backend.type === 'surge' ? 'Surge' : 'Clash'}+ className="w-full h-full object-contain"+ />+ </div>{!backend.enabled && (<span className="px-2 py-0.5 rounded-full bg-muted text-muted-foreground text-xs font-medium">{t("disabled")}@@ -1088,20 +1146,51 @@ export function BackendConfigDialog({/></Button>- <Button- variant="ghost"- size="icon"- className="h-8 w-8"- onClick={() => handleTest(backend)}- disabled={testingId === backend.id}- title={t("testConnection")}>- <RefreshCw- className={cn(- "w-4 h-4",- testingId === backend.id && "animate-spin",- )}- />- </Button>+ <TooltipProvider delayDuration={100}>+ <Tooltip>+ <TooltipTrigger asChild>+ <Button+ variant="ghost"+ size="icon"+ className={cn(+ "h-8 w-8",+ backend.health?.status === 'healthy' && "text-green-600 hover:text-green-700 hover:bg-green-50",+ backend.health?.status === 'unhealthy' && "text-red-500 hover:text-red-600 hover:bg-red-50",+ !backend.health && "text-gray-400"+ )}+ onClick={() => handleTest(backend)}+ disabled={testingId === backend.id}>+ <RefreshCw+ className={cn(+ "w-4 h-4",+ testingId === backend.id && "animate-spin",+ backend.health?.status === 'healthy' && !testingId && "text-green-500",+ backend.health?.status === 'unhealthy' && !testingId && "text-red-500",+ )}+ />+ </Button>+ </TooltipTrigger>+ <TooltipContent side="top" className="text-xs max-w-[200px]">+ <div className="flex flex-col gap-0.5">+ <span className="font-medium">{t("testConnection")}</span>+ {backend.health ? (+ <span className={cn(+ "text-[10px]",+ backend.health.status === 'healthy' ? "text-green-500" :+ backend.health.status === 'unhealthy' ? "text-red-500" : "text-gray-400"+ )}>+ {backend.health.message ||+ (backend.health.status === 'healthy' ? 'Healthy' :+ backend.health.status === 'unhealthy' ? 'Unhealthy' : 'Unknown')}+ {backend.health.latency && ` (${backend.health.latency}ms)`}+ </span>+ ) : (+ <span className="text-[10px] text-gray-400">Click to test connection</span>+ )}+ </div>+ </TooltipContent>+ </Tooltip>+ </TooltipProvider><Buttonvariant="ghost"@@ -1128,23 +1217,7 @@ export function BackendConfigDialog({)}</div>))}-- {/* Test Result */}- {testResult && (- <div- className={cn(- "p-3 rounded-lg flex items-center gap-2 text-sm",- testResult.success- ? "bg-green-500/10 text-green-600 border border-green-500/20"- : "bg-destructive/10 text-destructive border border-destructive/20",- )}>- {testResult.success ? (- <CheckCircle2 className="w-4 h-4 shrink-0" />- ) : (- <AlertCircle className="w-4 h-4 shrink-0" />- )}- {testResult.message}- </div>+ </>)}{/* Add New Backend */}@@ -1176,6 +1249,23 @@ export function BackendConfigDialog({className="h-9 mt-1"/></div>+ <div>+ <label className="text-xs font-medium">+ {t("type")}+ </label>+ <select+ value={formData.type}+ onChange={(e) =>+ setFormData({ ...formData, type: e.target.value as 'clash' | 'surge' })+ }+ className="h-9 mt-1 w-full rounded-md border border-input bg-background px-3 py-1 text-sm shadow-sm transition-colors focus-visible:outline-none focus-visible:ring-1 focus-visible:ring-ring"+ >+ <option value="clash">Clash / Mihomo</option>+ <option value="surge">Surge</option>+ </select>+ </div>+ </div>+ <div className="grid grid-cols-2 gap-3"><div><label className="text-xs font-medium">{t("host")} *@@ -1189,8 +1279,6 @@ export function BackendConfigDialog({className="h-9 mt-1"/></div>- </div>- <div className="grid grid-cols-3 gap-3"><div><label className="text-xs font-medium">{t("port")}@@ -1204,15 +1292,15 @@ export function BackendConfigDialog({className="h-9 mt-1"/></div>- <div className="col-span-2 flex items-center gap-2 pt-5">- <Switch- checked={formData.ssl}- onCheckedChange={(checked) =>- setFormData({ ...formData, ssl: checked })- }- />- <label className="text-sm">{t("useSsl")}</label>- </div>+ </div>+ <div className="flex items-center gap-2">+ <Switch+ checked={formData.ssl}+ onCheckedChange={(checked) =>+ setFormData({ ...formData, ssl: checked })+ }+ />+ <label className="text-sm">{t("useSsl")}</label></div><div><label className="text-xs font-medium">@@ -1224,7 +1312,7 @@ export function BackendConfigDialog({onChange={(e) =>setFormData({ ...formData, token: e.target.value })}- placeholder={t("tokenPlaceholder")}+ placeholder={formData.type === 'surge' ? t("tokenPlaceholderSurge") || "Surge API Key (x-key)" : t("tokenPlaceholder")}className="h-9 mt-1"/></div>@@ -1240,6 +1328,7 @@ export function BackendConfigDialog({port: "9090",ssl: false,token: "",+ type: "clash",});}}className="flex-shrink-0">diff --git a/apps/web/components/features/devices/interactive-device-stats.tsx b/apps/web/components/features/devices/interactive-device-stats.tsxindex 1194d9a..9cd1954 100644--- a/apps/web/components/features/devices/interactive-device-stats.tsx+++ b/apps/web/components/features/devices/interactive-device-stats.tsx@@ -225,7 +225,7 @@ export function InteractiveDeviceStats({<span className={cn("w-5 h-5 rounded-md text-[10px] font-bold flex items-center justify-center shrink-0", rankBadgeClass)}>{idx + 1}</span>- <div className="px-1.5 py-0.5 rounded-md text-[10px] font-medium text-white/90 truncate min-w-0 max-w-full flex items-center gap-1" style={{ backgroundColor: item.color }}>+ <div className="px-1.5 py-0.5 rounded-md text-[10px] font-medium text-white/90 truncate min-w-0 max-w-[140px] flex items-center gap-1" style={{ backgroundColor: item.color }}><span className="truncate">{item.name}</span></div></div>@@ -250,10 +250,10 @@ export function InteractiveDeviceStats({const badgeColor = item.rank === 0 ? "bg-amber-100 text-amber-700 dark:bg-amber-900/30 dark:text-amber-400" : item.rank === 1 ? "bg-slate-100 text-slate-700 dark:bg-slate-800 dark:text-slate-300" : item.rank === 2 ? "bg-orange-100 text-orange-700 dark:bg-orange-900/30 dark:text-orange-400" : "bg-muted text-muted-foreground";return (<button key={item.rawName} onClick={() => handleDeviceClick(item.rawName)} className={cn("w-full p-2.5 rounded-xl border text-left transition-all duration-200", isSelected ? "border-primary bg-primary/5 ring-1 ring-primary/20" : "border-border/50 bg-card/50 hover:bg-card hover:border-primary/30")}>- <div className="flex items-center gap-2 mb-1.5">+ <div className="flex items-center gap-2 mb-1.5 min-w-0"><span className={cn("w-5 h-5 rounded-md text-[10px] font-bold flex items-center justify-center shrink-0", badgeColor)}>{item.rank + 1}</span>- <span className="flex-1 text-sm font-medium truncate" title={item.name}>{item.name}</span>- <span className="text-sm font-bold tabular-nums shrink-0 whitespace-nowrap">{formatBytes(item.value)}</span>+ <span className="flex-1 text-sm font-medium truncate min-w-0 max-w-[120px] sm:max-w-[180px]" title={item.name}>{item.name}</span>+ <span className="text-sm font-bold tabular-nums shrink-0 whitespace-nowrap ml-auto">{formatBytes(item.value)}</span></div><div className="pl-7 space-y-1"><div className="h-1.5 rounded-full bg-muted overflow-hidden flex">diff --git a/apps/web/components/features/proxies/interactive-proxy-stats.tsx b/apps/web/components/features/proxies/interactive-proxy-stats.tsxindex 3f21f9a..9a0ebb1 100644--- a/apps/web/components/features/proxies/interactive-proxy-stats.tsx+++ b/apps/web/components/features/proxies/interactive-proxy-stats.tsx@@ -1,6 +1,6 @@"use client";-import { useState, useCallback, useMemo, useEffect } from "react";+import { useState, useCallback, useMemo, useEffect, useRef } from "react";import { BarChart3, Link2, Waypoints } from "lucide-react";import { useTranslations } from "next-intl";import { useQueryClient } from "@tanstack/react-query";@@ -68,6 +68,28 @@ function renderCustomBarLabel(props: any) {);}+// Hook to detect container width for responsive chart items+function useContainerWidth(ref: React.RefObject<HTMLElement | null>) {+ const [width, setWidth] = useState(0);++ useEffect(() => {+ if (!ref.current) return;++ const observer = new ResizeObserver((entries) => {+ for (const entry of entries) {+ setWidth(entry.contentRect.width);+ }+ });++ observer.observe(ref.current);+ setWidth(ref.current.getBoundingClientRect().width);++ return () => observer.disconnect();+ }, [ref]);++ return width;+}+export function InteractiveProxyStats({data,activeBackendId,@@ -96,6 +118,10 @@ export function InteractiveProxyStats({const [activeTab, setActiveTab] = useState("domains");const [detailPageSize, setDetailPageSize] = useState<PageSize>(10);const [showDomainBarLabels, setShowDomainBarLabels] = useState(true);++ // Ref for TOP DOMAINS card to detect container width+ const topDomainsCardRef = useRef<HTMLDivElement>(null);+ const topDomainsWidth = useContainerWidth(topDomainsCardRef);useEffect(() => {const media = window.matchMedia("(min-width: 640px)");@@ -191,11 +217,14 @@ export function InteractiveProxyStats({const selectedProxyData = useMemo(() => chartData.find(d => d.rawName === selectedProxy), [chartData, selectedProxy]);const domainChartData = useMemo(() => {+ // Show more items in wide container (single column layout)+ const itemCount = topDomainsWidth >= 500 ? 15 : 10;+ const maxNameLength = topDomainsWidth >= 500 ? 35 : 25;return [...proxyDomains].sort((a, b) => (b.totalDownload + b.totalUpload) - (a.totalDownload + a.totalUpload))- .slice(0, 10)+ .slice(0, itemCount).map((d, i) => ({- name: d.domain.length > 25 ? d.domain.slice(0, 22) + "..." : d.domain,+ name: d.domain.length > maxNameLength ? d.domain.slice(0, maxNameLength - 3) + "..." : d.domain,fullName: d.domain,total: d.totalDownload + d.totalUpload,download: d.totalDownload,@@ -203,7 +232,7 @@ export function InteractiveProxyStats({connections: d.totalConnections,color: COLORS[i % COLORS.length],}));- }, [proxyDomains]);+ }, [proxyDomains, topDomainsWidth]);const isBackendUnavailable = backendStatus === "unhealthy";const emptyHint = isBackendUnavailable@@ -236,9 +265,9 @@ export function InteractiveProxyStats({return (<div className="space-y-6">- <div className="grid grid-cols-1 lg:grid-cols-2 2xl:grid-cols-12 gap-6">+ <div className="grid grid-cols-1 md:grid-cols-2 xl:grid-cols-3 gap-4 lg:gap-6">{/* Pie Chart */}- <Card className="min-w-0 lg:col-span-1 2xl:col-span-3">+ <Card className="min-w-0"><CardHeader className="pb-2"><CardTitle className="text-sm font-semibold text-muted-foreground uppercase tracking-wider">{t("title")}</CardTitle></CardHeader>@@ -290,7 +319,7 @@ export function InteractiveProxyStats({</Card>{/* Proxy List */}- <Card className="min-w-0 lg:col-span-1 2xl:col-span-4">+ <Card className="min-w-0"><CardHeader className="pb-2"><CardTitle className="text-sm font-semibold text-muted-foreground uppercase tracking-wider">{t("title")}</CardTitle></CardHeader><CardContent className="p-3"><ScrollArea className="h-[280px] pr-3">@@ -332,8 +361,9 @@ export function InteractiveProxyStats({</CardContent></Card>- {/* Top Domains Chart */}- <Card className="min-w-0 lg:col-span-2 2xl:col-span-5">+ {/* Top Domains Chart - Full width on single column, adapts to container */}+ <div className="min-w-0 md:col-span-2 xl:col-span-1 @container">+ <Card className="min-w-0 h-full"><CardHeader className="pb-2"><div className="flex items-center justify-between"><CardTitle className="text-sm font-semibold text-muted-foreground uppercase tracking-wider flex items-center gap-2"><BarChart3 className="h-4 w-4" />{domainsT("title")}</CardTitle>@@ -354,11 +384,11 @@ export function InteractiveProxyStats({<p className="text-xs text-muted-foreground/80 mt-1 max-w-xs">{emptyHint}</p></div>) : (- <div className="h-[280px] w-full">+ <div className="h-[280px] @min-[500px]:h-[320px] w-full"><ResponsiveContainer width="100%" height="100%">- <BarChart data={domainChartData} layout="vertical" margin={{ left: 0, right: showDomainBarLabels ? 60 : 10, top: 5, bottom: 5 }}>+ <BarChart data={domainChartData} layout="vertical" margin={{ left: 0, right: showDomainBarLabels ? 70 : 10, top: 5, bottom: 5 }}><XAxis type="number" hide />- <YAxis type="category" dataKey="name" width={100} tick={{ fontSize: 11 }} tickLine={false} axisLine={false} />+ <YAxis type="category" dataKey="name" width={120} tick={{ fontSize: 11 }} tickLine={false} axisLine={false} /><RechartsTooltip content={({ active, payload }) => {if (active && payload && payload.length) {const item = payload[0].payload;@@ -376,6 +406,7 @@ export function InteractiveProxyStats({)}</CardContent></Card>+ </div></div>{/* Bottom: Tabs with shared table components */}diff --git a/apps/web/components/features/rules/interactive-rule-stats.tsx b/apps/web/components/features/rules/interactive-rule-stats.tsxindex c71ef51..076f535 100644--- a/apps/web/components/features/rules/interactive-rule-stats.tsx+++ b/apps/web/components/features/rules/interactive-rule-stats.tsx@@ -136,6 +136,28 @@ function renderCustomBarLabel(props: any) {);}+// Hook to detect container width for responsive chart items+function useContainerWidth(ref: React.RefObject<HTMLElement | null>) {+ const [width, setWidth] = useState(0);++ useEffect(() => {+ if (!ref.current) return;++ const observer = new ResizeObserver((entries) => {+ for (const entry of entries) {+ setWidth(entry.contentRect.width);+ }+ });++ observer.observe(ref.current);+ setWidth(ref.current.getBoundingClientRect().width);++ return () => observer.disconnect();+ }, [ref]);++ return width;+}+export function InteractiveRuleStats({data,activeBackendId,@@ -170,6 +192,11 @@ export function InteractiveRuleStats({const [ipPage, setIpPage] = useState(1);const [ipSearch, setIpSearch] = useState("");const [showDomainBarLabels, setShowDomainBarLabels] = useState(true);++ // Ref for TOP DOMAINS card to detect container width+ const topDomainsCardRef = useRef<HTMLDivElement>(null);+ const topDomainsWidth = useContainerWidth(topDomainsCardRef);+ const topDomainsItemCount = topDomainsWidth >= 500 ? 15 : 10;// Fetch Gateway rules to find zero-traffic rules.const { data: gatewayRules = null } = useGatewayRules({@@ -200,10 +227,14 @@ export function InteractiveRuleStats({const chartData = useMemo<RuleChartItem[]>(() => {if (!rulesData) return [];- const existingRuleNames = new Set(rulesData.map(r => r.rule));++ // Normalize rule names for deduplication (trim whitespace)+ const normalizeRuleName = (name: string) => name?.trim() || '';++ const existingRuleNames = new Set(rulesData.map(r => normalizeRuleName(r.rule)));const trafficItems: RuleChartItem[] = rulesData.map((rule, index) => ({- name: rule.rule,- rawName: rule.rule,+ name: normalizeRuleName(rule.rule),+ rawName: normalizeRuleName(rule.rule),value: rule.totalDownload + rule.totalUpload,download: rule.totalDownload,upload: rule.totalUpload,@@ -221,7 +252,17 @@ export function InteractiveRuleStats({if (gatewayRules?.rules) {const zeroTrafficItems: typeof trafficItems = [];for (const rule of gatewayRules.rules) {- const proxyGroup = rule.proxy;+ // Filter out internal/system rules that users typically don't configure manually or don't want to see+ // - 'GeoIP': Built-in country rules+ // - 'RuleSet': Surge's internal expansion of rulesets+ // - Empty payload: Invalid/Internal+ if (!rule.payload) continue;+ if (['GeoIP', 'RuleSet'].includes(rule.type)) continue;++ const proxyGroup = normalizeRuleName(rule.proxy);+ // Skip empty proxy names+ if (!proxyGroup) continue;+ // Skip if already exists (case-sensitive for emoji names)if (existingRuleNames.has(proxyGroup)) continue;existingRuleNames.add(proxyGroup);zeroTrafficItems.push({@@ -257,6 +298,11 @@ export function InteractiveRuleStats({return Math.max(...chartData.map(r => r.value));}, [chartData]);+ // Compute set of visible rule names for filtering the graph+ const visibleRuleNames = useMemo(() => {+ return new Set(chartData.map(item => item.rawName));+ }, [chartData]);+// Domain sort handlerconst handleDomainSort = (key: DomainSortKey) => {if (domainSortKey === key) {@@ -505,10 +551,13 @@ export function InteractiveRuleStats({// Chart dataconst domainChartData = useMemo<RuleDomainChartItem[]>(() => {if (!ruleDomains?.length) return [];+ // Show more items in wide container (single column layout)+ const itemCount = topDomainsWidth >= 500 ? 15 : 10;+ const maxNameLength = topDomainsWidth >= 500 ? 35 : 15;return ruleDomains- .slice(0, 10)+ .slice(0, itemCount).map((domain, index) => ({- name: domain.domain.length > 15 ? domain.domain.slice(0, 15) + "..." : domain.domain,+ name: domain.domain.length > maxNameLength ? domain.domain.slice(0, maxNameLength - 3) + "..." : domain.domain,fullDomain: domain.domain,total: domain.totalDownload + domain.totalUpload,download: domain.totalDownload,@@ -516,7 +565,7 @@ export function InteractiveRuleStats({connections: domain.totalConnections,color: CHART_COLORS[index % CHART_COLORS.length],}));- }, [ruleDomains]);+ }, [ruleDomains, topDomainsWidth]);const isBackendUnavailable = backendStatus === "unhealthy";const emptyHint = isBackendUnavailable@@ -579,9 +628,9 @@ export function InteractiveRuleStats({return (<div className="space-y-6">{/* Top Section: Three Column Layout */}- <div className="grid grid-cols-1 lg:grid-cols-2 2xl:grid-cols-12 gap-6">- {/* Left: Pie Chart (3 columns) */}- <Card className="min-w-0 lg:col-span-1 2xl:col-span-3">+ <div className="grid grid-cols-1 md:grid-cols-2 xl:grid-cols-3 gap-4 lg:gap-6">+ {/* Left: Pie Chart */}+ <Card className="min-w-0"><CardHeader className="pb-2"><CardTitle className="text-sm font-semibold text-muted-foreground uppercase tracking-wider">{t("distribution")}@@ -665,8 +714,8 @@ export function InteractiveRuleStats({</CardContent></Card>- {/* Middle: Rule List (4 columns) */}- <Card className="min-w-0 lg:col-span-1 2xl:col-span-4">+ {/* Middle: Rule List */}+ <Card className="min-w-0"><CardHeader className="pb-2"><CardTitle className="text-sm font-semibold text-muted-foreground uppercase tracking-wider">{t("ruleList")}@@ -697,61 +746,127 @@ export function InteractiveRuleStats({key={item.rawName}onClick={() => !noTraffic && handleRuleClick(item.rawName)}className={cn(- "w-full p-2.5 rounded-xl border text-left transition-all duration-200",+ "w-full p-2.5 rounded-xl border text-left transition-all duration-200 @container",noTraffic? "border-border/30 bg-card/30 opacity-50 cursor-default": isSelected? "border-primary bg-primary/5 ring-1 ring-primary/20": "border-border/50 bg-card/50 hover:bg-card hover:border-primary/30")}>- {/* Row 1: Rank + Name + Total */}- <div className="flex items-center gap-2 mb-1.5">- <span className={cn(- "w-5 h-5 rounded-md text-[10px] font-bold flex items-center justify-center shrink-0",- badgeColor- )}>- {noTraffic ? "โ" : item.rank + 1}- </span>-- <span className="flex-1 text-sm font-medium truncate" title={item.name}>- {item.name.length > 24 ? item.name.slice(0, 24) + "..." : item.name}- </span>+ {/* Layout for wide container (default) */}+ <div className="hidden @min-[200px]:block">+ {/* Row 1: Rank + Name + Total */}+ <div className="flex items-center gap-2 mb-1.5 min-w-0">+ <span className={cn(+ "w-5 h-5 rounded-md text-[10px] font-bold flex items-center justify-center shrink-0",+ badgeColor+ )}>+ {noTraffic ? "โ" : item.rank + 1}+ </span>++ <span+ className="flex-1 text-sm font-medium truncate min-w-0"+ title={item.name}+ >+ {item.name}+ </span>++ <span className="text-sm font-bold tabular-nums shrink-0 whitespace-nowrap ml-auto">+ {noTraffic ? (+ <span className="text-xs font-normal text-muted-foreground">{t("noTrafficRecord")}</span>+ ) : formatBytes(item.value)}+ </span>+ </div>- <span className="text-sm font-bold tabular-nums shrink-0 whitespace-nowrap">- {noTraffic ? (- <span className="text-xs font-normal text-muted-foreground">{t("noTrafficRecord")}</span>- ) : formatBytes(item.value)}- </span>+ {/* Row 2: Progress bar + Stats (hidden for zero-traffic) */}+ {!noTraffic && (+ <div className="pl-7 space-y-1">+ {/* Progress bar - dual color */}+ <div className="h-1.5 rounded-full bg-muted overflow-hidden flex">+ <div+ className="h-full bg-blue-500 dark:bg-blue-400"+ style={{ width: `${item.value > 0 ? (item.download / item.value) * barPercent : 0}%` }}+ />+ <div+ className="h-full bg-purple-500 dark:bg-purple-400"+ style={{ width: `${item.value > 0 ? (item.upload / item.value) * barPercent : 0}%` }}+ />+ </div>+ {/* Stats */}+ <div className="flex items-center justify-between text-xs text-muted-foreground">+ <div className="flex flex-wrap items-center gap-x-2 gap-y-0.5">+ <span className="text-blue-500 dark:text-blue-400 whitespace-nowrap">โ {formatBytes(item.download)}</span>+ <span className="text-purple-500 dark:text-blue-400 whitespace-nowrap">โ {formatBytes(item.upload)}</span>+ <span className="flex items-center gap-1 tabular-nums">+ <Link2 className="w-3 h-3" />+ {formatNumber(item.connections)}+ </span>+ </div>+ <span className="tabular-nums">{percentage.toFixed(1)}%</span>+ </div>+ </div>+ )}</div>- {/* Row 2: Progress bar + Stats (hidden for zero-traffic) */}- {!noTraffic && (- <div className="pl-7 space-y-1">- {/* Progress bar - dual color */}- <div className="h-1.5 rounded-full bg-muted overflow-hidden flex">- <div- className="h-full bg-blue-500 dark:bg-blue-400"- style={{ width: `${item.value > 0 ? (item.download / item.value) * barPercent : 0}%` }}- />- <div- className="h-full bg-purple-500 dark:bg-purple-400"- style={{ width: `${item.value > 0 ? (item.upload / item.value) * barPercent : 0}%` }}- />+ {/* Layout for narrow container (vertical stack) */}+ <div className="block @min-[200px]:hidden space-y-2">+ {/* Row 1: Rank + Name */}+ <div className="flex items-center gap-2">+ <span className={cn(+ "w-5 h-5 rounded-md text-[10px] font-bold flex items-center justify-center shrink-0",+ badgeColor+ )}>+ {noTraffic ? "โ" : item.rank + 1}+ </span>+ <span className="flex-1 text-sm font-medium line-clamp-2 leading-tight" title={item.name}>+ {item.name}+ </span></div>- {/* Stats */}- <div className="grid grid-cols-1 min-[300px]:grid-cols-[1fr_auto] gap-x-2 gap-y-0.5 text-xs text-muted-foreground">- <div className="flex flex-wrap items-center gap-x-2 gap-y-0.5">- <span className="text-blue-500 dark:text-blue-400 whitespace-nowrap">โ {formatBytes(item.download)}</span>- <span className="text-purple-500 dark:text-blue-400 whitespace-nowrap">โ {formatBytes(item.upload)}</span>- <span className="flex items-center gap-1 tabular-nums">- <Link2 className="w-3 h-3" />- {formatNumber(item.connections)}- </span>++ {/* Row 2: Stats Grid (hidden for zero-traffic) */}+ {!noTraffic && (+ <div className="pl-7 space-y-2">+ {/* Total Traffic */}+ <div className="flex items-center justify-between">+ <span className="text-xs text-muted-foreground">{t("total")}</span>+ <span className="text-sm font-bold tabular-nums">{formatBytes(item.value)}</span>+ </div>++ {/* Progress bar */}+ <div className="h-1.5 rounded-full bg-muted overflow-hidden flex">+ <div+ className="h-full bg-blue-500 dark:bg-blue-400"+ style={{ width: `${item.value > 0 ? (item.download / item.value) * 100 : 0}%` }}+ />+ <div+ className="h-full bg-purple-500 dark:bg-purple-400"+ style={{ width: `${item.value > 0 ? (item.upload / item.value) * 100 : 0}%` }}+ />+ </div>++ {/* Download / Upload / Connections */}+ <div className="grid grid-cols-3 gap-1 text-xs">+ <div className="text-center p-1 rounded bg-blue-50 dark:bg-blue-950/30">+ <div className="text-blue-500 dark:text-blue-400 mb-0.5">โ</div>+ <div className="font-medium tabular-nums truncate">{formatBytes(item.download)}</div>+ </div>+ <div className="text-center p-1 rounded bg-purple-50 dark:bg-purple-950/30">+ <div className="text-purple-500 dark:text-purple-400 mb-0.5">โ</div>+ <div className="font-medium tabular-nums truncate">{formatBytes(item.upload)}</div>+ </div>+ <div className="text-center p-1 rounded bg-muted/50">+ <div className="text-muted-foreground mb-0.5"><Link2 className="w-3 h-3 mx-auto" /></div>+ <div className="font-medium tabular-nums">{formatNumber(item.connections)}</div>+ </div></div>- <span className="tabular-nums text-right min-[300px]:text-right">{percentage.toFixed(1)}%</span></div>+ )}++ {/* Zero traffic message */}+ {noTraffic && (+ <div className="pl-7 text-xs text-muted-foreground">{t("noTrafficRecord")}</div>+ )}</div>- )}</button>);})}@@ -760,8 +875,9 @@ export function InteractiveRuleStats({</CardContent></Card>- {/* Right: Top Domains Chart (5 columns) */}- <Card className="min-w-0 lg:col-span-2 2xl:col-span-5">+ {/* Right: Top Domains Chart - Full width on single column, adapts to container */}+ <div ref={topDomainsCardRef} className="min-w-0 md:col-span-2 xl:col-span-1">+ <Card className="min-w-0 @container h-full"><CardHeader className="pb-2"><div className="flex items-center justify-between"><CardTitle className="text-sm font-semibold text-muted-foreground uppercase tracking-wider flex items-center gap-2">@@ -785,12 +901,12 @@ export function InteractiveRuleStats({<p className="text-xs text-muted-foreground/80 mt-1 max-w-xs">{emptyHint}</p></div>) : (- <div className="h-[280px] w-full min-w-0 overflow-hidden sm:overflow-visible">+ <div className="h-[280px] @min-[500px]:h-[360px] w-full min-w-0 overflow-hidden sm:overflow-visible"><ResponsiveContainer width="100%" height="100%"><BarChartdata={domainChartData}layout="vertical"- margin={{ top: 5, right: showDomainBarLabels ? 50 : 10, left: 5, bottom: 5 }}+ margin={{ top: 5, right: showDomainBarLabels ? 60 : 10, left: 0, bottom: 5 }}><CartesianGridstrokeDasharray="3 3"@@ -806,13 +922,13 @@ export function InteractiveRuleStats({axisLine={false}/><YAxis- type="category"- dataKey="name"- width={90}- tick={{ fontSize: 10, fill: "currentColor" }}- tickLine={false}- axisLine={false}- />+ type="category"+ dataKey="name"+ width={120}+ tick={{ fontSize: 10, fill: "currentColor" }}+ tickLine={false}+ axisLine={false}+ /><RechartsTooltipcontent={({ active, payload }) => {if (active && payload && payload.length) {@@ -873,6 +989,7 @@ export function InteractiveRuleStats({)}</CardContent></Card>+ </div></div>{/* Unified Chain Flow Visualization - shows all rules, highlights selected */}@@ -881,6 +998,7 @@ export function InteractiveRuleStats({activeBackendId={activeBackendId}timeRange={stableTimeRange}autoRefresh={autoRefresh}+ visibleRuleNames={visibleRuleNames}/>{/* Bottom Section: Domain List & IP Addresses with pagination */}diff --git a/apps/web/components/features/rules/rule-chain-flow.tsx b/apps/web/components/features/rules/rule-chain-flow.tsxindex 5b7baa6..cadf5b0 100644--- a/apps/web/components/features/rules/rule-chain-flow.tsx+++ b/apps/web/components/features/rules/rule-chain-flow.tsx@@ -38,6 +38,8 @@ import { useStableTimeRange } from "@/lib/hooks/use-stable-time-range";import { resolveActiveChains, type ActiveChainInfo } from "@/lib/active-chain";import { useTranslations } from "next-intl";import type { StatsSummary } from "@neko-master/shared";+import { useGatewayProviders, useGatewayProxies } from "@/hooks/api/use-gateway";+import { useGatewayRules } from "@/hooks/api/use-rules";// ---------- Types ----------@@ -64,6 +66,7 @@ interface UnifiedRuleChainFlowProps {activeBackendId?: number;timeRange?: TimeRange;autoRefresh?: boolean;+ visibleRuleNames?: Set<string>;}// ---------- Layout constants ----------@@ -803,6 +806,7 @@ function UnifiedRuleChainFlowInner({activeBackendId,timeRange,autoRefresh = true,+ visibleRuleNames,}: UnifiedRuleChainFlowProps) {const t = useTranslations("rules");// Round timeRange to the minute so WS/HTTP only re-fires on minute boundaries@@ -812,8 +816,6 @@ function UnifiedRuleChainFlowInner({const [error, setError] = useState<string | null>(null);const [showAll, setShowAll] = useState(false);const [activePolicyOnly, setActivePolicyOnly] = useState(false);- const [activeChainInfo, setActiveChainInfo] =- useState<ActiveChainInfo | null>(null);const prevRuleRef = useRef(selectedRule);const hasLoadedRef = useRef(false);const requestIdRef = useRef(0);@@ -997,39 +999,29 @@ function UnifiedRuleChainFlowInner({}}, [activeBackendId, wsEnabled]);- // Fetch active chain info for zero-traffic chain merge.- useEffect(() => {- setActiveChainInfo(null);- let cancelled = false;-- async function fetchActiveChains() {- try {- const [gatewayProviders, gatewayRules] = await Promise.all([- api.getGatewayProviders(activeBackendId),- api.getGatewayRules(activeBackendId),- ]);-- if (cancelled) return;-- const resolved = resolveActiveChains(gatewayProviders, gatewayRules);- setActiveChainInfo(resolved);- } catch {- if (!cancelled) setActivePolicyOnly(false);- }- }+ // Use React Query to fetch active chain info with caching+ const { data: gatewayProviders } = useGatewayProviders({+ activeBackendId,+ enabled: !!activeBackendId+ });+ const { data: gatewayProxies } = useGatewayProxies({+ activeBackendId,+ enabled: !!activeBackendId+ });+ const { data: gatewayRules } = useGatewayRules({+ activeBackendId,+ enabled: !!activeBackendId+ });- fetchActiveChains();- if (!autoRefresh) {- return () => {- cancelled = true;- };+ // Compute active chain info from cached data+ const activeChainInfo: ActiveChainInfo | null = useMemo(() => {+ if (!gatewayProviders || !gatewayRules) return null;+ try {+ return resolveActiveChains(gatewayProviders, gatewayRules, gatewayProxies || undefined);+ } catch {+ return null;}- const interval = setInterval(fetchActiveChains, 60000);- return () => {- cancelled = true;- clearInterval(interval);- };- }, [activeBackendId, autoRefresh]);+ }, [gatewayProviders, gatewayRules, gatewayProxies]);// When user selects a different rule, auto-disable showAlluseEffect(() => {@@ -1042,6 +1034,12 @@ function UnifiedRuleChainFlowInner({// Filter/Merge DAGconst filteredData = useMemo((): AllChainFlowData | null => {if (!data) return null;+ // If activeChainInfo is not loaded yet and we're not showing all policies,+ // we need to wait for it to properly filter the data.+ // Return empty data to show loading state instead of unfiltered data.+ if (!activeChainInfo && !activePolicyOnly) {+ return { nodes: [], links: [], rulePaths: {}, maxLayer: 0 };+ }if (!activeChainInfo) return data;const { activeNodeNames, activeLinkKeys, activeChains } = activeChainInfo;@@ -1052,18 +1050,40 @@ function UnifiedRuleChainFlowInner({// Determine which existing nodes to keepconst keepNodeIndices = new Set<number>();- if (!activePolicyOnly) {- // Keep ALL existing nodes if not filtering- data.nodes.forEach((_, i) => keepNodeIndices.add(i));- } else {- // Keep only active nodes from existing data- for (const [, idx] of existingNodeMap) {- if (activeNodeNames.has(data.nodes[idx].name)) {- keepNodeIndices.add(idx);- }- }++ // Helper to check if a node has traffic+ const nodeHasTraffic = (node: MergedChainNode) =>+ node.totalUpload > 0 || node.totalDownload > 0 || node.totalConnections > 0;++ // Build a normalized map of rulePaths to handle potential whitespace mismatches+ // (e.g. Surge rules might have leading/trailing spaces in API vs UI)+ const normalizedRulePaths = new Map<string, { nodeIndices: number[]; linkIndices: number[] }>();+ for (const [key, value] of Object.entries(data.rulePaths)) {+ normalizedRulePaths.set(key.trim(), value);}+ data.nodes.forEach((node, idx) => {+ // 1. If "All Policies" is checked, keep everything+ if (activePolicyOnly) {+ keepNodeIndices.add(idx);+ return;+ }++ // 2. If a specific rule is selected, always keep nodes in its path+ // Use normalized lookup+ if (selectedRule && normalizedRulePaths.get(selectedRule)?.nodeIndices.includes(idx)) {+ keepNodeIndices.add(idx);+ return;+ }++ // 3. Otherwise, only keep nodes with active traffic or relevant to active chains+ if (activeChains.size > 0 && (nodeHasTraffic(node) || activeNodeNames.has(node.name))) {+ if (nodeHasTraffic(node)) {+ keepNodeIndices.add(idx);+ }+ }+ });+// Build new nodes array (existing kept + zero-traffic for missing active)const newNodes: MergedChainNode[] = [];const nameToNewIdx = new Map<string, number>();@@ -1076,12 +1096,40 @@ function UnifiedRuleChainFlowInner({}// Second: add zero-traffic nodes for active chain members not in existing data- for (const [, chain] of activeChains) {+ // If "All Policies" is ON, we inject nodes for ALL chains.+ // If a whitelist (visibleRuleNames) is provided, we only show those.+ // Otherwise, we only inject for the currently selected rule.+ const chainsToInject = new Set<string>();+ if (activePolicyOnly) {+ if (visibleRuleNames && visibleRuleNames.size > 0) {+ // Only inject chains that are in the user's rule list+ for (const ruleName of visibleRuleNames) {+ if (activeChains.has(ruleName)) {+ chainsToInject.add(ruleName);+ }+ }+ } else {+ // Fallback: inject everything+ for (const ruleName of activeChains.keys()) {+ chainsToInject.add(ruleName);+ }+ }+ } else if (selectedRule && activeChains.has(selectedRule)) {+ chainsToInject.add(selectedRule);+ }++ for (const ruleName of chainsToInject) {+ const chain = activeChains.get(ruleName);+ if (!chain) continue;+for (let i = 0; i < chain.length; i++) {const name = chain[i];if (!nameToNewIdx.has(name)) {+ // Special built-in policies that are not actual proxies+ const isBuiltInPolicy = name === 'DIRECT' || name === 'REJECT' || name === 'REJECT-TINY';+ const isLastNode = i === chain.length - 1;const nodeType: "rule" | "group" | "proxy" =- i === 0 ? "rule" : i === chain.length - 1 ? "proxy" : "group";+ i === 0 ? "rule" : (isLastNode && !isBuiltInPolicy) ? "proxy" : "group";nameToNewIdx.set(name, newNodes.length);newNodes.push({name,@@ -1103,35 +1151,50 @@ function UnifiedRuleChainFlowInner({const linkKeySet = new Set<string>();// Existing links from data (filtered if needed)- for (const link of data.links) {- const srcName = data.nodes[link.source]?.name;- const tgtName = data.nodes[link.target]?.name;+ data.links.forEach((link, idx) => {+ const srcNode = data.nodes[link.source];+ const tgtNode = data.nodes[link.target];+ if (!srcNode || !tgtNode) return;- // If filtering, check if link is in active keys. If not filtering, keep it.- const shouldKeepLink =- !activePolicyOnly ||- (srcName && tgtName && activeLinkKeys.has(`${srcName}|${tgtName}`));-- if (shouldKeepLink && srcName && tgtName) {- const newSrc = nameToNewIdx.get(srcName);- const newTgt = nameToNewIdx.get(tgtName);- // Ensure both endpoints exist in our new node set- if (newSrc !== undefined && newTgt !== undefined) {- const key = `${newSrc}-${newTgt}`;- if (!linkKeySet.has(key)) {- linkKeySet.add(key);- newLinks.push({- source: newSrc,- target: newTgt,- rules: link.rules,- });- }+ const newSrc = nameToNewIdx.get(srcNode.name);+ const newTgt = nameToNewIdx.get(tgtNode.name);++ // Both endpoints must be in our kept nodes+ if (newSrc !== undefined && newTgt !== undefined) {+ // Keep link if "All Policies" OR if it's a Rule->Group link OR generally if endpoints are kept+ const isRuleLink = srcNode.nodeType === 'rule';++ // Check if the server-side analysis knows this link for the selected rule+ // This is crucial for Surge backend where client-side chain resolution might miss some links+ // Use normalized lookup here too+ const isServerKnownLink = selectedRule &&+ normalizedRulePaths.get(selectedRule)?.linkIndices?.includes(idx);++ const shouldKeepLink =+ activePolicyOnly ||+ isRuleLink ||+ isServerKnownLink ||+ activeLinkKeys.has(`${srcNode.name}|${tgtNode.name}`);++ if (shouldKeepLink) {+ const key = `${newSrc}-${newTgt}`;+ if (!linkKeySet.has(key)) {+ linkKeySet.add(key);+ newLinks.push({+ source: newSrc,+ target: newTgt,+ rules: link.rules,+ });+ }}}- }+ });// Zero-traffic links for active chains not yet covered- for (const [, chain] of activeChains) {+ for (const ruleName of chainsToInject) {+ const chain = activeChains.get(ruleName);+ if (!chain) continue;+for (let i = 0; i < chain.length - 1; i++) {const srcIdx = nameToNewIdx.get(chain[i]);const tgtIdx = nameToNewIdx.get(chain[i + 1]);@@ -1150,31 +1213,53 @@ function UnifiedRuleChainFlowInner({string,{ nodeIndices: number[]; linkIndices: number[] }> = {};++ // We try to rebuild paths for all active chains that have nodes presentfor (const [ruleName, chain] of activeChains) {const nodeIndices: number[] = [];const linkIndices: number[] = [];++ // Nodesfor (const name of chain) {const idx = nameToNewIdx.get(name);if (idx !== undefined) nodeIndices.push(idx);}- // Find link indices++ // Linksfor (let i = 0; i < chain.length - 1; i++) {const srcIdx = nameToNewIdx.get(chain[i]);const tgtIdx = nameToNewIdx.get(chain[i + 1]);if (srcIdx !== undefined && tgtIdx !== undefined) {- const linkIdx = newLinks.findIndex(- (l) => l.source === srcIdx && l.target === tgtIdx,- );- if (linkIdx !== -1) linkIndices.push(linkIdx);+ // Locate link in newLinks+ const linkIdx = newLinks.findIndex(l => l.source === srcIdx && l.target === tgtIdx);+ if (linkIdx !== -1) linkIndices.push(linkIdx);}}+if (nodeIndices.length > 0) {newRulePaths[ruleName] = { nodeIndices, linkIndices };}}// Compute maxLayer- const maxLayer = newNodes.reduce((max, n) => Math.max(max, n.layer), 0);+ // const maxLayer = newNodes.reduce((max, n) => Math.max(max, n.layer), 0);++ // [New Logic]+ // To align usage with user request: "Right-align all PROXY nodes".+ // 1. Find the deepest layer among all nodes (or specifically rule/group nodes which define the structure).+ let maxLayer = 0;+ for (const n of newNodes) {+ if (n.layer > maxLayer) maxLayer = n.layer;+ }+ // Ensure minimum depth to separate Rule -> Proxy+ if (maxLayer < 2) maxLayer = 2; // At least Rule(0) -> Gap(1) -> Proxy(2)++ // 2. Force all "proxy" nodes to be at maxLayer+ for (const n of newNodes) {+ if (n.nodeType === 'proxy') {+ n.layer = maxLayer;+ }+ }return {nodes: newNodes,@@ -1182,7 +1267,7 @@ function UnifiedRuleChainFlowInner({rulePaths: newRulePaths,maxLayer,};- }, [data, activePolicyOnly, activeChainInfo]);+ }, [data, activePolicyOnly, activeChainInfo, selectedRule, visibleRuleNames]);const renderData = filteredData;diff --git a/apps/web/components/ui/insight-skeleton.tsx b/apps/web/components/ui/insight-skeleton.tsxindex 6a58aaf..c45c4f5 100644--- a/apps/web/components/ui/insight-skeleton.tsx+++ b/apps/web/components/ui/insight-skeleton.tsx@@ -95,3 +95,39 @@ export function InsightDetailSectionSkeleton({ className }: { className?: string</div>);}++// Backend list skeleton for Settings dialog+export function BackendListSkeleton({ count = 3 }: { count?: number }) {+ return (+ <div className="space-y-3">+ {Array.from({ length: count }).map((_, idx) => (+ <div+ key={idx}+ className="p-4 rounded-lg border border-border bg-card"+ >+ <div className="flex items-center justify-between">+ <div className="flex items-center gap-3 flex-1 min-w-0">+ {/* Icon skeleton */}+ <div className="w-8 h-8 rounded-lg bg-muted/60 animate-pulse shrink-0" />+ <div className="flex-1 min-w-0 space-y-2">+ {/* Name skeleton */}+ <div className="h-5 w-32 rounded bg-muted/60 animate-pulse" />+ {/* URL skeleton */}+ <div className="h-4 w-48 rounded bg-muted/50 animate-pulse" />+ </div>+ </div>+ {/* Actions skeleton */}+ <div className="flex items-center gap-2">+ <div className="h-8 w-16 rounded-full bg-muted/60 animate-pulse" />+ <div className="h-8 w-8 rounded-lg bg-muted/50 animate-pulse" />+ <div className="h-8 w-8 rounded-lg bg-muted/50 animate-pulse" />+ <div className="h-8 w-8 rounded-lg bg-muted/50 animate-pulse" />+ </div>+ </div>+ </div>+ ))}+ {/* Add button skeleton */}+ <div className="h-12 rounded-lg border border-dashed border-border bg-card/50 animate-pulse" />+ </div>+ );+}diff --git a/apps/web/components/ui/sonner.tsx b/apps/web/components/ui/sonner.tsxindex 3af179c..ffd43e5 100644--- a/apps/web/components/ui/sonner.tsx+++ b/apps/web/components/ui/sonner.tsx@@ -2,6 +2,7 @@import { useTheme } from "next-themes"import { Toaster as Sonner } from "sonner"+import { CheckCircle2, XCircle, AlertTriangle, Info } from "lucide-react"type ToasterProps = React.ComponentProps<typeof Sonner>@@ -12,6 +13,12 @@ const Toaster = ({ ...props }: ToasterProps) => {<Sonnertheme={theme as ToasterProps["theme"]}className="toaster group"+ icons={{+ success: <CheckCircle2 className="w-4 h-4 text-emerald-500" />,+ error: <XCircle className="w-4 h-4 text-red-500" />,+ warning: <AlertTriangle className="w-4 h-4 text-amber-500" />,+ info: <Info className="w-4 h-4 text-blue-500" />,+ }}toastOptions={{classNames: {toast:@@ -21,6 +28,9 @@ const Toaster = ({ ...props }: ToasterProps) => {"group-[.toast]:bg-primary group-[.toast]:text-primary-foreground",cancelButton:"group-[.toast]:bg-muted group-[.toaster]:text-muted-foreground",+ success: "group-[.toaster]:border-emerald-500/30",+ error: "group-[.toaster]:border-red-500/30",+ warning: "group-[.toaster]:border-amber-500/30",},}}{...props}diff --git a/apps/web/hooks/api/use-gateway.ts b/apps/web/hooks/api/use-gateway.tsnew file mode 100644index 0000000..d9fb834--- /dev/null+++ b/apps/web/hooks/api/use-gateway.ts@@ -0,0 +1,89 @@+"use client";++import { useQuery, useMutation, useQueryClient } from "@tanstack/react-query";+import { api } from "@/lib/api";+import { QUERY_CONFIG } from "@/lib/query-config";++const PROVIDERS_KEY = "gatewayProviders";+const PROXIES_KEY = "gatewayProxies";+const RULES_KEY = "gatewayRules";++interface UseGatewayOptions {+ activeBackendId?: number;+ enabled?: boolean;+}++/**+ * ่ทๅ Gateway Providers (ๅธฆ็ผๅญ)+ */+export function useGatewayProviders({+ activeBackendId,+ enabled = true,+}: UseGatewayOptions) {+ return useQuery({+ queryKey: [PROVIDERS_KEY, { backendId: activeBackendId }],+ queryFn: () => api.getGatewayProviders(activeBackendId),+ enabled: !!activeBackendId && enabled,+ staleTime: 5 * 60 * 1000, // 5 ๅ้+ placeholderData: (previous) => previous,+ });+}++/**+ * ่ทๅ Gateway Proxies (ๅธฆ็ผๅญ)+ */+export function useGatewayProxies({+ activeBackendId,+ enabled = true,+}: UseGatewayOptions) {+ return useQuery({+ queryKey: [PROXIES_KEY, { backendId: activeBackendId }],+ queryFn: () => api.getGatewayProxies(activeBackendId),+ enabled: !!activeBackendId && enabled,+ staleTime: 5 * 60 * 1000, // 5 ๅ้+ placeholderData: (previous) => previous,+ });+}++/**+ * ่ทๅ Gateway Rules (ๅธฆ็ผๅญ)+ * ๆณจๆ: useGatewayRules ๅจ use-rules.ts ไธญๅทฒๅฎไน๏ผ่ฟ้ๅฏผๅบไปฅ้ฟๅ ้ๅค+ */+export { useGatewayRules as useGatewayRules } from "./use-rules";++/**+ * ๅทๆฐๆๆ Gateway ็ผๅญ+ */+export function useRefreshGatewayData() {+ const queryClient = useQueryClient();++ return useMutation({+ mutationFn: async (backendId?: number) => {+ const [providers, proxies, rules] = await Promise.all([+ api.getGatewayProviders(backendId),+ api.getGatewayProxies(backendId),+ api.getGatewayRules(backendId),+ ]);+ return { providers, proxies, rules };+ },+ onSuccess: (data, backendId) => {+ // ๆดๆฐ็ผๅญ+ queryClient.setQueryData([PROVIDERS_KEY, { backendId }], data.providers);+ queryClient.setQueryData([PROXIES_KEY, { backendId }], data.proxies);+ queryClient.setQueryData([RULES_KEY, { backendId }], data.rules);+ },+ });+}++/**+ * ไฝฟ Gateway ็ผๅญๅคฑๆ๏ผ็จไบๅผบๅถๅทๆฐ๏ผ+ */+export function useInvalidateGatewayCache() {+ const queryClient = useQueryClient();++ return () => {+ queryClient.invalidateQueries({ queryKey: [PROVIDERS_KEY] });+ queryClient.invalidateQueries({ queryKey: [PROXIES_KEY] });+ queryClient.invalidateQueries({ queryKey: [RULES_KEY] });+ };+}diff --git a/apps/web/lib/active-chain.ts b/apps/web/lib/active-chain.tsindex 964e87f..ac45379 100644--- a/apps/web/lib/active-chain.ts+++ b/apps/web/lib/active-chain.ts@@ -1,4 +1,5 @@-import type { GatewayProvidersResponse, GatewayRulesResponse } from "./api";+import type { GatewayProvidersResponse, GatewayRulesResponse, GatewayProxiesResponse } from "./api";+import { parseGatewayRule } from "@neko-master/shared";export interface ActiveChainInfo {/** Set of node names that are part of active chains */@@ -13,8 +14,13 @@ export interface ActiveChainInfo {* Build a map of group name โ currently selected proxy/group name* from the providers/proxies response.*/-function buildGroupNowMap(providers: GatewayProvidersResponse): Map<string, string> {+function buildGroupNowMap(+ providers: GatewayProvidersResponse,+ proxies?: GatewayProxiesResponse+): Map<string, string> {const map = new Map<string, string>();++ // 1. Add groups from Providersfor (const provider of Object.values(providers.providers)) {if (!provider.proxies) continue;for (const proxy of provider.proxies) {@@ -23,6 +29,15 @@ function buildGroupNowMap(providers: GatewayProvidersResponse): Map<string, stri}}}++ // 2. Add top-level groups from Proxies (Surge/Clash global proxies)+ if (proxies?.proxies) {+ for (const proxy of Object.values(proxies.proxies)) {+ if (proxy.now) {+ map.set(proxy.name, proxy.now);+ }+ }+ }return map;}@@ -51,29 +66,65 @@ function followChain(startGroup: string, groupNowMap: Map<string, string>): stri*/export function resolveActiveChains(providers: GatewayProvidersResponse,- rules: GatewayRulesResponse+ rules: GatewayRulesResponse,+ proxies?: GatewayProxiesResponse): ActiveChainInfo {- const groupNowMap = buildGroupNowMap(providers);+ // Use the global proxy list (from /gateway/proxies) as the primary source for 'now'+ // because it contains the accurate 'now' field we fetched from the backend.+ const groupNowMap = buildGroupNowMap(providers, proxies);+const activeNodeNames = new Set<string>();const activeLinkKeys = new Set<string>();const activeChains = new Map<string, string[]>();- const processedGroups = new Set<string>();+ // Helper to process a chain+ const processChainRecursive = (startNode: string) => {+ if (activeChains.has(startNode)) return activeChains.get(startNode)!;++ const chain = followChain(startNode, groupNowMap);+ activeChains.set(startNode, chain);++ // Mark nodes and links as active+ for (const name of chain) {+ activeNodeNames.add(name);+ }+ for (let i = 0; i < chain.length - 1; i++) {+ activeLinkKeys.add(`${chain[i]}|${chain[i+1]}`);+ }+ return chain;+ };- for (const rule of rules.rules) {- const targetGroup = rule.proxy;- if (processedGroups.has(targetGroup)) continue;- processedGroups.add(targetGroup);+ // 1. First pass: Resolve chains for all groups in the groupNowMap+ // This ensures that even groups not referenced by rules directly (but part of a chain) are resolved?+ // Actually, we only need to resolve starting from Rules or known start points.++ // 2. Process all rules+ if (rules && rules.rules) {+ for (const rawRule of rules.rules) {+ const parsed = parseGatewayRule(rawRule);+ if (!parsed || !parsed.proxy) continue;- const chain = followChain(targetGroup, groupNowMap);- activeChains.set(targetGroup, chain);+ const targetGroup = parsed.proxy;++ // Resolve the chain for the Target Group of this rule+ const groupChain = processChainRecursive(targetGroup);- for (const name of chain) {- activeNodeNames.add(name);- }- for (let i = 0; i < chain.length - 1; i++) {- activeLinkKeys.add(`${chain[i]}|${chain[i + 1]}`);- }+ const ruleName = parsed.payload;++ // Map Rule -> Group -> [Rest of Chain]+ // Note: For "FINAL" rules, ruleName might be empty or "FINAL".+ // In the UI, the rule node ID is usually the payload or normalized string.++ if (ruleName) {+ const ruleChain = [ruleName, ...groupChain];+ activeChains.set(ruleName, ruleChain);++ activeNodeNames.add(ruleName);+ if (groupChain.length > 0) {+ activeLinkKeys.add(`${ruleName}|${groupChain[0]}`);+ }+ }+ }}return { activeNodeNames, activeLinkKeys, activeChains };diff --git a/apps/web/lib/api.ts b/apps/web/lib/api.tsindex c7fcd8b..22e0fcd 100644--- a/apps/web/lib/api.ts+++ b/apps/web/lib/api.ts@@ -110,15 +110,24 @@ export interface TimeRange {end: string;}+export interface BackendHealth {+ status: 'healthy' | 'unhealthy' | 'unknown';+ lastChecked: number;+ message?: string;+ latency?: number;+}+export interface Backend {id: number;name: string;url: string;token: string;+ type?: 'clash' | 'surge';enabled: boolean;is_active: boolean;listening: boolean;hasToken?: boolean;+ health?: BackendHealth;created_at: string;updated_at: string;}@@ -139,6 +148,10 @@ export interface GatewayProvider {proxies: GatewayProviderProxy[];}+export interface GatewayProxiesResponse {+ proxies: Record<string, GatewayProviderProxy>;+}+export interface GatewayProvidersResponse {providers: Record<string, GatewayProvider>;}@@ -147,10 +160,15 @@ export interface GatewayRule {type: string;payload: string;proxy: string;+ size?: number;+ raw?: string;}export interface GatewayRulesResponse {rules: GatewayRule[];+ // Surge specific fields+ _source?: 'surge' | 'clash';+ _availablePolicies?: string[];}const DEFAULT_DB_STATS = {@@ -473,15 +491,18 @@ export const api = {getGatewayRules: (backendId?: number) =>fetchJson<GatewayRulesResponse>(buildUrl(`${API_BASE}/gateway/rules`, { backendId })),++ getGatewayProxies: (backendId?: number) =>+ fetchJson<GatewayProxiesResponse>(buildUrl(`${API_BASE}/gateway/proxies`, { backendId })),// Backend managementgetBackends: () =>fetchJson<Backend[]>(`${API_BASE}/backends`),- createBackend: (backend: { name: string; url: string; token?: string }) =>+ createBackend: (backend: { name: string; url: string; token?: string; type?: 'clash' | 'surge' }) =>fetchJson<{ id: number; isActive?: boolean; message: string }>(`${API_BASE}/backends`, 'POST', backend),- updateBackend: (id: number, backend: { name?: string; url?: string; token?: string; enabled?: boolean; listening?: boolean }) =>+ updateBackend: (id: number, backend: { name?: string; url?: string; token?: string; type?: 'clash' | 'surge'; enabled?: boolean; listening?: boolean }) =>fetchJson<{ message: string }>(`${API_BASE}/backends/${id}`, 'PUT', backend),deleteBackend: (id: number) =>@@ -496,8 +517,8 @@ export const api = {clearBackendData: (id: number) =>fetchJson<{ message: string }>(`${API_BASE}/backends/${id}/clear-data`, 'POST'),- testBackend: (url: string, token?: string) =>- fetchJson<{ success: boolean; message: string }>(`${API_BASE}/backends/test`, 'POST', { url, token }),+ testBackend: (url: string, token?: string, type?: 'clash' | 'surge') =>+ fetchJson<{ success: boolean; message: string }>(`${API_BASE}/backends/test`, 'POST', { url, token, type }),testBackendById: (id: number) =>fetchJson<{ success: boolean; message: string }>(`${API_BASE}/backends/${id}/test`, 'POST'),diff --git a/apps/web/messages/en.json b/apps/web/messages/en.jsonindex 7c8941a..2174913 100644--- a/apps/web/messages/en.json+++ b/apps/web/messages/en.json@@ -44,6 +44,8 @@"backendUnavailable": "Backend unavailable","backendUnavailableHint": "Data collection is interrupted. Please check backend connectivity.","backendStatusUnknown": "Backend status unknown",+ "backendUnhealthyTitle": "Backend Connection Abnormal",+ "backendUnhealthyDesc": "The current backend connection is abnormal, please check the backend status in Settings","logout": "Logout"},"timeRangePicker": {@@ -356,7 +358,9 @@"token": "Auth Token","useSsl": "Use HTTPS","namePlaceholder": "e.g., Router",+ "type": "Type","tokenPlaceholder": "Optional, if set please fill in",+ "tokenPlaceholderSurge": "Surge API Key (x-key)","tokenKeepPlaceholder": "Token already set, leave blank to keep","active": "Active","disabled": "Disabled",@@ -368,6 +372,7 @@"confirmDelete": "Confirm Delete","deleteDescription": "Are you sure you want to delete this backend configuration? This action cannot be undone.","noBackends": "No backend configurations",+ "addBackendHint": "Click the button below to add a new backend","testSuccess": "Connection successful","testFailed": "Connection failed","databaseStats": "Database Statistics",@@ -386,6 +391,7 @@"switchFailed": "Failed to switch backend","displaying": "Displaying","collecting": "Collecting",+ "unhealthy": "Abnormal","collect": "Collect","setDisplay": "Set Display","selectBackend": "Select Backend",diff --git a/apps/web/messages/zh.json b/apps/web/messages/zh.jsonindex 6e4b45d..eba3cd1 100644--- a/apps/web/messages/zh.json+++ b/apps/web/messages/zh.json@@ -44,6 +44,8 @@"backendUnavailable": "ๅ็ซฏ่ฟๆฅๅผๅธธ","backendUnavailableHint": "ๆฐๆฎ้้ไธญๆญ๏ผ่ฏทๆฃๆฅๅ็ซฏ่้็ถๆใ","backendStatusUnknown": "ๅ็ซฏ็ถๆๆช็ฅ",+ "backendUnhealthyTitle": "ๅ็ซฏ่ฟๆฅๅผๅธธ",+ "backendUnhealthyDesc": "ๅฝๅๅ็ซฏ่ฟๆฅๅผๅธธ๏ผ่ฏทๅๅพ่ฎพ็ฝฎๆฃๆฅๅ็ซฏ็ถๆ","logout": "้ๅบ็ปๅฝ"},"timeRangePicker": {@@ -355,7 +357,9 @@"token": "่ฎค่ฏไปค็","useSsl": "ไฝฟ็จ HTTPS","namePlaceholder": "ไพๅฆ๏ผ่ทฏ็ฑๅจ",+ "type": "็ฑปๅ","tokenPlaceholder": "ๅฏ้๏ผๅฆๆ่ฎพ็ฝฎ่ฏทๅกซๅ",+ "tokenPlaceholderSurge": "Surge API Key (x-key)","tokenKeepPlaceholder": "ๅทฒ่ฎพ็ฝฎไปค็๏ผ็็ฉบไฟๆไธๅ","active": "ๅฝๅๆดป่ท","disabled": "ๅทฒ็ฆ็จ",@@ -367,6 +371,7 @@"confirmDelete": "็กฎ่ฎคๅ ้ค","deleteDescription": "็กฎๅฎ่ฆๅ ้ค่ฟไธชๅ็ซฏ้ ็ฝฎๅ๏ผๆญคๆไฝไธๅฏๆค้ใ","noBackends": "ๆๆ ๅ็ซฏ้ ็ฝฎ",+ "addBackendHint": "็นๅปไธๆนๆ้ฎๆทปๅ ๆฐๅ็ซฏ","testSuccess": "่ฟๆฅๆๅ","testFailed": "่ฟๆฅๅคฑ่ดฅ","databaseStats": "ๆฐๆฎๅบ็ป่ฎก",@@ -385,6 +390,7 @@"switchFailed": "ๅ็ซฏๅๆขๅคฑ่ดฅ","displaying": "ๆพ็คบไธญ","collecting": "ๆถ้ไธญ",+ "unhealthy": "ๅผๅธธ","collect": "ๆถ้","setDisplay": "่ฎพไธบๆพ็คบ","selectBackend": "้ๆฉๅ็ซฏ",diff --git a/apps/web/public/icons/icon-clash.png b/apps/web/public/icons/icon-clash.pngnew file mode 100644index 0000000..e3dd91eBinary files /dev/null and b/apps/web/public/icons/icon-clash.png differdiff --git a/apps/web/public/icons/icon-surge.png b/apps/web/public/icons/icon-surge.pngnew file mode 100644index 0000000..c4c7ef2Binary files /dev/null and b/apps/web/public/icons/icon-surge.png differdiff --git a/assets/neko-master-surge.png b/assets/neko-master-surge.pngnew file mode 100644index 0000000..a57464eBinary files /dev/null and b/assets/neko-master-surge.png differdiff --git a/docs/architecture.en.md b/docs/architecture.en.mdnew file mode 100644index 0000000..3a70765--- /dev/null+++ b/docs/architecture.en.md@@ -0,0 +1,547 @@+# Neko Master Architecture++## System Overview++```+โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ+โ Frontend Layer โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ Next.js 16 (App Router) โ โ+โ โ โโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ Dashboard โ โ Overview โ โ Charts โ โ Interactive Tables โ โ โ+โ โ โ โ โ โ โ โ โ โ โ โ+โ โ โโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ React Query (TanStack) โ โ โ+โ โ โ - API data fetching and caching โ โ โ+โ โ โ - Optimistic updates and state management โ โ โ+โ โ โ - Auto-retry and error handling โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ useStatsWebSocket Hook โ โ โ+โ โ โ - WebSocket connection management โ โ โ+โ โ โ - Real-time data subscription โ โ โ+โ โ โ - Auto-reconnect and heartbeat โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ+ โ+ โ HTTP / WebSocket+ โ (Cookie Authentication)+ โผ+โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ+โ Collector Layer โ+โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ API Server (Fastify) โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ REST API Endpoints โ โ โ+โ โ โ โโ /api/backends - Backend management โ โ โ+โ โ โ โโ /api/stats - Statistics โ โ โ+โ โ โ โโ /api/auth - Authentication โ โ โ+โ โ โ โโ /api/domains - Domain statistics โ โ โ+โ โ โ โโ /api/ips - IP statistics โ โ โ+โ โ โ โโ /api/proxies - Proxy statistics โ โ โ+โ โ โ โโ /api/rules - Rule statistics โ โ โ+โ โ โ โโ /api/devices - Device statistics โ โ โ+โ โ โ โโ /api/gateway/* - Gateway proxy APIs โ โ โ+โ โ โ โโ /api/retention - Data retention config โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ WebSocket Server (ws) โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ Real-time Data Push โ โ โ+โ โ โ - Connection management (Client Connections) โ โ โ+โ โ โ - Subscription management (Range, Backend, PushInterval) โ โ โ+โ โ โ - Broadcast push (Broadcast Stats) โ โ โ+โ โ โ - Summary cache (2s TTL) โ โ โ+โ โ โ - Policy cache sync (Surge Policy Sync) โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ Gateway Collector(s) โ โ+โ โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ Clash Collector โ โ Clash Collector โ โ Surge Collector โ โ โ+โ โ โ (Clash/Mihomo) โ โ (Clash/Mihomo) โ โ (Surge) โ โ โ+โ โ โ โ โ โ โ โ โ โ+โ โ โ WebSocket Client โ โ WebSocket Client โ โ HTTP REST Polling โ โ โ+โ โ โ โ (Real-time) โ โ โ (Real-time) โ โ โ (2s Poll) โ โ โ+โ โ โ Gateway WS โ โ Gateway WS โ โ HTTP API โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ โ+โ โ Features: โ โ+โ โ - Connection pool management (Reconnection) โ โ+โ โ - Traffic tracking (Traffic Tracking) โ โ+โ โ - Delta calculation (Delta Calculation) โ โ+โ โ - Batch buffering (Batch Buffer, 30s flush) โ โ+โ โ - Policy cache sync (Policy Cache Sync, 10min) โ โ+โ โ - Exponential backoff retry โ โ+โ โ - GeoIP query integration โ โ+โ โ - Real-time data broadcast (โ RealtimeStore) โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ Policy Sync Service โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ Surge Policy Cache Sync (Background Task) โ โ โ+โ โ โ - Scheduled sync: /v1/policies + /v1/policy_groups/select โ โ โ+โ โ โ - Storage: surge_policy_cache table โ โ โ+โ โ โ - Cache TTL: 10 minutes โ โ โ+โ โ โ - Fallback: Direct API fetch โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ RealtimeStore (In-Memory Real-time Data) โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ Real-time delta cache (isolated by backend) โ โ โ+โ โ โ - summaryByBackend: Total traffic delta โ โ โ+โ โ โ - minuteByBackend: Minute-level traffic buckets โ โ โ+โ โ โ - domainByBackend: Domain deltas โ โ โ+โ โ โ - ipByBackend: IP deltas โ โ โ+โ โ โ - proxyByBackend: Proxy deltas โ โ โ+โ โ โ - deviceByBackend: Device deltas โ โ โ+โ โ โ - ruleByBackend: Rule deltas โ โ โ+โ โ โ - countryByBackend: Country/region deltas โ โ โ+โ โ โ โ โ โ+โ โ โ Functions: merge* methods merge DB data with real-time deltas โ โ โ+โ โ โ - mergeTopDomains() โ โ โ+โ โ โ - mergeProxyStats() โ โ โ+โ โ โ - mergeTrend() โ โ โ+โ โ โ - applySummaryDelta() โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ GeoIP Service โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ IP Geolocation Query โ โ โ+โ โ โ - IP-API.com (default) โ โ โ+โ โ โ - IPInfo.io (fallback) โ โ โ+โ โ โ - Local cache (LRU) โ โ โ+โ โ โ - Batch query optimization โ โ โ+โ โ โ - Failure cooldown mechanism โ โ โ+โ โ โ - IPv4/IPv6 dual-stack support โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ+ โ+ โ SQLite (better-sqlite3)+ โผ+โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ+โ Storage Layer โ+โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ SQLite Database (WAL Mode) โ โ+โ โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ Statistics Tables (partitioned by backend_id) โ โ โ+โ โ โ โ โ โ+โ โ โ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โ โ โ+โ โ โ โ domain_stats โ โ ip_stats โ โ proxy_stats โ โ โ โ+โ โ โ โ โ โ โ โ โ โ โ โ+โ โ โ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โ โ โ+โ โ โ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โ โ โ+โ โ โ โ rule_stats โ โ country_stats โ โ device_stats โ โ โ โ+โ โ โ โ โ โ โ โ โ โ โ โ+โ โ โ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โ โ โ+โ โ โ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โ โ โ+โ โ โ โminute_stats โ โ hourly_stats โ โ daily_stats โ โ โ โ+โ โ โ โ โ โ โ โ โ โ โ โ+โ โ โ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โ โ โ+โ โ โ โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ Configuration & Cache Tables โ โ โ+โ โ โ โ โ โ+โ โ โ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โ โ โ+โ โ โ โ backend_configsโ โ geoip_cache โ โ asn_cache โ โ โ โ+โ โ โ โ โ โ โ โ โ โ โ โ+โ โ โ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โ โ โ+โ โ โ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โ โ โ+โ โ โ โ auth_config โ โ retention_configโ โsurge_policy_cacheโ โ โ โ+โ โ โ โ โ โ โ โ โ โ โ โ+โ โ โ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ+ โฒ+ โ+โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ+โ Data Sources โ+โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ Clash / Mihomo Gateway โ โ Surge Gateway (v5+) โ โ+โ โ โ โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ /connections WebSocket โ โ โ โ HTTP REST API Endpoints โ โ โ+โ โ โ (Real-time Push) โ โ โ โ โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ โ โโ GET /v1/requests/recent โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ โ โ (Recent connections, 2s poll) โ โ โ+โ โ โ /traffic โ โ โ โ โโ GET /v1/policies โ โ โ+โ โ โ /rules โ โ โ โ โ (Policy list) โ โ โ+โ โ โ /proxies โ โ โ โ โโ GET /v1/policy_groups/select โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ โ โ (Policy group details) โ โ โ+โ โ โ โ โ โ โ โ+โ โ Connection Format: โ โ โ Enable: โ โ โ+โ โ { โ โ โ http-api = 127.0.0.1:9091 โ โ โ+โ โ "connections": [{ โ โ โ โ โ โ+โ โ "id": "uuid", โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ "metadata": { โ โ โ โ+โ โ "host": "example.com", โ โ Note: DNS resolved on proxy server, โ โ+โ โ "destinationIP": "1.2.3", โ โ cannot get real landing IP โ โ+โ โ "sourceIP": "192.168.x", โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ "chains": [...], โ โ+โ โ "rule": "RuleSet", โ โ+โ โ "upload": 1024, โ โ+โ โ "download": 2048 โ โ+โ โ }] โ โ+โ โ } โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ+```++---++## Data Flow++### 1. Clash Data Collection Flow++```+Clash Gateway+ โ+ โ WebSocket Push+ โ (per second / per connection update)+ โผ+ClashCollector+ โ+ โโโ 1. Traffic Tracking+ โ - Identify new/updated connections+ โ - Calculate delta (delta = current - last)+ โ - Extract proxy chains+ โ+ โโโ 2. Batch Buffer+ โ - Aggregate by domain+ip+chain+rule+ โ - 30s flush or 5000 entries trigger+ โ+ โโโ 3. GeoIP Query+ โ - Async IP geolocation query+ โ - Batch cache optimization+ โ+ โโโ 4. Data Write+ โ โโโโ SQLite (Persistence)+ โ โโโโ RealtimeStore (In-Memory)+ โ+ โโโ 5. Trigger Broadcast+ โโโโ WebSocketServer.broadcastStats()+```++### 2. Surge Data Collection Flow++```+Surge Gateway+ โ+ โ HTTP Polling (2s interval)+ โ GET /v1/requests/recent+ โผ+SurgeCollector+ โ+ โโโ 1. Request Handling+ โ - Exponential backoff retry (max 5)+ โ - Error handling and logging+ โ+ โโโ 2. Traffic Tracking+ โ - Use recentlyCompleted Map to prevent duplicates+ โ - Calculate delta (delta = current - last)+ โ - Extract policy path from notes+ โ+ โโโ 3. Policy Cache Sync+ โ - Background policy config sync+ โ - 10-minute scheduled update+ โ+ โโโ 4. Batch Buffer+ โ - Same aggregation logic as Clash+ โ - 30s flush+ โ+ โโโ 5. Data Write+ โ โโโโ SQLite (Persistence)+ โ โโโโ RealtimeStore (In-Memory)+ โ+ โโโ 6. Trigger Broadcast+ โโโโ WebSocketServer.broadcastStats()+```++### 3. Real-time Data Push Flow++```+WebSocketServer+ โ+ โโโ Client Connection+ โ - Cookie authentication verification+ โ - Establish ClientInfo (backend, range, interval)+ โ+ โโโ Data Preparation+ โ - Query base data from SQLite+ โ - RealtimeStore.merge*() merge real-time deltas+ โ - Summary cache (2s TTL)+ โ+ โโโ Broadcast Push+ โโโโ Full push (stats)+ โโโโ Trend data (trend)+ โโโโ Domain pagination (domains)+ โโโโ IP pagination (ips)+```++### 4. Frontend Data Fetching Flow++```+React Component+ โ+ โโโ Method 1: React Query (HTTP API)+ โ useTrafficTrend() โโโ api.getTrafficTrendAggregated()+ โ โ+ โ โผ+ โ Collector API+ โ+ โโโ Method 2: WebSocket Hook+ useStatsWebSocket() โโโ WebSocket Connection+ โ+ โโโ Subscribe message (subscribe)+ โโโ Receive push (stats/pong)+ โโโ Auto-reconnect+```++---++## Key Design Patterns++### Repository Pattern (Database Layer)++```+StatsDatabase (Facade Class, ~1000 lines)+ โ+ โโโ init() / migrations โโ DDL table creation, migration logic (kept in db.ts)+ โโโ getSummary() โโ Cross-table aggregation query (kept in db.ts)+ โ+ โโโ this.repos โโ Repository instance composition+ โโโ trafficWriter โโ Traffic writing (updateTrafficStats, batch)+ โโโ domain โโ Domain stats (6 methods)+ โโโ ip โโ IP stats + ASN/GeoIP (17 methods)+ โโโ rule โโ Rule stats + ChainFlow (10 methods)+ โโโ proxy โโ Proxy stats (3 methods)+ โโโ device โโ Device stats (3 methods)+ โโโ country โโ Country/region (3 methods)+ โโโ timeseries โโ Time-series data (5 methods)+ โโโ config โโ DB config/cleanup (12 methods)+ โโโ backend โโ Backend management (11 methods)+ โโโ auth โโ Auth config (2 methods)+ โโโ surge โโ Surge policy cache (4 methods)++BaseRepository (Abstract Base Class)+ โ+ โโโ parseMinuteRange() โโ Time range parsing+ โโโ splitChainParts() โโ Proxy chain parsing+ โโโ expandShortChainsForRules() โโ Rule chain expansion+ โโโ aggregateProxyStatsByFirstHop()+ โโโ allocateByWeights() โโ Weight-based allocation+ โโโ ... 13 shared utility methods total+```++### Dual Write Pattern++```+Collector receives traffic data+ โ+ โโโโ SQLite (Persistent Storage)+ โ โโ For historical queries, aggregate statistics+ โ+ โโโโ RealtimeStore (In-Memory Real-time)+ โโ For real-time display, low-latency push+```++### Delta Merge Pattern++```+Client requests stats+ โ+ โผ+Database base data (DB)+ โ+ โโโ Domain stats: 100MB+ โโโ Proxy stats: 50MB+ โโโ ...+ โ+ โผ+RealtimeStore.merge*(DB data, memory delta)+ โ+ โโโ Merged data (DB + real-time delta)+ Example: 100MB + 5MB (last 30 seconds)+```++### Multi-Backend Isolation++```+All data structures isolated by backendId:++SQLite: PRIMARY KEY (backend_id, domain)+RealtimeStore: Map<backendId, Map<key, Data>>+WebSocket: ClientInfo.backendId++Support mixed backends:+โโโ Backend #1: Clash (WebSocket real-time)+โโโ Backend #2: Clash (WebSocket real-time)+โโโ Backend #3: Surge (HTTP polling)+```++### Duplicate Prevention Pattern (Surge)++```+SurgeCollector+ โ+ โโโ recentlyCompleted Map+ โ key: requestId+ โ value: { finalUpload, finalDownload, completedAt }+ โ TTL: 5 minutes+ โ+ โโโ Processing logic+ โโโ New connection โ Record initial state+ โโโ Updating โ Calculate delta+ โโโ Completed โ Check Map, prevent duplicate counting+```++---++## Authentication Flow (Cookie-Based)++```+User Login+ โ+ โผ+POST /api/auth/login+ โ+ โผ+Server Verification โ Set HttpOnly Cookie (auth-token)+ โ+ โผ+Subsequent requests automatically carry Cookie+ โ+ โโโโ HTTP API Authentication+ โโโโ WebSocket Connection Authentication (req.headers.cookie)+```++---++## Performance Optimization++| Layer | Optimization Technique | Effect |+| ------------ | ------------------------------------- | --------------------------- |+| **Collect** | Batch Write (Batch Buffer) | Reduce 90% DB writes |+| **Collect** | GeoIP Batch Query + Cache | Reduce 80% external requests|+| **Collect** | Policy Cache Sync | Reduce 95% API calls |+| **Collect** | Exponential Backoff Retry | Improve connection stability|+| **Query** | RealtimeStore Delta Merge | Real-time data < 100ms |+| **Query** | WebSocket Summary Cache (2s TTL) | Reduce 70% DB queries |+| **Storage** | SQLite WAL Mode | Concurrent read/write |+| **Storage** | Data Retention Policy (Auto-cleanup) | Control storage growth |++---++## Responsive Layout Strategy++```+Dashboard Layout (Tailwind CSS)+ โ+ โโโ Single Column (Default)+ โ grid-cols-1+ โ โโ TOP DOMAINS full width, show more data+ โ+ โโโ Two Columns (md: 768px+)+ โ md:grid-cols-2+ โ โโ Cards auto-arrange+ โ+ โโโ Three Columns (xl: 1280px+)+ xl:grid-cols-3+ โโ Three columns side-by-side, narrow cards use vertical layout++Container Queries (@container)+ โ+ โโโ @min-[200px]: Narrow card vertical layout+ โโโ @min-[500px]: Wide card horizontal layout, show more data+```++---++## Project Directory Structure++```+neko-master/+โโโ apps/+โ โโโ web/ # Next.js Frontend+โ โ โโโ app/ # App Router+โ โ โโโ components/+โ โ โ โโโ features/ # Feature components+โ โ โ โ โโโ backend/ # Backend config+โ โ โ โ โโโ devices/ # Device stats+โ โ โ โ โโโ domains/ # Domain stats+โ โ โ โ โโโ proxies/ # Proxy stats+โ โ โ โ โโโ rules/ # Rule stats+โ โ โ โ โโโ stats/ # Stats charts+โ โ โ โโโ layout/ # Layout components+โ โ โ โโโ ui/ # Base UI components+โ โ โโโ hooks/api/ # API Hooks+โ โ โโโ lib/ # Utilities+โ โ โโโ messages/ # i18n translations+โ โ+โ โโโ collector/ # Fastify Data Collection Service+โ โโโ src/+โ โ โโโ collectors/ # Collector implementations+โ โ โ โโโ clash-collector.ts # Clash WebSocket+โ โ โ โโโ surge-collector.ts # Surge HTTP+โ โ โโโ database/+โ โ โ โโโ repositories/ # Repository Pattern+โ โ โ โโโ base.repository.ts # Abstract base (13 shared utils)+โ โ โ โโโ domain.repository.ts # Domain stats+โ โ โ โโโ ip.repository.ts # IP stats + ASN/GeoIP+โ โ โ โโโ rule.repository.ts # Rule stats + ChainFlow+โ โ โ โโโ proxy.repository.ts # Proxy stats+โ โ โ โโโ device.repository.ts # Device stats+โ โ โ โโโ country.repository.ts # Country/region stats+โ โ โ โโโ timeseries.repository.ts # Time-series data+โ โ โ โโโ traffic-writer.repository.ts # Traffic writing+โ โ โ โโโ config.repository.ts # DB config/cleanup+โ โ โ โโโ backend.repository.ts # Backend management+โ โ โ โโโ auth.repository.ts # Auth config+โ โ โ โโโ surge.repository.ts # Surge policy cache+โ โ โ โโโ index.ts # Barrel exports+โ โ โโโ db.ts # Facade (~1000 lines, DDL + delegates)+โ โ โโโ modules/ # Business modules+โ โ โ โโโ auth/ # Authentication+โ โ โ โโโ backend/ # Backend management+โ โ โ โโโ stats/ # Stats service+โ โ โ โโโ surge/ # Surge service+โ โ โ โโโ realtime/ # Real-time data+โ โ โ โโโ websocket/ # WebSocket+โ โ โโโ routes/ # API routes+โ โ โโโ services/ # Base services+โ โ โ โโโ geoip.ts # GeoIP service+โ โ โ โโโ policy-sync.ts # Policy sync+โ โ โ โโโ realtime-store.ts # Real-time data store+โ โ โโโ index.ts # Entry point+โ โโโ package.json+โ+โโโ packages/+โ โโโ shared/ # Shared type definitions+โ+โโโ docs/+โ โโโ architecture.md # This doc (Chinese)+โ โโโ architecture.en.md # This doc (English)+โ+โโโ docker-compose.yml+```diff --git a/docs/architecture.md b/docs/architecture.mdnew file mode 100644index 0000000..d7b94ae--- /dev/null+++ b/docs/architecture.md@@ -0,0 +1,546 @@+# Neko Master ไธๅกๆถๆๅพ++## ๆดไฝๆถๆๆฆ่ง++```+โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ+โ ๅ็ซฏๅฑ็คบๅฑ (Frontend) โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ Next.js 16 (App Router) โ โ+โ โ โโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ Dashboard โ โ Overview โ โ Charts โ โ Interactive Tables โ โ โ+โ โ โ (ไปช่กจๆฟ) โ โ (ๆฆ่ง) โ โ (ๅพ่กจ) โ โ (ไบคไบๅผ่กจๆ ผ) โ โ โ+โ โ โโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ React Query (TanStack) โ โ โ+โ โ โ - API ๆฐๆฎ่ทๅไธ็ผๅญ โ โ โ+โ โ โ - ไน่งๆดๆฐไธ็ถๆ็ฎก็ โ โ โ+โ โ โ - ่ชๅจ้่ฏไธ้่ฏฏๅค็ โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ useStatsWebSocket Hook โ โ โ+โ โ โ - WebSocket ่ฟๆฅ็ฎก็ โ โ โ+โ โ โ - ๅฎๆถๆฐๆฎ่ฎข้ โ โ โ+โ โ โ - ่ชๅจ้่ฟไธๅฟ่ทณ โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ+ โ+ โ HTTP / WebSocket+ โ (Cookie ่ฎค่ฏ)+ โผ+โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ+โ ๆๅกๅฑ (Collector) โ+โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ API Server (Fastify) โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ REST API Endpoints โ โ โ+โ โ โ โโ /api/backends - ๅ็ซฏ็ฎก็ โ โ โ+โ โ โ โโ /api/stats - ็ป่ฎกๆฐๆฎ โ โ โ+โ โ โ โโ /api/auth - ่ฎค่ฏ็ฎก็ โ โ โ+โ โ โ โโ /api/domains - ๅๅ็ป่ฎก โ โ โ+โ โ โ โโ /api/ips - IP ็ป่ฎก โ โ โ+โ โ โ โโ /api/proxies - ไปฃ็็ป่ฎก โ โ โ+โ โ โ โโ /api/rules - ่งๅ็ป่ฎก โ โ โ+โ โ โ โโ /api/devices - ่ฎพๅค็ป่ฎก โ โ โ+โ โ โ โโ /api/gateway/* - ็ฝๅ ณไปฃ็ๆฅๅฃ โ โ โ+โ โ โ โโ /api/retention - ๆฐๆฎไฟ็้ ็ฝฎ โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ WebSocket Server (ws) โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ ๅฎๆถๆฐๆฎๆจ้ โ โ โ+โ โ โ - ่ฟๆฅ็ฎก็ (Client Connections) โ โ โ+โ โ โ - ่ฎข้ ็ฎก็ (Range, Backend, PushInterval) โ โ โ+โ โ โ - ๅนฟๆญๆจ้ (Broadcast Stats) โ โ โ+โ โ โ - ๆ่ฆ็ผๅญ (Summary Cache, 2s TTL) โ โ โ+โ โ โ - ็ญ็ฅ็ผๅญๅๆญฅ (Surge Policy Sync) โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ Gateway Collector(s) โ โ+โ โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ Clash Collector โ โ Clash Collector โ โ Surge Collector โ โ โ+โ โ โ (Clash/Mihomo) โ โ (Clash/Mihomo) โ โ (Surge) โ โ โ+โ โ โ โ โ โ โ โ โ โ+โ โ โ WebSocket Client โ โ WebSocket Client โ โ HTTP REST Polling โ โ โ+โ โ โ โ (ๅฎๆถ) โ โ โ (ๅฎๆถ) โ โ โ (2s ่ฝฎ่ฏข) โ โ โ+โ โ โ Gateway WS โ โ Gateway WS โ โ HTTP API โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ โ+โ โ ๅ่ฝ๏ผ โ โ+โ โ - ่ฟๆฅๆฑ ็ฎก็ (Reconnection) โ โ+โ โ - ๆต้่ฟฝ่ธช (Traffic Tracking) โ โ+โ โ - ๅข้่ฎก็ฎ (Delta Calculation) โ โ+โ โ - ๆน้็ผๅฒ (Batch Buffer, 30s flush) โ โ+โ โ - ็ญ็ฅ็ผๅญๅๆญฅ (Policy Cache Sync, 10min) โ โ+โ โ - ๆๆฐ้้ฟ้่ฏ (Exponential Backoff) โ โ+โ โ - GeoIP ๆฅ่ฏข้ๆ โ โ+โ โ - ๅฎๆถๆฐๆฎๅนฟๆญ (โ RealtimeStore) โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ Policy Sync Service โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ Surge ็ญ็ฅ็ผๅญๅๆญฅ (่ๆฏไปปๅก) โ โ โ+โ โ โ - ๅฎๆถๅๆญฅ: /v1/policies + /v1/policy_groups/select โ โ โ+โ โ โ - ๅญๅจ: surge_policy_cache ่กจ โ โ โ+โ โ โ - ็ผๅญ่ฟๆ: 10 ๅ้ โ โ โ+โ โ โ - ๅ้ๆบๅถ: ็ดๆฅ API ่ทๅ โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ RealtimeStore (ๅ ๅญๅฎๆถๆฐๆฎ) โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ ๅฎๆถๅข้็ผๅญ (ๆ backend ้็ฆป) โ โ โ+โ โ โ - summaryByBackend: ๆปไฝๆต้ๅข้ โ โ โ+โ โ โ - minuteByBackend: ๅ้็บงๆต้ๆกถ โ โ โ+โ โ โ - domainByBackend: ๅๅๅข้ โ โ โ+โ โ โ - ipByBackend: IP ๅข้ โ โ โ+โ โ โ - proxyByBackend: ไปฃ็ๅข้ โ โ โ+โ โ โ - deviceByBackend: ่ฎพๅคๅข้ โ โ โ+โ โ โ - ruleByBackend: ่งๅๅข้ โ โ โ+โ โ โ - countryByBackend: ๅฝๅฎถ/ๅฐๅบๅข้ โ โ โ+โ โ โ โ โ โ+โ โ โ ๅ่ฝ๏ผmerge* ๆนๆณๅๅนถๆฐๆฎๅบๆฐๆฎไธๅฎๆถๅข้ โ โ โ+โ โ โ - mergeTopDomains() โ โ โ+โ โ โ - mergeProxyStats() โ โ โ+โ โ โ - mergeTrend() โ โ โ+โ โ โ - applySummaryDelta() โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ GeoIP Service โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ IP ๅฐ็ไฝ็ฝฎๆฅ่ฏข โ โ โ+โ โ โ - IP-API.com (้ป่ฎค) โ โ โ+โ โ โ - IPInfo.io (ๅค้) โ โ โ+โ โ โ - ๆฌๅฐ็ผๅญ (LRU) โ โ โ+โ โ โ - ๆน้ๆฅ่ฏขไผๅ โ โ โ+โ โ โ - ๅคฑ่ดฅๅทๅดๆบๅถ โ โ โ+โ โ โ - IPv4/IPv6 ๅๆ ๆฏๆ โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ+ โ+ โ SQLite (better-sqlite3)+ โผ+โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ+โ ๆฐๆฎๅญๅจๅฑ (Storage) โ+โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ SQLite Database (WAL Mode) โ โ+โ โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ ็ป่ฎก่กจ (ๆ backend_id ๅๅบ) โ โ โ+โ โ โ โ โ โ+โ โ โ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โ โ โ+โ โ โ โ domain_stats โ โ ip_stats โ โ proxy_stats โ โ โ โ+โ โ โ โ (ๅๅ็ป่ฎก) โ โ (IP ็ป่ฎก) โ โ (ไปฃ็็ป่ฎก) โ โ โ โ+โ โ โ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โ โ โ+โ โ โ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โ โ โ+โ โ โ โ rule_stats โ โ country_stats โ โ device_stats โ โ โ โ+โ โ โ โ (่งๅ็ป่ฎก) โ โ (ๅฝๅฎถ็ป่ฎก) โ โ (่ฎพๅค็ป่ฎก) โ โ โ โ+โ โ โ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โ โ โ+โ โ โ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โ โ โ+โ โ โ โminute_stats โ โ hourly_stats โ โ daily_stats โ โ โ โ+โ โ โ โ(ๅ้็บง่ฏฆ็ป) โ โ (ๅฐๆถ็บง่ๅ) โ โ (ๆฅ็บง่ๅ) โ โ โ โ+โ โ โ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โ โ โ+โ โ โ โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ ้ ็ฝฎไธ็ผๅญ่กจ โ โ โ+โ โ โ โ โ โ+โ โ โ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โ โ โ+โ โ โ โ backend_configsโ โ geoip_cache โ โ asn_cache โ โ โ โ+โ โ โ โ (ๅ็ซฏ้ ็ฝฎ) โ โ (GeoIP ็ผๅญ) โ โ (ASN ็ผๅญ) โ โ โ โ+โ โ โ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โ โ โ+โ โ โ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โ โ โ+โ โ โ โ auth_config โ โ retention_configโ โsurge_policy_cacheโ โ โ โ+โ โ โ โ (่ฎค่ฏ้ ็ฝฎ) โ โ (ไฟ็็ญ็ฅ) โ โ(Surge็ญ็ฅ็ผๅญ) โ โ โ โ+โ โ โ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโ โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ+ โฒ+ โ+โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ+โ ๆฐๆฎๆบๅฑ (Data Sources) โ+โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ Clash / Mihomo Gateway โ โ Surge Gateway (v5+) โ โ+โ โ โ โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ โ /connections WebSocket โ โ โ โ HTTP REST API Endpoints โ โ โ+โ โ โ (ๅฎๆถๆจ้) โ โ โ โ โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ โ โโ GET /v1/requests/recent โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ โ โ (ๆ่ฟ่ฟๆฅ๏ผ2s ่ฝฎ่ฏข) โ โ โ+โ โ โ /traffic โ โ โ โ โโ GET /v1/policies โ โ โ+โ โ โ /rules โ โ โ โ โ (็ญ็ฅๅ่กจ) โ โ โ+โ โ โ /proxies โ โ โ โ โโ GET /v1/policy_groups/select โ โ โ+โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ โ โ (็ญ็ฅ็ป่ฏฆๆ ) โ โ โ+โ โ โ โ โ โ โ โ+โ โ ่ฟๆฅๆ ผๅผ๏ผ โ โ โ ๅผๅฏๆนๆณ๏ผ โ โ โ+โ โ { โ โ โ http-api = 127.0.0.1:9091 โ โ โ+โ โ "connections": [{ โ โ โ โ โ โ+โ โ "id": "uuid", โ โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ+โ โ "metadata": { โ โ โ โ+โ โ "host": "example.com", โ โ ๆณจๆ๏ผDNS ๅจไปฃ็ๆๅกๅจ่งฃๆ๏ผ โ โ+โ โ "destinationIP": "1.2.3", โ โ ๆ ๆณ่ทๅ็ๅฎ่ฝๅฐ IP โ โ+โ โ "sourceIP": "192.168.x", โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โ โ "chains": [...], โ โ+โ โ "rule": "RuleSet", โ โ+โ โ "upload": 1024, โ โ+โ โ "download": 2048 โ โ+โ โ }] โ โ+โ โ } โ โ+โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ+โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ+```++---++## ๆฐๆฎๆต่ฏฆ่งฃ++### 1. Clash ๆฐๆฎ้้ๆต็จ++```+Clash Gateway+ โ+ โ WebSocket ๆจ้+ โ (ๆฏ็ง/ๆฏ่ฟๆฅๆดๆฐ)+ โผ+ClashCollector+ โ+ โโโ 1. ๆต้่ฟฝ่ธช (Traffic Tracking)+ โ - ่ฏๅซๆฐ่ฟๆฅ/ๆดๆฐ่ฟๆฅ+ โ - ่ฎก็ฎๅข้ (delta = current - last)+ โ - ๆๅไปฃ็้พ (chains)+ โ+ โโโ 2. ๆน้็ผๅฒ (Batch Buffer)+ โ - ๆ domain+ip+chain+rule ่ๅ+ โ - 30s ๅฎๆถ flush ๆ 5000 ๆก่งฆๅ+ โ+ โโโ 3. GeoIP ๆฅ่ฏข+ โ - ๅผๆญฅๆฅ่ฏข IP ๅฐ็ไฝ็ฝฎ+ โ - ๆน้็ผๅญไผๅ+ โ+ โโโ 4. ๆฐๆฎๅๅ ฅ+ โ โโโโ SQLite (ๆไน ๅ)+ โ โโโโ RealtimeStore (ๅ ๅญๅฎๆถ)+ โ+ โโโ 5. ่งฆๅๅนฟๆญ+ โโโโ WebSocketServer.broadcastStats()+```++### 2. Surge ๆฐๆฎ้้ๆต็จ++```+Surge Gateway+ โ+ โ HTTP ่ฝฎ่ฏข (2s ้ด้)+ โ GET /v1/requests/recent+ โผ+SurgeCollector+ โ+ โโโ 1. ่ฏทๆฑๅค็+ โ - ๆๆฐ้้ฟ้่ฏ (ๆๅค 5 ๆฌก)+ โ - ้่ฏฏๅค็ๅๆฅๅฟ่ฎฐๅฝ+ โ+ โโโ 2. ๆต้่ฟฝ่ธช (Traffic Tracking)+ โ - ไฝฟ็จ recentlyCompleted Map ้ฒ้ๅค+ โ - ่ฎก็ฎๅข้ (delta = current - last)+ โ - ไป notes ๆๅ็ญ็ฅ่ทฏๅพ+ โ+ โโโ 3. ็ญ็ฅ็ผๅญๅๆญฅ+ โ - ๅๅฐๅๆญฅ็ญ็ฅ้ ็ฝฎ+ โ - 10 ๅ้ๅฎๆถๆดๆฐ+ โ+ โโโ 4. ๆน้็ผๅฒ+ โ - ไธ Clash ็ธๅ็่ๅ้ป่พ+ โ - 30s ๅฎๆถ flush+ โ+ โโโ 5. ๆฐๆฎๅๅ ฅ+ โ โโโโ SQLite (ๆไน ๅ)+ โ โโโโ RealtimeStore (ๅ ๅญๅฎๆถ)+ โ+ โโโ 6. ่งฆๅๅนฟๆญ+ โโโโ WebSocketServer.broadcastStats()+```++### 3. ๅฎๆถๆฐๆฎๆจ้ๆต็จ++```+WebSocketServer+ โ+ โโโ ๅฎขๆท็ซฏ่ฟๆฅ+ โ - Cookie ่ฎค่ฏ้ช่ฏ+ โ - ๅปบ็ซ ClientInfo (backend, range, interval)+ โ+ โโโ ๆฐๆฎๅๅค+ โ - ไป SQLite ๆฅ่ฏขๅบ็กๆฐๆฎ+ โ - RealtimeStore.merge*() ๅๅนถๅฎๆถๅข้+ โ - ๆ่ฆ็ผๅญ (2s TTL)+ โ+ โโโ ๅนฟๆญๆจ้+ โโโโ ๅ จ้ๆจ้ (stats)+ โโโโ ่ถๅฟๆฐๆฎ (trend)+ โโโโ ๅๅๅ้กต (domains)+ โโโโ IP ๅ้กต (ips)+```++### 4. ๅ็ซฏๆฐๆฎ่ทๅๆต็จ++```+React Component+ โ+ โโโ ๆนๅผ 1: React Query (HTTP API)+ โ useTrafficTrend() โโโ api.getTrafficTrendAggregated()+ โ โ+ โ โผ+ โ Collector API+ โ+ โโโ ๆนๅผ 2: WebSocket Hook+ useStatsWebSocket() โโโ WebSocket ่ฟๆฅ+ โ+ โโโ ่ฎข้ ๆถๆฏ (subscribe)+ โโโ ๆฅๆถๆจ้ (stats/pong)+ โโโ ่ชๅจ้่ฟ+```++---++## ๅ ณ้ฎ่ฎพ่ฎกๆจกๅผ++### Repository ๆจกๅผ (Database Layer)++```+StatsDatabase (้จ้ข็ฑป, ~1000 ่ก)+ โ+ โโโ init() / migrations โโ DDL ๅปบ่กจใ่ฟ็งป้ป่พ (ไฟ็ๅจ db.ts)+ โโโ getSummary() โโ ่ทจ่กจ่ๅๆฅ่ฏข (ไฟ็ๅจ db.ts)+ โ+ โโโ this.repos โโ Repository ๅฎไพ็ปๅ+ โโโ trafficWriter โโ ๆต้ๅๅ ฅ (updateTrafficStats, batch)+ โโโ domain โโ ๅๅ็ป่ฎก (6 ๆนๆณ)+ โโโ ip โโ IP ็ป่ฎก + ASN/GeoIP (17 ๆนๆณ)+ โโโ rule โโ ่งๅ็ป่ฎก + ChainFlow (10 ๆนๆณ)+ โโโ proxy โโ ไปฃ็็ป่ฎก (3 ๆนๆณ)+ โโโ device โโ ่ฎพๅค็ป่ฎก (3 ๆนๆณ)+ โโโ country โโ ๅฝๅฎถ/ๅฐๅบ (3 ๆนๆณ)+ โโโ timeseries โโ ๆถๅบๆฐๆฎ (5 ๆนๆณ)+ โโโ config โโ ๆฐๆฎๅบ้ ็ฝฎ/ๆธ ็ (12 ๆนๆณ)+ โโโ backend โโ ๅ็ซฏ็ฎก็ (11 ๆนๆณ)+ โโโ auth โโ ่ฎค่ฏ้ ็ฝฎ (2 ๆนๆณ)+ โโโ surge โโ Surge ็ญ็ฅ็ผๅญ (4 ๆนๆณ)++BaseRepository (ๆฝ่ฑกๅบ็ฑป)+ โ+ โโโ parseMinuteRange() โโ ๆถ้ด่ๅด่งฃๆ+ โโโ splitChainParts() โโ ไปฃ็้พ่งฃๆ+ โโโ expandShortChainsForRules() โโ ่งๅ้พๅฑๅผ+ โโโ aggregateProxyStatsByFirstHop()+ โโโ allocateByWeights() โโ ๆๆ้ๅ้ + โโโ ... ๅ ฑ 13 ไธชๅ ฑไบซๅทฅๅ ทๆนๆณ+```++### ๅๅๆจกๅผ (Dual Write)++```+Collector ๆถๅฐๆต้ๆฐๆฎ+ โ+ โโโโ SQLite (ๆไน ๅๅญๅจ)+ โ โโ ็จไบๅๅฒๆฅ่ฏขใ่ๅ็ป่ฎก+ โ+ โโโโ RealtimeStore (ๅ ๅญๅฎๆถ)+ โโ ็จไบๅฎๆถๅฑ็คบใไฝๅปถ่ฟๆจ้+```++### ๅข้ๅๅนถๆจกๅผ (Delta Merge)++```+ๅฎขๆท็ซฏ่ฏทๆฑ stats+ โ+ โผ+ๆฐๆฎๅบๅบ็กๆฐๆฎ (DB)+ โ+ โโโ ๅๅ็ป่ฎก: 100MB+ โโโ ไปฃ็็ป่ฎก: 50MB+ โโโ ...+ โ+ โผ+RealtimeStore.merge*(DBๆฐๆฎ, ๅ ๅญๅข้)+ โ+ โโโ ๅๅนถๅๆฐๆฎ (DB + ๅฎๆถๅข้)+ ไพๅฆ: 100MB + 5MB (ๆ่ฟ30็ง)+```++### ๅคๅ็ซฏ้็ฆป (Multi-Backend Isolation)++```+ๆๆๆฐๆฎ็ปๆๆ backendId ้็ฆป:++SQLite: PRIMARY KEY (backend_id, domain)+RealtimeStore: Map<backendId, Map<key, Data>>+WebSocket: ClientInfo.backendId++ๆฏๆๆททๅๅ็ซฏ:+โโโ Backend #1: Clash (WebSocket ๅฎๆถ)+โโโ Backend #2: Clash (WebSocket ๅฎๆถ)+โโโ Backend #3: Surge (HTTP ่ฝฎ่ฏข)+```++### ้ฒ้ๅค่ฎก็ฎๆจกๅผ (Surge)++```+SurgeCollector+ โ+ โโโ recentlyCompleted Map+ โ key: requestId+ โ value: { finalUpload, finalDownload, completedAt }+ โ TTL: 5 ๅ้+ โ+ โโโ ๅค็้ป่พ+ โโโ ๆฐ่ฟๆฅ โ ่ฎฐๅฝๅๅง็ถๆ+ โโโ ๆดๆฐไธญ โ ่ฎก็ฎๅข้+ โโโ ๅทฒๅฎๆ โ ๆฃๆฅ Map๏ผ้ฒๆญข้ๅค่ฎกๅ ฅ+```++---++## ่ฎค่ฏๆต็จ (Cookie-Based)++```+็จๆท็ปๅฝ+ โ+ โผ+POST /api/auth/login+ โ+ โผ+ๆๅก็ซฏ้ช่ฏ โ ่ฎพ็ฝฎ HttpOnly Cookie (auth-token)+ โ+ โผ+ๅ็ปญ่ฏทๆฑ่ชๅจๆบๅธฆ Cookie+ โ+ โโโโ HTTP API ่ฎค่ฏ+ โโโโ WebSocket ่ฟๆฅ่ฎค่ฏ (req.headers.cookie)+```++---++## ๆง่ฝไผๅ็ญ็ฅ++| ๅฑ้ข | ไผๅๆๆฎต | ๆๆ |+| -------- | ------------------------------ | ------------------ |+| **้้** | ๆน้ๅๅ ฅ (Batch Buffer) | ๅๅฐ 90% DB ๅๅ ฅ |+| **้้** | GeoIP ๆน้ๆฅ่ฏข + ็ผๅญ | ๅๅฐ 80% ๅค้จ่ฏทๆฑ |+| **้้** | ็ญ็ฅ็ผๅญๅๆญฅ (Policy Cache) | ๅๅฐ 95% API ่ฐ็จ |+| **้้** | ๆๆฐ้้ฟ้่ฏ | ๆ้ซ่ฟๆฅ็จณๅฎๆง |+| **ๆฅ่ฏข** | RealtimeStore ๅข้ๅๅนถ | ๅฎๆถๆฐๆฎ < 100ms |+| **ๆฅ่ฏข** | WebSocket ๆ่ฆ็ผๅญ (2s TTL) | ๅๅฐ 70% DB ๆฅ่ฏข |+| **ๅญๅจ** | SQLite WAL Mode | ๅนถๅ่ฏปๅไผๅ |+| **ๅญๅจ** | ๆฐๆฎไฟ็็ญ็ฅ (่ชๅจๆธ ็) | ๆงๅถๅญๅจๅข้ฟ |++---++## ๅๅบๅผๅธๅฑ็ญ็ฅ++```+Dashboard ๅธๅฑ (Tailwind CSS)+ โ+ โโโ ๅๅๅธๅฑ (้ป่ฎค)+ โ grid-cols-1+ โ โโ TOP DOMAINS ๆๆปกๅฎฝๅบฆ๏ผๆพ็คบๆดๅคๆฐๆฎ+ โ+ โโโ ๅๅๅธๅฑ (md: 768px+)+ โ md:grid-cols-2+ โ โโ ๅก็่ช้ๅบๆๅ+ โ+ โโโ ไธๅๅธๅฑ (xl: 1280px+)+ xl:grid-cols-3+ โโ ไธๅๅนถๆ๏ผ็ชๅก็ไฝฟ็จๅ็ดๅธๅฑ++ๅฎนๅจๆฅ่ฏข (@container)+ โ+ โโโ @min-[200px]: ็ชๅก็ๅ็ดๅธๅฑ+ โโโ @min-[500px]: ๅฎฝๅก็ๆฐดๅนณๅธๅฑ๏ผๆพ็คบๆดๅคๆฐๆฎ+```++---++## ้กน็ฎ็ฎๅฝ็ปๆ++```+neko-master/+โโโ apps/+โ โโโ web/ # Next.js ๅ็ซฏ+โ โ โโโ app/ # App Router+โ โ โโโ components/+โ โ โ โโโ features/ # ๅ่ฝ็ปไปถ+โ โ โ โ โโโ backend/ # ๅ็ซฏ้ ็ฝฎ+โ โ โ โ โโโ devices/ # ่ฎพๅค็ป่ฎก+โ โ โ โ โโโ domains/ # ๅๅ็ป่ฎก+โ โ โ โ โโโ proxies/ # ไปฃ็็ป่ฎก+โ โ โ โ โโโ rules/ # ่งๅ็ป่ฎก+โ โ โ โ โโโ stats/ # ็ป่ฎกๅพ่กจ+โ โ โ โโโ layout/ # ๅธๅฑ็ปไปถ+โ โ โ โโโ ui/ # ๅบ็ก UI ็ปไปถ+โ โ โโโ hooks/api/ # API Hooks+โ โ โโโ lib/ # ๅทฅๅ ทๅบ+โ โ โโโ messages/ # i18n ็ฟป่ฏ+โ โ+โ โโโ collector/ # Fastify ๆฐๆฎ้้ๆๅก+โ โโโ src/+โ โ โโโ collectors/ # ้้ๅจๅฎ็ฐ+โ โ โ โโโ clash-collector.ts # Clash WebSocket+โ โ โ โโโ surge-collector.ts # Surge HTTP+โ โ โโโ database/+โ โ โ โโโ repositories/ # Repository ๆจกๅผ+โ โ โ โโโ base.repository.ts # ๆฝ่ฑกๅบ็ฑป (13 ไธชๅ ฑไบซๅทฅๅ ท)+โ โ โ โโโ domain.repository.ts # ๅๅ็ป่ฎก+โ โ โ โโโ ip.repository.ts # IP ็ป่ฎก + ASN/GeoIP+โ โ โ โโโ rule.repository.ts # ่งๅ็ป่ฎก + ChainFlow+โ โ โ โโโ proxy.repository.ts # ไปฃ็็ป่ฎก+โ โ โ โโโ device.repository.ts # ่ฎพๅค็ป่ฎก+โ โ โ โโโ country.repository.ts # ๅฝๅฎถ/ๅฐๅบ็ป่ฎก+โ โ โ โโโ timeseries.repository.ts # ๆถๅบๆฐๆฎ+โ โ โ โโโ traffic-writer.repository.ts # ๆต้ๅๅ ฅ+โ โ โ โโโ config.repository.ts # ๆฐๆฎๅบ้ ็ฝฎ/ๆธ ็+โ โ โ โโโ backend.repository.ts # ๅ็ซฏ็ฎก็+โ โ โ โโโ auth.repository.ts # ่ฎค่ฏ้ ็ฝฎ+โ โ โ โโโ surge.repository.ts # Surge ็ญ็ฅ็ผๅญ+โ โ โ โโโ index.ts # ็ปไธๅฏผๅบ+โ โ โโโ db.ts # ้จ้ข็ฑป (~1000 ่ก, DDL + ๅงๆ)+โ โ โโโ modules/ # ไธๅกๆจกๅ+โ โ โ โโโ auth/ # ่ฎค่ฏ+โ โ โ โโโ backend/ # ๅ็ซฏ็ฎก็+โ โ โ โโโ stats/ # ็ป่ฎกๆๅก+โ โ โ โโโ surge/ # Surge ๆๅก+โ โ โ โโโ realtime/ # ๅฎๆถๆฐๆฎ+โ โ โ โโโ websocket/ # WebSocket+โ โ โโโ routes/ # API ่ทฏ็ฑ+โ โ โโโ services/ # ๅบ็กๆๅก+โ โ โ โโโ geoip.ts # GeoIP ๆๅก+โ โ โ โโโ policy-sync.ts # ็ญ็ฅๅๆญฅ+โ โ โ โโโ realtime-store.ts # ๅฎๆถๆฐๆฎๅญๅจ+โ โ โโโ index.ts # ๅ ฅๅฃ+โ โโโ package.json+โ+โโโ packages/+โ โโโ shared/ # ๅ ฑไบซ็ฑปๅๅฎไน+โ+โโโ docs/+โ โโโ architecture.md # ๆฌๆๆกฃ+โ+โโโ docker-compose.yml+```diff --git a/package.json b/package.jsonindex d1f61b3..e148e3e 100644--- a/package.json+++ b/package.json@@ -1,6 +1,6 @@{"name": "neko-master",- "version": "1.2.6",+ "version": "1.2.7","description": "Modern traffic analytics for edge gateways","private": true,"workspaces": [diff --git a/packages/shared/src/gateway-utils.ts b/packages/shared/src/gateway-utils.tsnew file mode 100644index 0000000..29032b8--- /dev/null+++ b/packages/shared/src/gateway-utils.ts@@ -0,0 +1,120 @@+/**+ * Gateway API ๅทฅๅ ทๅฝๆฐ+ * ็จไบๅค็ Clash/Surge ๅ็ซฏ็้็จ้ป่พ+ */++export type BackendType = 'clash' | 'surge';++export interface BackendConfig {+ id: number;+ name: string;+ url: string;+ token: string;+ type: BackendType;+ enabled: boolean;+ is_active: boolean;+ listening: boolean;+ created_at: string;+ updated_at: string;+}++/**+ * ๆๅปบ Gateway API ่ฏทๆฑๅคด+ */+export function buildGatewayHeaders(+ backend: Pick<BackendConfig, 'type' | 'token'>,+ extraHeaders: Record<string, string> = {}+): Record<string, string> {+ const headers: Record<string, string> = {+ 'Accept': 'application/json',+ ...extraHeaders,+ };++ if (backend.token) {+ if (backend.type === 'surge') {+ headers['x-key'] = backend.token;+ } else {+ headers['Authorization'] = `Bearer ${backend.token}`;+ }+ }++ return headers;+}++/**+ * ่งฃๆ Surge ่งๅๅญ็ฌฆไธฒ+ * ๆฏๆๆ ผๅผ: TYPE,PAYLOAD,POLICY ๆ TYPE,POLICY+ */+export function parseSurgeRule(raw: string): { type: string; payload: string; policy: string } | null {+ const trimmed = raw.trim();++ // Skip empty lines and comments+ if (!trimmed || trimmed.startsWith('#')) {+ return null;+ }++ const parts = trimmed.split(',').map(p => p.trim());+ if (parts.length < 2) {+ return null;+ }++ const type = parts[0];++ // Helper to remove surrounding quotes from policy names+ const unquote = (str: string) => str?.replace(/^["']|["']$/g, '') || '';++ // Handle different rule types+ if (type === 'FINAL') {+ return { type: 'MATCH', payload: '*', policy: unquote(parts[1]) || 'DIRECT' };+ }++ if (type === 'GEOIP') {+ return { type: 'GEOIP', payload: parts[1] || '', policy: unquote(parts[2]) || 'DIRECT' };+ }++ if (type === 'RULE-SET') {+ return { type: 'RULE-SET', payload: parts[1] || '', policy: unquote(parts[2]) || 'DIRECT' };+ }++ // Generic format: TYPE,payload,policy+ if (parts.length >= 3) {+ return { type, payload: parts[1], policy: unquote(parts[2]) };+ }++ // Fallback: TYPE,policy+ return { type, payload: '', policy: unquote(parts[1]) || 'DIRECT' };+}++/**+ * ่งฃๆ Surge ่งๅ๏ผๆฏๆๅฏน่ฑกๆๅญ็ฌฆไธฒๆ ผๅผ๏ผ+ * ็จไบๅ็ซฏ active-chain.ts+ */+export function parseGatewayRule(rule: unknown): { payload?: string; proxy: string } | null {+ if (typeof rule === 'string') {+ const parsed = parseSurgeRule(rule);+ if (!parsed) return null;+ return {+ payload: parsed.payload,+ proxy: parsed.policy,+ };+ } else if (typeof rule === 'object' && rule !== null) {+ const r = rule as Record<string, string>;+ return {+ payload: r.payload,+ proxy: r.proxy,+ };+ }+ return null;+}++/**+ * ๆๅ Gateway ๅบ็ก URL+ * ๅฐ WebSocket URL ่ฝฌๆขไธบ HTTP URL๏ผ็งป้ค่ทฏๅพ+ */+export function getGatewayBaseUrl(url: string): string {+ return url+ .replace(/^ws:\/\//, 'http://')+ .replace(/^wss:\/\//, 'https://')+ .replace(/\/connections\/?$/, '')+ .replace(/\/$/, '');+}diff --git a/packages/shared/src/index.ts b/packages/shared/src/index.tsindex f242eda..f1343b5 100644--- a/packages/shared/src/index.ts+++ b/packages/shared/src/index.ts@@ -238,3 +238,81 @@ export interface AuthState {forceAccessControlOff?: boolean;showcaseMode?: boolean;}++// Surge API Types+export interface SurgeRequest {+ id: string;+ time: number;+ timestamp?: string;+ policyName: string;+ originalPolicyName: string;+ rule: string;+ processPath: string;+ remoteHost: string;+ remoteAddress?: string;+ remotePort?: number;+ localAddress?: string;+ localPort?: number;+ sourceAddress?: string;+ sourcePort?: number;+ inBytes: number;+ outBytes: number;+ inCurrentSpeed?: number;+ outCurrentSpeed?: number;+ inMaxSpeed?: number;+ outMaxSpeed?: number;+ status?: string;+ completed?: boolean;+ disconnected?: boolean;+ failed?: boolean;+ rejected?: boolean;+ startDate?: number;+ completedDate?: number;+ setupCompletedDate?: number;+ URL?: string;+ method?: string;+ notes?: string[];+}++export interface SurgeRequestsData {+ requests: SurgeRequest[];+}++export interface SurgePolicy {+ name: string;+ type: string;+ lineHash: string;+}++export interface SurgePolicyGroup {+ name: string;+ type: string;+ lineHash: string;+ policy: string;+ policies: string[];+ icon?: string;+}++export interface SurgePoliciesData {+ proxies: SurgePolicy[];+ groups: SurgePolicyGroup[];+}++// Surge Rules API Response+export interface SurgeRuleItem {+ type: string;+ payload: string;+ policy: string;+ raw: string;+}++export interface SurgeRulesData {+ rules: SurgeRuleItem[];+ availablePolicies: string[];+}++// Backend Type+export type BackendType = 'clash' | 'surge';++// Gateway utilities+export * from './gateway-utils.js';