chore: update submodule refs, add plan files
NexusGuard CI / server-core-test (push) Failing after 3s
NexusGuard CI / server-core-build (push) Has been skipped
NexusGuard CI / device-agent-test (push) Failing after 3s
NexusGuard CI / device-agent-cross-build (amd64, linux) (push) Has been skipped
NexusGuard CI / device-agent-cross-build (amd64, windows) (push) Has been skipped
NexusGuard CI / device-agent-cross-build (arm64, linux) (push) Has been skipped
NexusGuard CI / dashboard-test (push) Failing after 3s
NexusGuard CI / dashboard-dist (push) Has been skipped
NexusGuard CI / server-core-test (push) Failing after 3s
NexusGuard CI / server-core-build (push) Has been skipped
NexusGuard CI / device-agent-test (push) Failing after 3s
NexusGuard CI / device-agent-cross-build (amd64, linux) (push) Has been skipped
NexusGuard CI / device-agent-cross-build (amd64, windows) (push) Has been skipped
NexusGuard CI / device-agent-cross-build (arm64, linux) (push) Has been skipped
NexusGuard CI / dashboard-test (push) Failing after 3s
NexusGuard CI / dashboard-dist (push) Has been skipped
This commit is contained in:
datadunia
@@ -0,0 +1,304 @@
|
||||
# Traffic Performance Optimization
|
||||
|
||||
## TL;DR
|
||||
|
||||
> **Quick Summary**: Fix TrafficHistory performance — silent auto-refresh (no loading flash), limit data fetched, optimize chart rendering, and add server-side pagination.
|
||||
>
|
||||
> **Deliverables**:
|
||||
> - Silent auto-refresh (no loading state during background updates)
|
||||
> - API limit parameter to cap data fetched
|
||||
> - Client-side chart downsampling (max 200 points)
|
||||
> - Smart CSV export (current page or all with progress)
|
||||
> - Auto-refresh interval increased to 30s
|
||||
>
|
||||
> **Estimated Effort**: Medium
|
||||
> **Parallel Execution**: YES - 2 waves
|
||||
> **Critical Path**: Backend limit → Frontend fetch → Chart/table optimizations
|
||||
|
||||
---
|
||||
|
||||
## Context
|
||||
|
||||
### Original Request
|
||||
User reports: "terlalu banyak data record. realtime tidak smooth (masih ada warna loading). record table terdownload semua."
|
||||
|
||||
### Architecture Finding
|
||||
- **SSE endpoint exists** (`/devices/stream`) but only streams device STATUS, not traffic data
|
||||
- **Frontend has NO EventSource consumer** — SSE endpoint is orphaned
|
||||
- **Traffic uses pure REST polling** — every 10s, fetch ALL records → loading flash
|
||||
- **No WebSocket anywhere** in the codebase
|
||||
- **TrafficRecorder** stores data in Redis (24h TTL) → syncs to PostgreSQL every 5min
|
||||
|
||||
### Why NOT WebSocket/SSE for Traffic (Yet)
|
||||
1. SSE doesn't support custom `Authorization` headers — token must be query param or cookie (security tradeoff)
|
||||
2. Existing SSE only handles device status — would need new SSE channel for traffic
|
||||
3. Traffic data is already in Redis with 24h TTL — REST with limit is sufficient
|
||||
4. **Recommended**: Fix REST performance first → evaluate SSE for traffic in future iteration
|
||||
|
||||
---
|
||||
|
||||
## Work Objectives
|
||||
|
||||
### Core Objective
|
||||
Make TrafficHistory page smooth, fast, and non-blocking — no loading flash, limited data, optimized rendering.
|
||||
|
||||
### Concrete Deliverables
|
||||
- `apps/server-core/api/traffic.go` — Add `limit` query parameter
|
||||
- `apps/dashboard-ui/src/views/TrafficHistory.vue` — Silent refresh, smart pagination, optimized export
|
||||
- `apps/dashboard-ui/src/components/TrafficChart.vue` — Downsample data for SVG rendering
|
||||
|
||||
### Must Have
|
||||
- Auto-refresh does NOT show loading state (silent background update)
|
||||
- Auto-refresh does NOT reset pagination page
|
||||
- API supports `limit` parameter (default 500, max 5000)
|
||||
- Chart downsamples to max 200 data points
|
||||
- CSV export shows progress or limits to current page
|
||||
- Auto-refresh interval 30s (was 10s)
|
||||
|
||||
### Must NOT Have (Guardrails)
|
||||
- Do NOT add WebSocket infrastructure (future iteration)
|
||||
- Do NOT change TrafficRecorder or Redis storage
|
||||
- Do NOT change the SSE device status endpoint
|
||||
- Do NOT change the POST /traffic/report endpoint
|
||||
- Do NOT change TrafficChart's visual appearance
|
||||
- Do NOT remove the auto-refresh feature
|
||||
|
||||
---
|
||||
|
||||
## Verification Strategy
|
||||
|
||||
### QA Policy
|
||||
- Frontend: `npm run build` passes
|
||||
- Backend: `go build ./...` passes
|
||||
- Grep: no `setInterval` with < 20000ms interval
|
||||
- Manual check: no loading flash during auto-refresh
|
||||
|
||||
---
|
||||
|
||||
## Execution Strategy
|
||||
|
||||
### Parallel Execution Waves
|
||||
|
||||
```
|
||||
Wave 1 (Backend + Frontend foundation):
|
||||
├── Task 1: Add limit param to traffic API [quick]
|
||||
├── Task 2: Silent auto-refresh + pagination fix [quick]
|
||||
├── Task 3: Chart downsampling [quick]
|
||||
|
||||
Wave 2 (Integration + Polish):
|
||||
├── Task 4: CSV export optimization [quick]
|
||||
├── Task 5: Build verify [quick]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## TODOs
|
||||
|
||||
- [ ] 1. Add limit parameter to traffic API
|
||||
|
||||
**What to do**:
|
||||
- In `apps/server-core/api/traffic.go`, modify `parseTimeRange` to also parse `limit` query parameter
|
||||
- Add `limit` parameter to `GetSummary`: `limit := c.DefaultQuery("limit", "500")`
|
||||
- Parse limit as int, cap at 5000 max
|
||||
- Apply `.Limit(limit)` to the GORM query in `GetSummary`
|
||||
- Also add limit to `GetDeviceTraffic` and `GetNodeTraffic`
|
||||
- Return `total_count` in response alongside `total_records` (total available before limit)
|
||||
|
||||
**Must NOT do**:
|
||||
- Do NOT change TrafficRecorder
|
||||
- Do NOT change Redis storage
|
||||
- Do NOT change POST /traffic/report
|
||||
|
||||
**References**:
|
||||
- `apps/server-core/api/traffic.go` — Full file (90 lines). `parseTimeRange` at line 75, `GetSummary` at line 55
|
||||
- `apps/server-core/internal/traffic/recorder.go` — `TrafficRecord` struct at line 15
|
||||
|
||||
**QA Scenarios:**
|
||||
```
|
||||
Scenario: API respects limit parameter
|
||||
Tool: Bash (curl)
|
||||
Steps:
|
||||
1. Start dev server
|
||||
2. curl -H "Authorization: Bearer <token>" "http://localhost:8080/api/v1/traffic/summary?from=...&to=...&limit=10"
|
||||
Expected Result: Response contains max 10 records, total_count shows actual total
|
||||
Evidence: .sisyphus/evidence/task-1-api-limit.txt
|
||||
|
||||
Scenario: Build passes
|
||||
Tool: Bash
|
||||
Steps:
|
||||
1. cd apps/server-core && go build ./...
|
||||
Expected Result: Exit code 0
|
||||
Evidence: .sisyphus/evidence/task-1-build.txt
|
||||
```
|
||||
|
||||
**Commit**: YES (groups with 2-5)
|
||||
|
||||
---
|
||||
|
||||
- [ ] 2. Silent auto-refresh + pagination fix
|
||||
|
||||
**What to do**:
|
||||
- Modify `fetchTrafficData` to accept optional `silent` parameter (default false)
|
||||
- When `silent=true`: skip `loading.value = true`, skip `currentPage.value = 1`
|
||||
- Auto-refresh interval calls `fetchTrafficData(true)` — silent mode
|
||||
- Manual "Apply Filters" calls `fetchTrafficData()` — shows loading, resets page
|
||||
- Change interval from 10000ms to 30000ms
|
||||
- Add `?limit=500` to API URLs
|
||||
- Store `totalCount` from API response for pagination display
|
||||
- Update pagination display to show "of X total" using totalCount
|
||||
|
||||
**Must NOT do**:
|
||||
- Do NOT remove auto-refresh
|
||||
- Do NOT change the date filtering logic
|
||||
- Do NOT change the chart component
|
||||
|
||||
**References**:
|
||||
- `apps/dashboard-ui/src/views/TrafficHistory.vue` — Lines 177-205 (fetchTrafficData), 283-293 (interval)
|
||||
- `apps/dashboard-ui/src/services/api.ts` — Base axios instance
|
||||
|
||||
**QA Scenarios:**
|
||||
```
|
||||
Scenario: No loading flash during auto-refresh
|
||||
Tool: Playwright
|
||||
Steps:
|
||||
1. Open Traffic History page
|
||||
2. Wait for initial load
|
||||
3. Observe for 35 seconds — no loading bar should appear after initial load
|
||||
Expected Result: Loading indicator does NOT flash during background refresh
|
||||
Evidence: .sisyphus/evidence/task-2-no-flash.txt
|
||||
|
||||
Scenario: Build passes
|
||||
Tool: Bash
|
||||
Steps:
|
||||
1. cd apps/dashboard-ui && npm run build
|
||||
Expected Result: Exit code 0
|
||||
Evidence: .sisyphus/evidence/task-2-build.txt
|
||||
```
|
||||
|
||||
**Commit**: YES (groups with 1, 3-5)
|
||||
|
||||
---
|
||||
|
||||
- [ ] 3. Chart downsampling
|
||||
|
||||
**What to do**:
|
||||
- In `TrafficHistory.vue`, add a `chartDataLimited` computed that limits chart data to max 200 points
|
||||
- If data > 200 points, downsample by averaging every N points (N = Math.ceil(data.length / 200))
|
||||
- Pass `chartDataLimited` to TrafficChart instead of `chartData`
|
||||
- Keep full `trafficData` for table pagination and CSV export
|
||||
|
||||
**Must NOT do**:
|
||||
- Do NOT change TrafficChart.vue component
|
||||
- Do NOT change the SVG rendering logic
|
||||
|
||||
**References**:
|
||||
- `apps/dashboard-ui/src/views/TrafficHistory.vue` — Lines 150-157 (chartData computed)
|
||||
|
||||
**QA Scenarios:**
|
||||
```
|
||||
Scenario: Chart receives max 200 data points
|
||||
Tool: Bash
|
||||
Steps:
|
||||
1. Grep for chartDataLimited in TrafficHistory.vue
|
||||
Expected Result: Computed property exists with 200-point cap
|
||||
Evidence: .sisyphus/evidence/task-3-downsample.txt
|
||||
|
||||
Scenario: Build passes
|
||||
Tool: Bash
|
||||
Steps:
|
||||
1. cd apps/dashboard-ui && npm run build
|
||||
Expected Result: Exit code 0
|
||||
Evidence: .sisyphus/evidence/task-3-build.txt
|
||||
```
|
||||
|
||||
**Commit**: YES (groups with 1-2, 4-5)
|
||||
|
||||
---
|
||||
|
||||
- [ ] 4. CSV export optimization
|
||||
|
||||
**What to do**:
|
||||
- Change exportToCSV to export only `paginatedData` (current page) by default
|
||||
- Add a confirmation: "Export all X records or just current page?"
|
||||
- Or simpler: always export current filtered data (not limited by pagination)
|
||||
- Keep export fast by limiting to filtered dataset
|
||||
|
||||
**Must NOT do**:
|
||||
- Do NOT add async CSV generation (overkill)
|
||||
- Do NOT change the download mechanism
|
||||
|
||||
**References**:
|
||||
- `apps/dashboard-ui/src/views/TrafficHistory.vue` — Lines 239-263 (exportToCSV)
|
||||
|
||||
**QA Scenarios:**
|
||||
```
|
||||
Scenario: CSV export works
|
||||
Tool: Bash
|
||||
Steps:
|
||||
1. cd apps/dashboard-ui && npm run build
|
||||
Expected Result: Exit code 0
|
||||
Evidence: .sisyphus/evidence/task-4-build.txt
|
||||
```
|
||||
|
||||
**Commit**: YES (groups with 1-3, 5)
|
||||
|
||||
---
|
||||
|
||||
- [ ] 5. Build verify all changes
|
||||
|
||||
**What to do**:
|
||||
- Run `cd apps/server-core && go build ./...`
|
||||
- Run `cd apps/dashboard-ui && npm run build`
|
||||
- Grep for `setInterval` with interval < 20000ms in TrafficHistory.vue
|
||||
- Verify no `loading.value = true` in auto-refresh path
|
||||
|
||||
**References**:
|
||||
- All modified files
|
||||
|
||||
**QA Scenarios:**
|
||||
```
|
||||
Scenario: Full build passes
|
||||
Tool: Bash
|
||||
Steps:
|
||||
1. cd apps/server-core && go build ./...
|
||||
2. cd apps/dashboard-ui && npm run build
|
||||
Expected Result: Both exit code 0
|
||||
Evidence: .sisyphus/evidence/task-5-full-build.txt
|
||||
|
||||
Scenario: No aggressive polling
|
||||
Tool: Bash
|
||||
Steps:
|
||||
1. grep -n "setInterval" apps/dashboard-ui/src/views/TrafficHistory.vue
|
||||
Expected Result: Interval >= 20000ms
|
||||
Evidence: .sisyphus/evidence/task-5-polling-check.txt
|
||||
```
|
||||
|
||||
**Commit**: YES (final commit)
|
||||
|
||||
---
|
||||
|
||||
## Commit Strategy
|
||||
|
||||
- **Commit E**: All traffic performance changes
|
||||
- Files: `api/traffic.go`, `TrafficHistory.vue`
|
||||
- Pre-commit: `go build ./... && cd ../dashboard-ui && npm run build`
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
### Verification Commands
|
||||
```bash
|
||||
cd apps/server-core && go build ./... # Expected: exit 0
|
||||
cd apps/dashboard-ui && npm run build # Expected: ✓ built in Xs
|
||||
grep "setInterval" apps/dashboard-ui/src/views/TrafficHistory.vue # Expected: 30000
|
||||
grep -c "loading.value = true" apps/dashboard-ui/src/views/TrafficHistory.vue # Expected: 1 (only manual refresh)
|
||||
```
|
||||
|
||||
### Final Checklist
|
||||
- [ ] Auto-refresh is silent (no loading flash)
|
||||
- [ ] Auto-refresh does not reset pagination
|
||||
- [ ] API supports limit parameter
|
||||
- [ ] Chart renders max 200 data points
|
||||
- [ ] Auto-refresh interval 30s
|
||||
- [ ] Both backend and frontend build successfully
|
||||
Reference in New Issue
Block a user