Nexus-Guard-Suite/.omo/plans/archive/traffic-performance.md

Traffic Performance Optimization

TL;DR

Quick Summary: Fix TrafficHistory performance — silent auto-refresh (no loading flash), limit data fetched, optimize chart rendering, and add server-side pagination.

Deliverables:

• Silent auto-refresh (no loading state during background updates)
• API limit parameter to cap data fetched
• Client-side chart downsampling (max 200 points)
• Smart CSV export (current page or all with progress)
• Auto-refresh interval increased to 30s

Estimated Effort: Medium Parallel Execution: YES - 2 waves Critical Path: Backend limit → Frontend fetch → Chart/table optimizations

---

Context

Original Request

User reports: "terlalu banyak data record. realtime tidak smooth (masih ada warna loading). record table terdownload semua."

Architecture Finding

• SSE endpoint exists (/devices/stream) but only streams device STATUS, not traffic data
• Frontend has NO EventSource consumer — SSE endpoint is orphaned
• Traffic uses pure REST polling — every 10s, fetch ALL records → loading flash
• No WebSocket anywhere in the codebase
• TrafficRecorder stores data in Redis (24h TTL) → syncs to PostgreSQL every 5min

Why NOT WebSocket/SSE for Traffic (Yet)

1. SSE doesn't support custom Authorization headers — token must be query param or cookie (security tradeoff)
2. Existing SSE only handles device status — would need new SSE channel for traffic
3. Traffic data is already in Redis with 24h TTL — REST with limit is sufficient
4. Recommended: Fix REST performance first → evaluate SSE for traffic in future iteration

---

Work Objectives

Core Objective

Make TrafficHistory page smooth, fast, and non-blocking — no loading flash, limited data, optimized rendering.

Concrete Deliverables

• apps/server-core/api/traffic.go — Add limit query parameter
• apps/dashboard-ui/src/views/TrafficHistory.vue — Silent refresh, smart pagination, optimized export
• apps/dashboard-ui/src/components/TrafficChart.vue — Downsample data for SVG rendering

Must Have

• Auto-refresh does NOT show loading state (silent background update)
• Auto-refresh does NOT reset pagination page
• API supports limit parameter (default 500, max 5000)
• Chart downsamples to max 200 data points
• CSV export shows progress or limits to current page
• Auto-refresh interval 30s (was 10s)

Must NOT Have (Guardrails)

• Do NOT add WebSocket infrastructure (future iteration)
• Do NOT change TrafficRecorder or Redis storage
• Do NOT change the SSE device status endpoint
• Do NOT change the POST /traffic/report endpoint
• Do NOT change TrafficChart's visual appearance
• Do NOT remove the auto-refresh feature

---

Verification Strategy

QA Policy

• Frontend: npm run build passes
• Backend: go build ./... passes
• Grep: no setInterval with < 20000ms interval
• Manual check: no loading flash during auto-refresh

---

Execution Strategy

Parallel Execution Waves

Wave 1 (Backend + Frontend foundation):
├── Task 1: Add limit param to traffic API [quick]
├── Task 2: Silent auto-refresh + pagination fix [quick]
├── Task 3: Chart downsampling [quick]

Wave 2 (Integration + Polish):
├── Task 4: CSV export optimization [quick]
├── Task 5: Build verify [quick]

---

TODOs

• 1. Add limit parameter to traffic API

  What to do:

  • In apps/server-core/api/traffic.go, modify parseTimeRange to also parse limit query parameter
  • Add limit parameter to GetSummary: limit := c.DefaultQuery("limit", "500")
  • Parse limit as int, cap at 5000 max
  • Apply .Limit(limit) to the GORM query in GetSummary
  • Also add limit to GetDeviceTraffic and GetNodeTraffic
  • Return total_count in response alongside total_records (total available before limit)

  Must NOT do:

  • Do NOT change TrafficRecorder
  • Do NOT change Redis storage
  • Do NOT change POST /traffic/report

  References:

  • apps/server-core/api/traffic.go — Full file (90 lines). parseTimeRange at line 75, GetSummary at line 55
  • apps/server-core/internal/traffic/recorder.go — TrafficRecord struct at line 15

  QA Scenarios:

  Scenario: API respects limit parameter
    Tool: Bash (curl)
    Steps:
      1. Start dev server
      2. curl -H "Authorization: Bearer <token>" "http://localhost:8080/api/v1/traffic/summary?from=...&to=...&limit=10"
    Expected Result: Response contains max 10 records, total_count shows actual total
    Evidence: .sisyphus/evidence/task-1-api-limit.txt
  
  Scenario: Build passes
    Tool: Bash
    Steps:
      1. cd apps/server-core && go build ./...
    Expected Result: Exit code 0
    Evidence: .sisyphus/evidence/task-1-build.txt
  

  Commit: YES (groups with 2-5)

---

• 2. Silent auto-refresh + pagination fix

  What to do:

  • Modify fetchTrafficData to accept optional silent parameter (default false)
  • When silent=true: skip loading.value = true, skip currentPage.value = 1
  • Auto-refresh interval calls fetchTrafficData(true) — silent mode
  • Manual "Apply Filters" calls fetchTrafficData() — shows loading, resets page
  • Change interval from 10000ms to 30000ms
  • Add ?limit=500 to API URLs
  • Store totalCount from API response for pagination display
  • Update pagination display to show "of X total" using totalCount

  Must NOT do:

  • Do NOT remove auto-refresh
  • Do NOT change the date filtering logic
  • Do NOT change the chart component

  References:

  • apps/dashboard-ui/src/views/TrafficHistory.vue — Lines 177-205 (fetchTrafficData), 283-293 (interval)
  • apps/dashboard-ui/src/services/api.ts — Base axios instance

  QA Scenarios:

  Scenario: No loading flash during auto-refresh
    Tool: Playwright
    Steps:
      1. Open Traffic History page
      2. Wait for initial load
      3. Observe for 35 seconds — no loading bar should appear after initial load
    Expected Result: Loading indicator does NOT flash during background refresh
    Evidence: .sisyphus/evidence/task-2-no-flash.txt
  
  Scenario: Build passes
    Tool: Bash
    Steps:
      1. cd apps/dashboard-ui && npm run build
    Expected Result: Exit code 0
    Evidence: .sisyphus/evidence/task-2-build.txt
  

  Commit: YES (groups with 1, 3-5)

---

• 3. Chart downsampling

  What to do:

  • In TrafficHistory.vue, add a chartDataLimited computed that limits chart data to max 200 points
  • If data > 200 points, downsample by averaging every N points (N = Math.ceil(data.length / 200))
  • Pass chartDataLimited to TrafficChart instead of chartData
  • Keep full trafficData for table pagination and CSV export

  Must NOT do:

  • Do NOT change TrafficChart.vue component
  • Do NOT change the SVG rendering logic

  References:

  • apps/dashboard-ui/src/views/TrafficHistory.vue — Lines 150-157 (chartData computed)

  QA Scenarios:

  Scenario: Chart receives max 200 data points
    Tool: Bash
    Steps:
      1. Grep for chartDataLimited in TrafficHistory.vue
    Expected Result: Computed property exists with 200-point cap
    Evidence: .sisyphus/evidence/task-3-downsample.txt
  
  Scenario: Build passes
    Tool: Bash
    Steps:
      1. cd apps/dashboard-ui && npm run build
    Expected Result: Exit code 0
    Evidence: .sisyphus/evidence/task-3-build.txt
  

  Commit: YES (groups with 1-2, 4-5)

---

• 4. CSV export optimization

  What to do:

  • Change exportToCSV to export only paginatedData (current page) by default
  • Add a confirmation: "Export all X records or just current page?"
  • Or simpler: always export current filtered data (not limited by pagination)
  • Keep export fast by limiting to filtered dataset

  Must NOT do:

  • Do NOT add async CSV generation (overkill)
  • Do NOT change the download mechanism

  References:

  • apps/dashboard-ui/src/views/TrafficHistory.vue — Lines 239-263 (exportToCSV)

  QA Scenarios:

  Scenario: CSV export works
    Tool: Bash
    Steps:
      1. cd apps/dashboard-ui && npm run build
    Expected Result: Exit code 0
    Evidence: .sisyphus/evidence/task-4-build.txt
  

  Commit: YES (groups with 1-3, 5)

---

• 5. Build verify all changes

  What to do:

  • Run cd apps/server-core && go build ./...
  • Run cd apps/dashboard-ui && npm run build
  • Grep for setInterval with interval < 20000ms in TrafficHistory.vue
  • Verify no loading.value = true in auto-refresh path

  References:

  • All modified files

  QA Scenarios:

  Scenario: Full build passes
    Tool: Bash
    Steps:
      1. cd apps/server-core && go build ./...
      2. cd apps/dashboard-ui && npm run build
    Expected Result: Both exit code 0
    Evidence: .sisyphus/evidence/task-5-full-build.txt
  
  Scenario: No aggressive polling
    Tool: Bash
    Steps:
      1. grep -n "setInterval" apps/dashboard-ui/src/views/TrafficHistory.vue
    Expected Result: Interval >= 20000ms
    Evidence: .sisyphus/evidence/task-5-polling-check.txt
  

  Commit: YES (final commit)

---

Commit Strategy

• Commit E: All traffic performance changes
  • Files: api/traffic.go, TrafficHistory.vue
  • Pre-commit: go build ./... && cd ../dashboard-ui && npm run build

---

Success Criteria

Verification Commands

cd apps/server-core && go build ./...          # Expected: exit 0
cd apps/dashboard-ui && npm run build           # Expected: ✓ built in Xs
grep "setInterval" apps/dashboard-ui/src/views/TrafficHistory.vue  # Expected: 30000
grep -c "loading.value = true" apps/dashboard-ui/src/views/TrafficHistory.vue  # Expected: 1 (only manual refresh)

Final Checklist

• Auto-refresh is silent (no loading flash)
• Auto-refresh does not reset pagination
• API supports limit parameter
• Chart renders max 200 data points
• Auto-refresh interval 30s
• Both backend and frontend build successfully