289 lines
14 KiB
Markdown
289 lines
14 KiB
Markdown
# PROJECT KNOWLEDGE BASE
|
|
|
|
**Generated:** 2026-06-20
|
|
**Branch:** `main`
|
|
|
|
## OVERVIEW
|
|
NexusGuard SD-WAN Suite: Enterprise Zero-Trust SD-WAN with WireGuard tunneling, centralized IPAM, and real-time nftables network isolation. Monorepo with 3 git submodules: Go backend (Gin), Vue 3 dashboard, Go device agent.
|
|
|
|
## TOPOLOGY
|
|
|
|
| Host | SSH | Role |
|
|
|------|-----|------|
|
|
| Production server | `root@172.20.8.191` | Runs server-core, Postgres, Redis, nginx, nftables, WireGuard |
|
|
| Gitea server | `root@172.20.8.92` | Private Git hosting (`git.datadunia.com`) |
|
|
|
|
- Server project folder: `/root/Nexus-Guard-Suite`
|
|
- Deploy: `./update.sh` (don't build manually)
|
|
- Actual WireGuard wg0 IP: `10.172.21.1/24` (on server 172.20.8.191)
|
|
- Agent WG IPs: dynamic from pool `10.172.21.0/24`
|
|
|
|
## STRUCTURE
|
|
```
|
|
./
|
|
├── apps/
|
|
│ ├── server-core/ # Go/Gin API backend (submodule)
|
|
│ ├── dashboard-ui/ # Vue 3 + Vite frontend (submodule)
|
|
│ └── device-agent/ # Go stealth daemon + system tray (submodule)
|
|
├── docker-compose.yml # Production orchestration
|
|
├── docker-compose.dev.yml# Dev (air hot-reload)
|
|
├── Makefile # up/down/dev/migrate/reset-db
|
|
├── setup.sh # First-run: generate .env + random keys
|
|
├── update.sh # Docker update: pull/build/migrate
|
|
├── nexusguard-install.sh # Native install (systemd + nginx)
|
|
├── nexusguard-uninstall.sh
|
|
├── .env.example # DB/JWT/SALT/VITE config template
|
|
├── .gitmodules # 3 submodules → git.datadunia.com
|
|
└── .opencode/ # IDE agent config (tooling, not project code)
|
|
```
|
|
|
|
**CRITICAL**: `apps/*` are **git submodules** — clone with `--recurse-submodules`.
|
|
|
|
## WHERE TO LOOK
|
|
| Task | Location | Notes |
|
|
|------|----------|-------|
|
|
| API handlers | `apps/server-core/api/` | 17 files: auth, devices, peers, rules, share, provisioning, servers, wg |
|
|
| Backend core | `apps/server-core/internal/` | auth, config, firewall, heartbeat, ipam, models, wgmanager |
|
|
| Dev migration | `apps/server-core/main_dev.go` | GORM AutoMigrate (build tag `dev`) |
|
|
| Firewall rules | `apps/server-core/internal/firewall/` | nftables Linux rules |
|
|
| gRPC signaling | `apps/server-core/signaling/` | Manager + Server: gRPC session tracking, Connect handler, recv loop |
|
|
| Dashboard views | `apps/dashboard-ui/src/views/` | Vue SFC pages |
|
|
| Dashboard API client | `apps/dashboard-ui/src/api/` | Axios API modules |
|
|
| Dashboard stores | `apps/dashboard-ui/src/stores/` | Pinia state stores |
|
|
| Agent client | `apps/device-agent/internal/client/` | Provisioning + heartbeat |
|
|
| Agent signaling | `apps/device-agent/internal/signaling/` | gRPC connect with fallback + reconnect |
|
|
| Agent tunnel | `apps/device-agent/internal/tunnel/` | Memory-injected WireGuard |
|
|
| Shared crypto | `apps/*/shared/crypto/encryptor.go` | AES-256-GCM (duplicated identical) |
|
|
| CI workflows | `apps/*/.gitea/workflows/build.yml` | Gitea Actions per submodule |
|
|
| Build config | `apps/dashboard-ui/vite.config.ts` | Vite 8 + Vue + TailwindCSS v4 |
|
|
| Source of truth | `apps/server-core/docs/` | API_SPEC, KEY_ROTATION, PEER_DISCOVERY |
|
|
|
|
## SIGNALING ARCHITECTURE (CRITICAL)
|
|
|
|
### Topology
|
|
```
|
|
┌──────────────┐ ┌─────────────────┐ ┌──────────────┐
|
|
│ Dashboard │──HTTP──▶│ Server Core │◀─WG────▶│ Device Agent │
|
|
│ (Vue 3) │ :8080 │ (Go/Gin) │ :51820 │ (Go) │
|
|
└──────────────┘ │ │ └──────────────┘
|
|
│ Port 8080: │ │
|
|
│ - HTTP API │ ┌────┴────┐
|
|
│ - gRPC Signal │ │ TUN (wg)│
|
|
│ (cmux) │ │ Memory │
|
|
└─────────────────┘ └─────────┘
|
|
```
|
|
|
|
### Transport Fallback Chain (Agent → Server)
|
|
1. gRPC via HTTPS domain (TLS) → `italy-twenty.gl.at.ply.gg:443`
|
|
2. gRPC via WireGuard IP (insecure, tunnel-encrypted) → `10.172.21.1:8080`
|
|
3. HTTP heartbeat (fallback) → `serverURL/api/v1/heartbeat`
|
|
|
|
### Heartbeat = PRIMARY Channel
|
|
Always runs. Handles:
|
|
- Health check (30s interval)
|
|
- Config sync (detects config changes → rebuild tunnel)
|
|
- Handshake monitoring (rebuilds tunnel if lastHandshake > 120s)
|
|
- Recovery after failure (wasFailing → OnRecovered → full rebuild)
|
|
|
|
### gRPC = BONUS Channel
|
|
Best-effort. Handles:
|
|
- Real-time commands: Suspend, Resume, ConfigUpdate, Reconnect, Disconnect
|
|
- StatusReport from agent (tunnel_up, lastHandshake, state)
|
|
- Ping/Pong keepalive (20s)
|
|
|
|
### gRPC Port Multiplexing
|
|
HTTP + gRPC share port 8080 via `cmux`:
|
|
- Server: `cmux.New(lis)` → match gRPC by `content-type` header, match HTTP by `Any()`
|
|
- Agent connects to same port for both HTTP API and gRPC
|
|
|
|
### Key Design Decisions
|
|
- Agent NEVER destroys tunnel on heartbeat failure — only rebuilds
|
|
- `OnFailure = log only`, `OnRecovered = full rebuild`
|
|
- gRPC OnDisconnect/OnGRPCFailed just log — heartbeat continues
|
|
- Heartbeat reads `last_handshake_time_sec` from WG IPC to detect stale tunnel
|
|
- gRPC StatusReport sends handshake age to server every 30s
|
|
- Server WG IP read from actual kernel interface (`net.InterfaceByName`), NOT from stale DB
|
|
|
|
### Agent Connection Lifecycle
|
|
1. Provision → register with server, get WireGuard config
|
|
2. Start tunnel (memory-injected, no disk files)
|
|
3. Start heartbeat (always, primary channel)
|
|
4. Start gRPC (if ServerWGIP available, bonus channel)
|
|
5. On heartbeat config change → rebuild tunnel
|
|
6. On heartbeat stale handshake → rebuild tunnel
|
|
7. On heartbeat failure+recovery → rebuild tunnel
|
|
8. On gRPC suspend → stop tunnel, heartbeat continues
|
|
9. On gRPC resume → rebuild tunnel from server config
|
|
|
|
### Protobuf Messages
|
|
- **Agent → Server**: HelloMessage, HeartbeatAck, StatusReport, PingMessage
|
|
- **Server → Agent**: ConfigUpdate, SuspendCommand, ResumeCommand, ReconnectCommand, DisconnectCommand, KeepAlive, PongMessage
|
|
|
|
## CODE MAP
|
|
| Symbol | Type | Location | Role |
|
|
|--------|------|----------|------|
|
|
| `main()` (server-core) | func | `apps/server-core/main.go` | Entry: CLI flags + Gin init |
|
|
| `main()` (device-agent) | func | `apps/device-agent/main.go` | Entry: systray + agent daemon lifecycle |
|
|
| `onReady()` / `onExit()` | func | `apps/device-agent/main.go` | System tray setup and cleanup |
|
|
| `startAgent()` / `stopAgent()` | func | `apps/device-agent/main.go` | Agent connect/disconnect lifecycle |
|
|
| `generateIcon()` | func | `apps/device-agent/icon.go` | 16x16 shield icon for tray |
|
|
| `config.Load()` | func | `apps/server-core/internal/config/` | Env-based config loader |
|
|
| `config.LoadConfFile()` | func | `apps/server-core/internal/config/config_loader.go` | Config file parser (.env / nexusguard.conf) |
|
|
| `auth.Init()` | func | `apps/server-core/internal/auth/` | JWT sign/verify init |
|
|
| `firewall.InitNetwork()` | func | `apps/server-core/internal/firewall/` | nftables table/set creation |
|
|
| `ipam.AllocateIP()` | func | `apps/server-core/internal/ipam/` | IP pool allocation from CIDR |
|
|
| `wgmanager.SetConfig()` | func | `apps/server-core/internal/wgmanager/` | WireGuard config push |
|
|
| `wgmanager.GetInterfaceAddress()` | func | `apps/server-core/internal/wgmanager/` | Read actual WG interface IP from kernel |
|
|
| `models.AutoMigrate()` | func | `apps/server-core/internal/models/` | GORM schema migration |
|
|
| `encrypt()` / `decrypt()` | func | `apps/*/shared/crypto/encryptor.go` | AES-256-GCM (identical) |
|
|
| `StartHeartbeat()` | func | `apps/device-agent/internal/client/heartbeat.go` | Heartbeat loop + handshake monitoring |
|
|
| `checkHandshake()` | func | `apps/device-agent/internal/client/heartbeat.go` | Read WG IPC handshake time |
|
|
| `ConnectAndRun()` | func | `apps/device-agent/internal/signaling/client.go` | gRPC connect with fallback + reconnect |
|
|
| `statusLoop()` | func | `apps/device-agent/internal/signaling/client.go` | Sends StatusReport every 30s |
|
|
| `NewManager()` | func | `apps/server-core/signaling/manager.go` | gRPC session tracking |
|
|
| `NewServer()` | func | `apps/server-core/signaling/server.go` | gRPC Connect handler + recv loop |
|
|
|
|
## CONVENTIONS
|
|
- **Go**: Standard layout (`main.go` in root, `internal/`, `api/`)
|
|
- **Vue 3**: Composition API + `<script setup lang="ts">` throughout
|
|
- **State management**: Pinia stores in `src/stores/`
|
|
- **API client**: Axios-based services in `src/api/` + base `src/services/`
|
|
- **Styling**: TailwindCSS v4 (no PostCSS — `@tailwindcss/vite` plugin)
|
|
- **DB**: GORM ORM, PostgreSQL, AutoMigrate in dev / `-migrate-prod` in prod
|
|
- **Naming**: `UPPER_SNAKE_CASE` env vars, `camelCase` Go vars, `PascalCase` exported Go
|
|
- **Auth**: JWT, admin-only enforced by middleware pattern
|
|
- **Build tags**: `//go:build dev` for AutoMigrate
|
|
- **Capabilities**: Server containers need `NET_ADMIN` + `NET_RAW`
|
|
- **Ports**: API 8080, Dashboard 80 (Nginx), Postgres 5432, Redis 6379
|
|
- **Config**: Docker uses `.env`, native uses `/etc/nexusguard/nexusguard.conf`
|
|
|
|
## ANTI-PATTERNS (THIS PROJECT)
|
|
- **NEVER** `nft flush table` — only atomic add/remove
|
|
- **NEVER** commit temp/debug/test files (`nft-fix.sh`, `temp_*.txt` etc) in project root; use `./tests` folder
|
|
- **NEVER** log plaintext or encryption keys
|
|
- **NEVER** reopen completed phases/commits — fix forward only
|
|
- **NEVER** rebuild `shared/crypto/encryptor.go` — copy identical file
|
|
- **NEVER** commit build artifacts (binaries, `dist/`)
|
|
- **NEVER** force push
|
|
- **NEVER** create cross-phase workarounds
|
|
|
|
## WIREGUARD AllowedIPs — SERVER vs CLIENT (CRITICAL)
|
|
|
|
WireGuard `AllowedIPs` has **two different meanings** depending on context. Mixing them causes only 1 peer to work.
|
|
|
|
### Rule
|
|
| Context | Where | Value | Purpose |
|
|
|---------|-------|-------|---------|
|
|
| **Server-side** (kernel `wg set`) | `SyncLocalPeers()` → `peer_sync.go` | `InternalIP/32` per peer | WireGuard routing table — each IP must belong to exactly ONE peer |
|
|
| **Client-side** (`.conf` file) | `getDeviceConfig()` → `peers.go` | `EndpointAllowedIPs` from DB (e.g. `/24`, `0.0.0.0/0`) | Tells client which traffic routes through VPN tunnel |
|
|
| **Firewall** (nftables) | `AddForwardRule()` → `nftables_linux.go` | `EndpointAllowedIPs` from DB | Controls which IPs peer can reach via forwarding |
|
|
|
|
### Why
|
|
WireGuard uses `AllowedIPs` as an **internal routing table**. When two peers share the same `/24`, WireGuard assigns the AllowedIPs to the **last peer configured only** — the first peer gets `(none)`. This is not a bug; it's how WireGuard routing works.
|
|
|
|
### Anti-pattern
|
|
```go
|
|
// WRONG — uses client config for server-side routing
|
|
if d.EndpointAllowedIPs != "" {
|
|
allowedIPs = d.EndpointAllowedIPs // "10.172.21.0/24" ← SAME for both peers!
|
|
}
|
|
```
|
|
|
|
### Correct pattern
|
|
```go
|
|
// CORRECT — server-side always /32 per peer
|
|
allowedIPs := *d.InternalIP + "/32" // "10.172.21.2/32" for gogo2
|
|
if d.AllowInternet {
|
|
allowedIPs = "0.0.0.0/0"
|
|
}
|
|
// Do NOT override with EndpointAllowedIPs here
|
|
```
|
|
|
|
### Database field: `endpoint_allowed_ips`
|
|
- Used for **client config** and **firewall rules**
|
|
- NOT used for server-side WireGuard kernel config
|
|
- Example: `10.172.21.0/24` allows peer to reach full subnet via firewall + routes full subnet through VPN on client
|
|
|
|
## FIREWALL CHAINS — INPUT vs FORWARD (CRITICAL)
|
|
|
|
nftables traffic enters different chains depending on destination:
|
|
- **Traffic TO server's own IP** (e.g. 10.172.21.1) → **INPUT chain**
|
|
- **Traffic THROUGH server** (peer-to-peer, e.g. 10.172.21.2 → 10.172.21.3) → **FORWARD chain**
|
|
|
|
### Rule routing in code
|
|
| Destination | Chain | Method |
|
|
|-------------|-------|--------|
|
|
| Server's own `interface_address` | INPUT | `AddInputFirewallRule()` |
|
|
| Other peer IPs / subnets | FORWARD | `AddFirewallRule()` |
|
|
|
|
### Detection logic (syncRuleToFirewall)
|
|
```go
|
|
serverIP = strings.Split(localServer.InterfaceAddress, "/")[0]
|
|
destBase = strings.Split(destCIDR, "/")[0]
|
|
if destBase == serverIP {
|
|
fw.AddInputFirewallRule(...) // → INPUT chain
|
|
} else {
|
|
fw.AddFirewallRule(...) // → FORWARD chain
|
|
}
|
|
```
|
|
|
|
### Anti-pattern
|
|
```go
|
|
// WRONG — all rules go to FORWARD, server IP rules are dead
|
|
fw.AddFirewallRule(...) // for dest=10.172.21.1 → enters FORWARD chain → never matched
|
|
```
|
|
|
|
### Correct pattern
|
|
```go
|
|
// CORRECT — detect server IP, route to correct chain
|
|
if destBase == serverIP {
|
|
fw.AddInputFirewallRule(...) // → INPUT chain (matches traffic TO server)
|
|
} else {
|
|
fw.AddFirewallRule(...) // → FORWARD chain (matches traffic THROUGH server)
|
|
}
|
|
```
|
|
|
|
### Why
|
|
Traffic to server's own IP is processed by INPUT chain, not FORWARD chain. Placing rules in FORWARD chain for server-bound traffic makes them dead rules that never match.
|
|
|
|
## UNIQUE STYLES
|
|
- **Zero-Attack Surface**: `/auth/register` locked; admin via `-create-admin` CLI only
|
|
- **Stealth Agent**: No `/etc/wireguard/` — config in memory only
|
|
- **Duplicate crypto**: `encryptor.go` copy-pasted in server-core + device-agent (known debt, do not deduplicate)
|
|
- **nftables default Accept**: Contradicts Zero-Trust "Default DROP" — intentional gap
|
|
|
|
## COMMANDS
|
|
```bash
|
|
# Docker
|
|
make up # Start all services
|
|
make down # Stop all services
|
|
make dev # Start with hot-reload (air)
|
|
make logs # Tail all logs
|
|
make migrate # Run DB migration (requires local Go)
|
|
make reset-db # Nuke PG volume + recreate + migrate
|
|
|
|
bash update.sh # Smart update (rebuild only if changes)
|
|
bash update.sh --force # Force rebuild
|
|
|
|
# Native Install
|
|
sudo bash nexusguard-install.sh
|
|
sudo bash nexusguard-uninstall.sh
|
|
sudo bash nexusguard-uninstall.sh --remove-db
|
|
|
|
# Development
|
|
cd apps/server-core && go run -tags dev .
|
|
cd apps/dashboard-ui && npm run dev
|
|
cd apps/device-agent && go run .
|
|
|
|
# Admin
|
|
go run -tags dev ./apps/server-core -create-admin -user admin -pass "..."
|
|
sudo /usr/local/bin/nexusguard-server -create-admin -user admin -pass "..."
|
|
```
|
|
|
|
## NOTES
|
|
- Submodules → private Gitea (`git.datadunia.com`); CI via Gitea Actions per submodule
|
|
- Go versions diverge: server-core `1.25.7`, device-agent `1.25.1`
|
|
- No root linter configs (`.golangci.yml`, `.eslintrc`, `.editorconfig`)
|
|
- Shell scripts use deprecated `docker-compose` v1, Makefile uses `docker compose` v2
|
|
- `package.json` name is `"temp-ui"` (stale scaffold remnant)
|