devops/wireguard-vpn

Fork 0

Files

T

datadunia 3eae539a56 feat: WGRplane Hybrid - Go-native with nftables + Multi-Webhook

2026-05-03 18:03:38 +07:00

18 KiB

Raw Blame History

Plan: WireGuard Remote Plane Control (WGRplane) Integration

Metadata

Field	Value
Plan ID	`wireguard-remote-plane-wgrplane`
Date	2026-05-03
Planner	Prometheus
Status	Ready for Execution

Goal

Build WireGuard Remote Plane Control app (WGRplane) — identical feature parity with WGDashboard (donaldzou/WGDashboard) — with added policy.json API integration. Deploy as git submodule at /app from https://git.datadunia.com/hainzero/WGRplane.git. Create README and push submodule for initialization.

Scope

IN (Explicitly Included)

Initialize /app as submodule from https://git.datadunia.com/hainzero/WGRplane.git
Full WGDashboard feature parity: peer CRUD, QR codes, real-time monitoring, scheduling, TOTP auth, multi-server, plugins, i18n, themes
New wg-engine-api (Go/Golang) with policy.json API endpoints
Policy.json API: GET /api/policy, POST /api/policy, reload trigger
#Access migration: API with #Access fallback (API tries first, falls back to #Access comment parsing)
Custom API authentication: wg-rplane-datadunia header
Atomic writes + flock locking (follow existing patterns from wg-policy-lib.sh)
Create README.md for submodule with full documentation
Push submodule to remote for initialization
Test strategy: bats for shell scripts, Go testing for wg-engine-api

OUT (Explicitly Excluded)

NO modifications to existing .sh scripts (wg-sync-policy.sh, wg-policy-engine.sh, etc.)
No CI pipeline (none exists in repo; document manual test commands instead)
No changes to core shell script logic (only new components)

Key Decisions (from Interview)

Decision	Choice	Rationale
wg-engine-api tech stack	Go (Golang)	Compiled binary, single executable, lightweight, different from WGDashboard's Python
Migration strategy	API with #Access fallback	API tries first, falls back to `#Access` comment parsing. Maximum compatibility.
Feature scope	Full WGDashboard Parity	All features: peer CRUD, QR codes, scheduling, TOTP, multi-server, plugins, etc.
Authentication	Custom header: `wg-rplane-datadunia`	Header-based auth per user spec
API storage	Separate `api-policy.json`	API-managed policies stored separately, merged with `#Access` at runtime
API port	10087	Avoid conflict with WGDashboard's default 10086
Policy merge logic	API overrides #Access	For same client IP, API policy takes precedence over `#Access` comment

Architecture & Data Flow (New)

wg0.conf (with/without #Access)
        ↓
wg-engine-api (Go) -- reads API storage (api-policy.json)
        ↓                             ↓
        +-- GET /api/policy (merged: API + #Access fallback)
        +-- POST /api/policy (writes to api-policy.json, triggers sync)
        ↓
policy.json (merged: API overrides #Access)
        ↓
wg-policy-engine.sh (unchanged)
        ↓
iptables / ipset rules

Key Changes from Original Flow:

New: wg-engine-api (Go) becomes PRIMARY generator of policy.json
Merge Logic: wg-engine-api reads both api-policy.json (API-managed) and wg0.conf (#Access), merges them (API overrides)
Fallback: If no API policy exists for a client, fall back to #Access comment
Locking: wg-engine-api uses SAME lock file (/var/lock/wg-policy.lock) with flock
Atomic Writes: Follow pattern from wg-policy-lib.sh (write to tmp, then mv)

Task Sections

Phase A: Submodule Initialization

Task A1: Initialize WGRplane submodule at /app
- File: /app (new submodule directory)
- Command: git submodule add https://git.datadunia.com/hainzero/WGRplane.git app
- Followed by: git submodule update --init --recursive
- QA: git submodule status shows app with commit hash, no errors
- QA: /app directory exists with WGRplane files (Python/Flask backend, Vue.js frontend)
Task A2: Verify WGRplane structure and dependencies
- File: /app (submodule contents)
- Inspect: ls /app — should contain Python backend, Vue.js frontend, requirements.txt
- Verify WGDashboard-equivalent structure: app.py or similar Flask entry point
- QA: WGRplane files present, Python/Flask + Vue.js stack confirmed
- QA: cat /app/requirements.txt shows Flask, SQLite, other dependencies
Task A3: Create Go module for wg-engine-api in /app
- File: /app/wg-engine-api/main.go (new)
- File: /app/wg-engine-api/go.mod (new)
- Command: cd /app/wg-engine-api && go mod init git.datadunia.com/hainzero/WGRplane/wg-engine-api
- Dependencies: github.com/gorilla/mux (router), github.com/coreos/go-systemd (optional)
- QA: ls /app/wg-engine-api/ shows main.go, go.mod, go.sum
- QA: cd /app/wg-engine-api && go build succeeds without errors

Phase B: WGRplane Base Setup (WGDashboard Parity)

Task B1: Review WGDashboard features for parity checklist
- Reference: Librarian findings (bg_f53966bd) — full feature list
- Features to implement: peer CRUD, QR codes, real-time monitoring, scheduling, TOTP auth, multi-server, plugins, i18n, themes
- File: /app/README.md (document feature parity status)
- QA: Checklist created with ALL WGDashboard features mapped to WGRplane implementation status
Task B2: Configure WGRplane to use port 10086 (WGDashboard default)
- File: /app/app.py or /app/config.json (WGRplane config)
- Set: app_port = 10086 (consistent with WGDashboard)
- Ensure: Does not conflict with wg-engine-api on port 10087
- QA: curl http://localhost:10086 returns WGRplane dashboard page
- QA: Port 10086 in use by WGRplane, 10087 available for wg-engine-api
Task B3: Integrate WGRplane with existing WireGuard config path
- File: /app/app.py (WGRplane backend)
- Set WireGuard config path: /etc/wireguard/wg0.conf (consistent with existing scripts)
- QA: WGRplane can read /etc/wireguard/wg0.conf and list peers
- QA: WGRplane "Add Peer" creates valid WireGuard config entries
Task B4: Add policy.json API awareness to WGRplane frontend
- File: /app/src/views/ or /app/src/components/ (Vue.js components)
- Add: New UI section for "Policy API" (link to http://localhost:10087/api/policy)
- Note: WGRplane frontend will proxy or link to Go API (decision: proxy via Flask or direct link)
- QA: WGRplane UI shows "Policy API" section with link to localhost:10087
- QA: Clicking link opens http://localhost:10087/api/policy (with auth header)

Phase C: Go wg-engine-api Development

Task C1: Implement Go API server skeleton with routing
- File: /app/wg-engine-api/main.go
- Framework: github.com/gorilla/mux (router)
- Port: 10087 (avoid conflict with WGDashboard's 10086)
- Endpoints skeleton: GET /api/policy, POST /api/policy, POST /api/reload
- Auth middleware: Check wg-rplane-datadunia header
- QA: go build succeeds, binary runs on port 10087
- QA: curl -H "wg-rplane-datadunia: test" http://localhost:10087/api/policy returns 200 or 401 (if auth enforced)
Task C2: Implement locking mechanism (flock) in Go
- File: /app/wg-engine-api/main.go (lock function)
- Lock file: /var/lock/wg-policy.lock (SAME as existing scripts)
- Implementation: Use syscall.Flock() or exec flock command
- Follow pattern from wg-policy-lib.sh: flock -x -w 10
- QA: Simultaneous API calls do not corrupt policy.json
- QA: Lock acquired within 10 seconds, else return 503 (timeout)
Task C3: Implement atomic write for policy.json in Go
- File: /app/wg-engine-api/main.go (write function)
- Pattern: Write to tmp file → mv (atomic, same filesystem)
- Reference: wg-sync-policy.sh lines 124-126: mv -f "$tmp_policy" "$POLICY_FILE"
- Tmp path: /etc/wireguard/policy.json.tmp
- QA: policy.json never partially written (crash during write doesn't corrupt)
- QA: jq empty /etc/wireguard/policy.json validates JSON after write
Task C4: Add CLI flag for sync without HTTP server
- File: /app/wg-engine-api/main.go (flag parsing)
- Flag: --sync (perform merge + write to policy.json, then exit)
- Use case: Called by wg-policy.service instead of wg-sync-policy.sh
- QA: ./wg-engine-api --sync exits 0, updates policy.json
- QA: After --sync, wg-policy-ctl policy shows merged data

Phase D: Policy API Implementation

Task D1: Implement GET /api/policy (merged: API + #Access fallback)
- File: /app/wg-engine-api/main.go (GET handler)
- Step 1: Read API storage (/etc/wireguard/api-policy.json)
- Step 2: Parse wg0.conf for #Access comments (fallback, using Go or exec wg-sync-policy.sh)
- Step 3: Merge (API entries OVERRIDE #Access for same IP)
- Step 4: Return merged JSON with same structure as policy.json
- QA: curl -H "wg-rplane-datadunia: VALID" http://localhost:10087/api/policy returns merged JSON
- QA: Client with API policy + #Access → API policy wins in response
- QA: Client with ONLY #Access → fallback returns #Access value
Task D2: Implement POST /api/policy (update API-managed policy)
- File: /app/wg-engine-api/main.go (POST handler)
- Input: JSON body {"ip": "10.0.0.2", "access": ["1.1.1.1/32"], "internet": true}
- Validate: IP and CIDRs using Go validation functions (port from wg-policy-lib.sh)
- Save to: /etc/wireguard/api-policy.json (API-managed storage)
- After save: Acquire lock → read/merge → atomic write to policy.json → trigger wg-policy-engine.sh
- QA: POST returns 200, api-policy.json updated
- QA: policy.json updated with merged data (API overrides #Access)
- QA: wg-policy-ctl rules shows new targets after POST
Task D3: Implement POST /api/reload (trigger policy engine)
- File: /app/wg-engine-api/main.go (reload handler)
- Action: Exec /usr/local/bin/wg-policy-engine.sh
- Optional: Also exec /usr/local/bin/wg-sync-policy.sh first (if #Access fallback needed)
- QA: curl -X POST -H "wg-rplane-datadunia: VALID" http://localhost:10087/api/reload returns 200
- QA: After reload, wg-policy-ctl status shows engine applied successfully
Task D4: Implement authentication middleware
- File: /app/wg-engine-api/main.go (middleware)
- Header: wg-rplane-datadunia
- Validation: Check header exists and matches configured token (from env or config file)
- Return: 401 Unauthorized if missing/invalid
- QA: curl http://localhost:10087/api/policy (no header) → 401
- QA: curl -H "wg-rplane-datadunia: wrong" http://localhost:10087/api/policy → 401
- QA: curl -H "wg-rplane-datadunia: VALID" http://localhost:10087/api/policy → 200
Task D5: Create API storage file (api-policy.json) with schema
- File: /etc/wireguard/api-policy.json (new, API-managed)
- Schema: Same as policy.json but ONLY API-managed entries:
```
{
  "clients": {
    "10.0.0.2": {
      "name": "10.0.0.2",
      "access": ["1.1.1.1/32"],
      "internet": true
    }
  }
}
```
- Initialize: Empty {"clients": {}} on first run
- QA: api-policy.json exists after first API call
- QA: JSON structure matches policy.json schema

Phase E: #Access Migration & Merge Logic

Task E1: Implement #Access comment parser in Go (fallback)
- File: /app/wg-engine-api/main.go (parse function)
- Method: Exec wg-sync-policy.sh OR parse wg0.conf directly in Go
- Prefer: Parse wg0.conf in Go (avoid exec dependency)
- Logic: Read [Peer] blocks, extract #Access and #Internet lines
- QA: Go parser extracts same data as wg-sync-policy.sh awk script
- QA: curl GET /api/policy with no API policy returns #Access data correctly
Task E2: Implement merge logic (API overrides #Access)
- File: /app/wg-engine-api/main.go (merge function)
- Logic: For each client IP:
  1. Start with #Access parsed data (fallback)
  2. Override with API-managed data (from api-policy.json)
  3. API takes precedence for same IP
- Output: Merged JSON matching policy.json structure
- QA: Client with API policy "access": ["1.1.1.1/32"] + #Access 2.2.2.2/32 → GET returns ["1.1.1.1/32"]
- QA: Client with ONLY #Access 2.2.2.2/32 → GET returns ["2.2.2.2/32"]
Task E3: Handle internet flag merge
- File: /app/wg-engine-api/main.go (merge function extension)
- Logic: Same as access merge — API internet flag overrides #Internet comment
- QA: Client with API internet: true + no #Internet in wg0.conf → GET returns true
- QA: Client with API internet: false + #Internet true in wg0.conf → GET returns false
Task E4: Update wg-policy.service to use Go API sync (optional, recommended)
- File: wg-policy.service (systemd unit)
- Change: ExecStartPre from wg-sync-policy.sh to wg-engine-api --sync
- Note: NOT modifying .sh scripts (only systemd unit)
- QA: systemctl daemon-reload && systemctl restart wg-policy.service succeeds
- QA: Service uses Go API for sync instead of shell script

Phase F: Integration & Testing

Task F1: Add bats test framework for shell script validation
- File: /tests/ (new directory) or use existing pattern
- Test cases: Policy.json validation, JSON structure, lock file behavior
- Install: apt install bats (add to install.sh if needed)
- QA: bats /tests/policy.bats passes all test cases
- QA: Test coverage for wg-policy-ctl validate command
Task F2: Add Go tests for wg-engine-api
- File: /app/wg-engine-api/main_test.go (new)
- Test cases: Auth middleware, GET/POST handlers, merge logic, lock mechanism
- Run: cd /app/wg-engine-api && go test ./...
- QA: go test passes with >80% coverage
- QA: Mock wg0.conf and api-policy.json for isolated tests
Task F3: Integration test: Full flow validation
- Test: POST to API → policy.json updated → iptables rules applied
- Steps:
  1. curl -X POST ... http://localhost:10087/api/policy (add client)
  2. Verify policy.json updated (check with wg-policy-ctl policy)
  3. Verify iptables rules (check with wg-policy-ctl rules)
- QA: All 3 steps succeed in sequence
- QA: Fallback to #Access works when API has no entry for client
Task F4: Manual test documentation in README
- File: /app/README.md (test section)
- Document: How to run bats tests, Go tests, manual QA scenarios
- Note: No CI (none exists in repo), document manual commands
- QA: README has clear "Testing" section with commands
- QA: New developer can follow README to run all tests

Phase G: Documentation & Push

Task G1: Create comprehensive README.md for WGRplane submodule
- File: /app/README.md (new or update existing)
- Sections: Overview, Architecture, API Endpoints, Authentication, Integration with WGDashboard, Testing, Deployment
- Document: Go API endpoint (http://localhost:10087/api/policy), auth header wg-rplane-datadunia
- QA: README.md exists with all sections
- QA: cat /app/README.md shows complete documentation
Task G2: Document integration between WGRplane (Python) and wg-engine-api (Go)
- File: /app/README.md (integration section)
- Explain: WGRplane on port 10086, Go API on port 10087
- Note: Frontend can proxy API requests or link directly
- QA: README has "Integration" section with port numbers and proxy examples
- QA: Developer understands how Python Flask talks to Go API
Task G3: Push WGRplane submodule to remote
- Commands:
```
cd /app
git add .
git commit -m "Init WGRplane submodule with Go wg-engine-api"
git push origin main  # or master, depending on remote default
```
- QA: git push succeeds, remote updated
- QA: git submodule status in parent repo shows app with commit hash
Task G4: Update parent repo to reference pushed submodule
- Commands:
```
cd /path/to/03.wireguard-policy
git add .gitmodules app
git commit -m "Add WGRplane submodule with policy.json API"
git push
```
- QA: Parent repo pushed with submodule reference
- QA: Fresh clone of parent repo can git submodule update --init --recursive successfully

Final Verification Wave

QA Scenarios (ALL must pass before marking work complete):

Submodule init: git submodule status shows app pointing to https://git.datadunia.com/hainzero/WGRplane.git
API auth: curl -H "wg-rplane-datadunia: wrong" http://localhost:10087/api/policy returns 401 Unauthorized
Policy retrieval: curl -H "wg-rplane-datadunia: VALID_TOKEN" http://localhost:10087/api/policy returns merged JSON (API + #Access fallback)
Policy update: curl -X POST -H "Content-Type: application/json" -H "wg-rplane-datadunia: VALID_TOKEN" -d '{"ip": "10.0.0.2", "access": ["1.1.1.1/32"]}' http://localhost:10087/api/policy returns 200 and updates policy.json
#Access fallback: Client with NO API policy but HAS #Access in wg0.conf → API returns #Access value in GET
iptables application: After POST, run wg-policy-ctl rules → new target visible in WG_POLICY chain
Lock conflict prevention: Simultaneous API call and wg-sync-policy.sh do not corrupt policy.json
README exists: /app/README.md present with full documentation
Submodule pushed: git push in /app succeeds, remote initialized

User Confirmation Required: Run ALL QA scenarios above and confirm "okay" before marking work complete.

18 KiB Raw Blame History

Plan: WireGuard Remote Plane Control (WGRplane) Integration

Metadata

Goal

Scope

IN (Explicitly Included)

OUT (Explicitly Excluded)

Key Decisions (from Interview)

Architecture & Data Flow (New)

Key Changes from Original Flow:

Task Sections

Phase A: Submodule Initialization

Phase B: WGRplane Base Setup (WGDashboard Parity)

Phase C: Go wg-engine-api Development

Phase D: Policy API Implementation

Phase E: #Access Migration & Merge Logic

Phase F: Integration & Testing

Phase G: Documentation & Push

Phase G: Documentation & Push

Final Verification Wave

18 KiB

Raw Blame History