feat: WGRplane Hybrid - Go-native with nftables + Multi-Webhook
This commit is contained in:
datadunia
+346
@@ -0,0 +1,346 @@
|
||||
# Plan: WireGuard Remote Plane Control (WGRplane) Integration
|
||||
|
||||
## Metadata
|
||||
|
||||
| Field | Value |
|
||||
|-------|-------|
|
||||
| **Plan ID** | `wireguard-remote-plane-wgrplane` |
|
||||
| **Date** | 2026-05-03 |
|
||||
| **Planner** | Prometheus |
|
||||
| **Status** | Ready for Execution |
|
||||
|
||||
---
|
||||
|
||||
## Goal
|
||||
|
||||
Build WireGuard Remote Plane Control app (**WGRplane**) — identical feature parity with **WGDashboard** (donaldzou/WGDashboard) — with added **policy.json API** integration. Deploy as git submodule at `/app` from `https://git.datadunia.com/hainzero/WGRplane.git`. Create README and push submodule for initialization.
|
||||
|
||||
---
|
||||
|
||||
## Scope
|
||||
|
||||
### IN (Explicitly Included)
|
||||
- Initialize `/app` as submodule from `https://git.datadunia.com/hainzero/WGRplane.git`
|
||||
- Full WGDashboard feature parity: peer CRUD, QR codes, real-time monitoring, scheduling, TOTP auth, multi-server, plugins, i18n, themes
|
||||
- New **`wg-engine-api`** (Go/Golang) with policy.json API endpoints
|
||||
- Policy.json API: `GET /api/policy`, `POST /api/policy`, reload trigger
|
||||
- **`#Access` migration**: API with `#Access` fallback (API tries first, falls back to `#Access` comment parsing)
|
||||
- Custom API authentication: `wg-rplane-datadunia` header
|
||||
- Atomic writes + flock locking (follow existing patterns from `wg-policy-lib.sh`)
|
||||
- Create README.md for submodule with full documentation
|
||||
- Push submodule to remote for initialization
|
||||
- Test strategy: `bats` for shell scripts, Go testing for `wg-engine-api`
|
||||
|
||||
### OUT (Explicitly Excluded)
|
||||
- **NO modifications to existing `.sh` scripts** (`wg-sync-policy.sh`, `wg-policy-engine.sh`, etc.)
|
||||
- No CI pipeline (none exists in repo; document manual test commands instead)
|
||||
- No changes to core shell script logic (only new components)
|
||||
|
||||
---
|
||||
|
||||
## Key Decisions (from Interview)
|
||||
|
||||
| Decision | Choice | Rationale |
|
||||
|----------|--------|-----------|
|
||||
| **wg-engine-api tech stack** | **Go (Golang)** | Compiled binary, single executable, lightweight, different from WGDashboard's Python |
|
||||
| **Migration strategy** | **API with #Access fallback** | API tries first, falls back to `#Access` comment parsing. Maximum compatibility. |
|
||||
| **Feature scope** | **Full WGDashboard Parity** | All features: peer CRUD, QR codes, scheduling, TOTP, multi-server, plugins, etc. |
|
||||
| **Authentication** | **Custom header: `wg-rplane-datadunia`** | Header-based auth per user spec |
|
||||
| **API storage** | **Separate `api-policy.json`** | API-managed policies stored separately, merged with `#Access` at runtime |
|
||||
| **API port** | **10087** | Avoid conflict with WGDashboard's default 10086 |
|
||||
| **Policy merge logic** | **API overrides #Access** | For same client IP, API policy takes precedence over `#Access` comment |
|
||||
|
||||
---
|
||||
|
||||
## Architecture & Data Flow (New)
|
||||
|
||||
```
|
||||
wg0.conf (with/without #Access)
|
||||
↓
|
||||
wg-engine-api (Go) -- reads API storage (api-policy.json)
|
||||
↓ ↓
|
||||
+-- GET /api/policy (merged: API + #Access fallback)
|
||||
+-- POST /api/policy (writes to api-policy.json, triggers sync)
|
||||
↓
|
||||
policy.json (merged: API overrides #Access)
|
||||
↓
|
||||
wg-policy-engine.sh (unchanged)
|
||||
↓
|
||||
iptables / ipset rules
|
||||
```
|
||||
|
||||
### Key Changes from Original Flow:
|
||||
1. **New**: `wg-engine-api` (Go) becomes PRIMARY generator of `policy.json`
|
||||
2. **Merge Logic**: `wg-engine-api` reads both `api-policy.json` (API-managed) and `wg0.conf` (`#Access`), merges them (API overrides)
|
||||
3. **Fallback**: If no API policy exists for a client, fall back to `#Access` comment
|
||||
4. **Locking**: `wg-engine-api` uses SAME lock file (`/var/lock/wg-policy.lock`) with `flock`
|
||||
5. **Atomic Writes**: Follow pattern from `wg-policy-lib.sh` (write to tmp, then `mv`)
|
||||
|
||||
---
|
||||
|
||||
## Task Sections
|
||||
|
||||
### Phase A: Submodule Initialization
|
||||
|
||||
- [ ] **Task A1**: Initialize WGRplane submodule at `/app`
|
||||
- File: `/app` (new submodule directory)
|
||||
- Command: `git submodule add https://git.datadunia.com/hainzero/WGRplane.git app`
|
||||
- Followed by: `git submodule update --init --recursive`
|
||||
- QA: `git submodule status` shows `app` with commit hash, no errors
|
||||
- QA: `/app` directory exists with WGRplane files (Python/Flask backend, Vue.js frontend)
|
||||
|
||||
- [ ] **Task A2**: Verify WGRplane structure and dependencies
|
||||
- File: `/app` (submodule contents)
|
||||
- Inspect: `ls /app` — should contain Python backend, Vue.js frontend, requirements.txt
|
||||
- Verify WGDashboard-equivalent structure: `app.py` or similar Flask entry point
|
||||
- QA: WGRplane files present, Python/Flask + Vue.js stack confirmed
|
||||
- QA: `cat /app/requirements.txt` shows Flask, SQLite, other dependencies
|
||||
|
||||
- [ ] **Task A3**: Create Go module for `wg-engine-api` in `/app`
|
||||
- File: `/app/wg-engine-api/main.go` (new)
|
||||
- File: `/app/wg-engine-api/go.mod` (new)
|
||||
- Command: `cd /app/wg-engine-api && go mod init git.datadunia.com/hainzero/WGRplane/wg-engine-api`
|
||||
- Dependencies: `github.com/gorilla/mux` (router), `github.com/coreos/go-systemd` (optional)
|
||||
- QA: `ls /app/wg-engine-api/` shows `main.go`, `go.mod`, `go.sum`
|
||||
- QA: `cd /app/wg-engine-api && go build` succeeds without errors
|
||||
|
||||
### Phase B: WGRplane Base Setup (WGDashboard Parity)
|
||||
|
||||
- [ ] **Task B1**: Review WGDashboard features for parity checklist
|
||||
- Reference: Librarian findings (bg_f53966bd) — full feature list
|
||||
- Features to implement: peer CRUD, QR codes, real-time monitoring, scheduling, TOTP auth, multi-server, plugins, i18n, themes
|
||||
- File: `/app/README.md` (document feature parity status)
|
||||
- QA: Checklist created with ALL WGDashboard features mapped to WGRplane implementation status
|
||||
|
||||
- [ ] **Task B2**: Configure WGRplane to use port 10086 (WGDashboard default)
|
||||
- File: `/app/app.py` or `/app/config.json` (WGRplane config)
|
||||
- Set: `app_port = 10086` (consistent with WGDashboard)
|
||||
- Ensure: Does not conflict with `wg-engine-api` on port 10087
|
||||
- QA: `curl http://localhost:10086` returns WGRplane dashboard page
|
||||
- QA: Port 10086 in use by WGRplane, 10087 available for wg-engine-api
|
||||
|
||||
- [ ] **Task B3**: Integrate WGRplane with existing WireGuard config path
|
||||
- File: `/app/app.py` (WGRplane backend)
|
||||
- Set WireGuard config path: `/etc/wireguard/wg0.conf` (consistent with existing scripts)
|
||||
- QA: WGRplane can read `/etc/wireguard/wg0.conf` and list peers
|
||||
- QA: WGRplane "Add Peer" creates valid WireGuard config entries
|
||||
|
||||
- [ ] **Task B4**: Add policy.json API awareness to WGRplane frontend
|
||||
- File: `/app/src/views/` or `/app/src/components/` (Vue.js components)
|
||||
- Add: New UI section for "Policy API" (link to `http://localhost:10087/api/policy`)
|
||||
- Note: WGRplane frontend will proxy or link to Go API (decision: proxy via Flask or direct link)
|
||||
- QA: WGRplane UI shows "Policy API" section with link to `localhost:10087`
|
||||
- QA: Clicking link opens `http://localhost:10087/api/policy` (with auth header)
|
||||
|
||||
### Phase C: Go wg-engine-api Development
|
||||
|
||||
- [ ] **Task C1**: Implement Go API server skeleton with routing
|
||||
- File: `/app/wg-engine-api/main.go`
|
||||
- Framework: `github.com/gorilla/mux` (router)
|
||||
- Port: **10087** (avoid conflict with WGDashboard's 10086)
|
||||
- Endpoints skeleton: `GET /api/policy`, `POST /api/policy`, `POST /api/reload`
|
||||
- Auth middleware: Check `wg-rplane-datadunia` header
|
||||
- QA: `go build` succeeds, binary runs on port 10087
|
||||
- QA: `curl -H "wg-rplane-datadunia: test" http://localhost:10087/api/policy` returns 200 or 401 (if auth enforced)
|
||||
|
||||
- [ ] **Task C2**: Implement locking mechanism (flock) in Go
|
||||
- File: `/app/wg-engine-api/main.go` (lock function)
|
||||
- Lock file: `/var/lock/wg-policy.lock` (SAME as existing scripts)
|
||||
- Implementation: Use `syscall.Flock()` or exec `flock` command
|
||||
- Follow pattern from `wg-policy-lib.sh`: `flock -x -w 10`
|
||||
- QA: Simultaneous API calls do not corrupt `policy.json`
|
||||
- QA: Lock acquired within 10 seconds, else return 503 (timeout)
|
||||
|
||||
- [ ] **Task C3**: Implement atomic write for policy.json in Go
|
||||
- File: `/app/wg-engine-api/main.go` (write function)
|
||||
- Pattern: Write to tmp file → `mv` (atomic, same filesystem)
|
||||
- Reference: `wg-sync-policy.sh` lines 124-126: `mv -f "$tmp_policy" "$POLICY_FILE"`
|
||||
- Tmp path: `/etc/wireguard/policy.json.tmp`
|
||||
- QA: `policy.json` never partially written (crash during write doesn't corrupt)
|
||||
- QA: `jq empty /etc/wireguard/policy.json` validates JSON after write
|
||||
|
||||
- [ ] **Task C4**: Add CLI flag for sync without HTTP server
|
||||
- File: `/app/wg-engine-api/main.go` (flag parsing)
|
||||
- Flag: `--sync` (perform merge + write to `policy.json`, then exit)
|
||||
- Use case: Called by `wg-policy.service` instead of `wg-sync-policy.sh`
|
||||
- QA: `./wg-engine-api --sync` exits 0, updates `policy.json`
|
||||
- QA: After `--sync`, `wg-policy-ctl policy` shows merged data
|
||||
|
||||
### Phase D: Policy API Implementation
|
||||
|
||||
- [ ] **Task D1**: Implement `GET /api/policy` (merged: API + #Access fallback)
|
||||
- File: `/app/wg-engine-api/main.go` (GET handler)
|
||||
- Step 1: Read API storage (`/etc/wireguard/api-policy.json`)
|
||||
- Step 2: Parse `wg0.conf` for `#Access` comments (fallback, using Go or exec `wg-sync-policy.sh`)
|
||||
- Step 3: Merge (API entries OVERRIDE `#Access` for same IP)
|
||||
- Step 4: Return merged JSON with same structure as `policy.json`
|
||||
- QA: `curl -H "wg-rplane-datadunia: VALID" http://localhost:10087/api/policy` returns merged JSON
|
||||
- QA: Client with API policy + `#Access` → API policy wins in response
|
||||
- QA: Client with ONLY `#Access` → fallback returns `#Access` value
|
||||
|
||||
- [ ] **Task D2**: Implement `POST /api/policy` (update API-managed policy)
|
||||
- File: `/app/wg-engine-api/main.go` (POST handler)
|
||||
- Input: JSON body `{"ip": "10.0.0.2", "access": ["1.1.1.1/32"], "internet": true}`
|
||||
- Validate: IP and CIDRs using Go validation functions (port from `wg-policy-lib.sh`)
|
||||
- Save to: `/etc/wireguard/api-policy.json` (API-managed storage)
|
||||
- After save: Acquire lock → read/merge → atomic write to `policy.json` → trigger `wg-policy-engine.sh`
|
||||
- QA: POST returns 200, `api-policy.json` updated
|
||||
- QA: `policy.json` updated with merged data (API overrides #Access)
|
||||
- QA: `wg-policy-ctl rules` shows new targets after POST
|
||||
|
||||
- [ ] **Task D3**: Implement `POST /api/reload` (trigger policy engine)
|
||||
- File: `/app/wg-engine-api/main.go` (reload handler)
|
||||
- Action: Exec `/usr/local/bin/wg-policy-engine.sh`
|
||||
- Optional: Also exec `/usr/local/bin/wg-sync-policy.sh` first (if #Access fallback needed)
|
||||
- QA: `curl -X POST -H "wg-rplane-datadunia: VALID" http://localhost:10087/api/reload` returns 200
|
||||
- QA: After reload, `wg-policy-ctl status` shows engine applied successfully
|
||||
|
||||
- [ ] **Task D4**: Implement authentication middleware
|
||||
- File: `/app/wg-engine-api/main.go` (middleware)
|
||||
- Header: `wg-rplane-datadunia`
|
||||
- Validation: Check header exists and matches configured token (from env or config file)
|
||||
- Return: 401 Unauthorized if missing/invalid
|
||||
- QA: `curl http://localhost:10087/api/policy` (no header) → 401
|
||||
- QA: `curl -H "wg-rplane-datadunia: wrong" http://localhost:10087/api/policy` → 401
|
||||
- QA: `curl -H "wg-rplane-datadunia: VALID" http://localhost:10087/api/policy` → 200
|
||||
|
||||
- [ ] **Task D5**: Create API storage file (`api-policy.json`) with schema
|
||||
- File: `/etc/wireguard/api-policy.json` (new, API-managed)
|
||||
- Schema: Same as `policy.json` but ONLY API-managed entries:
|
||||
```json
|
||||
{
|
||||
"clients": {
|
||||
"10.0.0.2": {
|
||||
"name": "10.0.0.2",
|
||||
"access": ["1.1.1.1/32"],
|
||||
"internet": true
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
- Initialize: Empty `{"clients": {}}` on first run
|
||||
- QA: `api-policy.json` exists after first API call
|
||||
- QA: JSON structure matches `policy.json` schema
|
||||
|
||||
### Phase E: #Access Migration & Merge Logic
|
||||
|
||||
- [ ] **Task E1**: Implement #Access comment parser in Go (fallback)
|
||||
- File: `/app/wg-engine-api/main.go` (parse function)
|
||||
- Method: Exec `wg-sync-policy.sh` OR parse `wg0.conf` directly in Go
|
||||
- Prefer: Parse `wg0.conf` in Go (avoid exec dependency)
|
||||
- Logic: Read `[Peer]` blocks, extract `#Access` and `#Internet` lines
|
||||
- QA: Go parser extracts same data as `wg-sync-policy.sh` awk script
|
||||
- QA: `curl GET /api/policy` with no API policy returns `#Access` data correctly
|
||||
|
||||
- [ ] **Task E2**: Implement merge logic (API overrides #Access)
|
||||
- File: `/app/wg-engine-api/main.go` (merge function)
|
||||
- Logic: For each client IP:
|
||||
1. Start with `#Access` parsed data (fallback)
|
||||
2. Override with API-managed data (from `api-policy.json`)
|
||||
3. API takes precedence for same IP
|
||||
- Output: Merged JSON matching `policy.json` structure
|
||||
- QA: Client with API policy `"access": ["1.1.1.1/32"]` + `#Access 2.2.2.2/32` → GET returns `["1.1.1.1/32"]`
|
||||
- QA: Client with ONLY `#Access 2.2.2.2/32` → GET returns `["2.2.2.2/32"]`
|
||||
|
||||
- [ ] **Task E3**: Handle `internet` flag merge
|
||||
- File: `/app/wg-engine-api/main.go` (merge function extension)
|
||||
- Logic: Same as access merge — API `internet` flag overrides `#Internet` comment
|
||||
- QA: Client with API `internet: true` + no `#Internet` in wg0.conf → GET returns `true`
|
||||
- QA: Client with API `internet: false` + `#Internet true` in wg0.conf → GET returns `false`
|
||||
|
||||
- [ ] **Task E4**: Update `wg-policy.service` to use Go API sync (optional, recommended)
|
||||
- File: `wg-policy.service` (systemd unit)
|
||||
- Change: `ExecStartPre` from `wg-sync-policy.sh` to `wg-engine-api --sync`
|
||||
- Note: NOT modifying `.sh` scripts (only systemd unit)
|
||||
- QA: `systemctl daemon-reload && systemctl restart wg-policy.service` succeeds
|
||||
- QA: Service uses Go API for sync instead of shell script
|
||||
|
||||
### Phase F: Integration & Testing
|
||||
|
||||
- [ ] **Task F1**: Add `bats` test framework for shell script validation
|
||||
- File: `/tests/` (new directory) or use existing pattern
|
||||
- Test cases: Policy.json validation, JSON structure, lock file behavior
|
||||
- Install: `apt install bats` (add to `install.sh` if needed)
|
||||
- QA: `bats /tests/policy.bats` passes all test cases
|
||||
- QA: Test coverage for `wg-policy-ctl validate` command
|
||||
|
||||
- [ ] **Task F2**: Add Go tests for `wg-engine-api`
|
||||
- File: `/app/wg-engine-api/main_test.go` (new)
|
||||
- Test cases: Auth middleware, GET/POST handlers, merge logic, lock mechanism
|
||||
- Run: `cd /app/wg-engine-api && go test ./...`
|
||||
- QA: `go test` passes with >80% coverage
|
||||
- QA: Mock `wg0.conf` and `api-policy.json` for isolated tests
|
||||
|
||||
- [ ] **Task F3**: Integration test: Full flow validation
|
||||
- Test: POST to API → policy.json updated → iptables rules applied
|
||||
- Steps:
|
||||
1. `curl -X POST ... http://localhost:10087/api/policy` (add client)
|
||||
2. Verify `policy.json` updated (check with `wg-policy-ctl policy`)
|
||||
3. Verify iptables rules (check with `wg-policy-ctl rules`)
|
||||
- QA: All 3 steps succeed in sequence
|
||||
- QA: Fallback to `#Access` works when API has no entry for client
|
||||
|
||||
- [ ] **Task F4**: Manual test documentation in README
|
||||
- File: `/app/README.md` (test section)
|
||||
- Document: How to run bats tests, Go tests, manual QA scenarios
|
||||
- Note: No CI (none exists in repo), document manual commands
|
||||
- QA: README has clear "Testing" section with commands
|
||||
- QA: New developer can follow README to run all tests
|
||||
|
||||
### Phase G: Documentation & Push
|
||||
|
||||
### Phase G: Documentation & Push
|
||||
|
||||
- [ ] **Task G1**: Create comprehensive README.md for WGRplane submodule
|
||||
- File: `/app/README.md` (new or update existing)
|
||||
- Sections: Overview, Architecture, API Endpoints, Authentication, Integration with WGDashboard, Testing, Deployment
|
||||
- Document: Go API endpoint (`http://localhost:10087/api/policy`), auth header `wg-rplane-datadunia`
|
||||
- QA: README.md exists with all sections
|
||||
- QA: `cat /app/README.md` shows complete documentation
|
||||
|
||||
- [ ] **Task G2**: Document integration between WGRplane (Python) and wg-engine-api (Go)
|
||||
- File: `/app/README.md` (integration section)
|
||||
- Explain: WGRplane on port 10086, Go API on port 10087
|
||||
- Note: Frontend can proxy API requests or link directly
|
||||
- QA: README has "Integration" section with port numbers and proxy examples
|
||||
- QA: Developer understands how Python Flask talks to Go API
|
||||
|
||||
- [ ] **Task G3**: Push WGRplane submodule to remote
|
||||
- Commands:
|
||||
```bash
|
||||
cd /app
|
||||
git add .
|
||||
git commit -m "Init WGRplane submodule with Go wg-engine-api"
|
||||
git push origin main # or master, depending on remote default
|
||||
```
|
||||
- QA: `git push` succeeds, remote updated
|
||||
- QA: `git submodule status` in parent repo shows app with commit hash
|
||||
|
||||
- [ ] **Task G4**: Update parent repo to reference pushed submodule
|
||||
- Commands:
|
||||
```bash
|
||||
cd /path/to/03.wireguard-policy
|
||||
git add .gitmodules app
|
||||
git commit -m "Add WGRplane submodule with policy.json API"
|
||||
git push
|
||||
```
|
||||
- QA: Parent repo pushed with submodule reference
|
||||
- QA: Fresh clone of parent repo can `git submodule update --init --recursive` successfully
|
||||
|
||||
---
|
||||
|
||||
## Final Verification Wave
|
||||
|
||||
**QA Scenarios (ALL must pass before marking work complete):**
|
||||
|
||||
1. **Submodule init**: `git submodule status` shows `app` pointing to `https://git.datadunia.com/hainzero/WGRplane.git`
|
||||
2. **API auth**: `curl -H "wg-rplane-datadunia: wrong" http://localhost:10087/api/policy` returns **401 Unauthorized**
|
||||
3. **Policy retrieval**: `curl -H "wg-rplane-datadunia: VALID_TOKEN" http://localhost:10087/api/policy` returns merged JSON (API + #Access fallback)
|
||||
4. **Policy update**: `curl -X POST -H "Content-Type: application/json" -H "wg-rplane-datadunia: VALID_TOKEN" -d '{"ip": "10.0.0.2", "access": ["1.1.1.1/32"]}' http://localhost:10087/api/policy` returns **200** and updates `policy.json`
|
||||
5. **#Access fallback**: Client with NO API policy but HAS `#Access` in `wg0.conf` → API returns `#Access` value in GET
|
||||
6. **iptables application**: After POST, run `wg-policy-ctl rules` → new target visible in `WG_POLICY` chain
|
||||
7. **Lock conflict prevention**: Simultaneous API call and `wg-sync-policy.sh` do not corrupt `policy.json`
|
||||
8. **README exists**: `/app/README.md` present with full documentation
|
||||
9. **Submodule pushed**: `git push` in `/app` succeeds, remote initialized
|
||||
|
||||
**User Confirmation Required**: Run ALL QA scenarios above and confirm **"okay"** before marking work complete.
|
||||
Reference in New Issue
Block a user