Mesh Networking
Mesh Networking
Section titled “Mesh Networking”Bifrost supports optional Hamachi-like mesh networking for creating virtual LANs between peers. This feature enables direct peer-to-peer connectivity with automatic NAT traversal, encryption, and routing.
Overview
Section titled “Overview”The mesh networking feature provides:
- Virtual LAN: Create private networks between distributed peers
- NAT Traversal: Automatic hole-punching with STUN/TURN/ICE support
- P2P Encryption: All traffic encrypted with ChaCha20-Poly1305
- Mesh Routing: Automatic route discovery and multi-hop relaying
- TAP/TUN Device Support: Layer 2 (Ethernet) and Layer 3 (IP) networking
High-Level Architecture
Section titled “High-Level Architecture”flowchart TB
subgraph LocalPeer["Local Peer"]
App[Application]
Device[TUN/TAP Device]
MeshNode[Mesh Node]
P2PManager[P2P Manager]
Router[Mesh Router]
Protocol[Routing Protocol]
end
subgraph Infrastructure["Infrastructure Services"]
Discovery[Discovery Server]
STUN[STUN Servers]
TURN[TURN Relay]
end
subgraph RemotePeers["Remote Peers"]
PeerA[Peer A]
PeerB[Peer B]
PeerC[Peer C]
end
App <--> Device
Device <--> MeshNode
MeshNode --> P2PManager
MeshNode --> Router
Router --> Protocol
P2PManager <-.->|Register/Heartbeat| Discovery
P2PManager <-.->|NAT Detection| STUN
P2PManager <-.->|Relay Allocation| TURN
P2PManager <-->|Direct UDP| PeerA
P2PManager <-->|Direct UDP| PeerB
P2PManager <-.->|Via TURN| PeerC
style MeshNode fill:#4a90e2,stroke:#2c5aa0,color:#fff
style P2PManager fill:#7b68ee,stroke:#5a4fcf,color:#fff
style TURN fill:#ff6b6b,stroke:#cc5555,color:#fff
P2P Connectivity
Section titled “P2P Connectivity”Connection Types
Section titled “Connection Types”Bifrost supports three types of peer connections:
| Type | Description | Latency | Use Case |
|---|---|---|---|
| Direct | UDP hole-punching through NAT | Lowest | Most connections |
| Relayed | Traffic via TURN server | Medium | Symmetric NAT |
| Multi-Hop | Traffic via other peers | Highest | Fallback when TURN unavailable |
Connection Flow
Section titled “Connection Flow”sequenceDiagram
participant A as Peer A
participant D as Discovery Server
participant S as STUN Server
participant T as TURN Server
participant B as Peer B
Note over A,B: 1. Discovery & NAT Detection
A->>D: Register with public key & endpoints
A->>S: STUN Binding Request
S-->>A: Mapped Address (public IP:port)
B->>D: Register with public key & endpoints
B->>S: STUN Binding Request
S-->>B: Mapped Address (public IP:port)
Note over A,B: 2. Peer Discovery
D-->>A: Peer B joined (endpoints, public key)
D-->>B: Peer A joined (endpoints, public key)
Note over A,B: 3. Connection Attempt
A->>B: Try Direct (host candidates)
A->>B: Try Direct (server-reflexive candidates)
alt Direct Connection Succeeds
A<<->>B: Encrypted P2P Connection
else Direct Fails (Symmetric NAT)
A->>T: TURN Allocate
T-->>A: Relay Address
A->>T: Create Permission for B
A->>B: Exchange relay candidates
A<<->>T: Data via TURN
T<<->>B: Data via TURN
end
Note over A,B: 4. Handshake & Encryption
A->>B: Handshake Init (static + ephemeral public key, timestamp, MAC)
Note over B: Authorize key, verify MAC, reject replays
B-->>A: Handshake Response (static + ephemeral public key, timestamp, MAC)
Note over A,B: Derive per-session keys from the handshake transcript
A<<->>B: Encrypted data (ChaCha20-Poly1305)
STUN/TURN/ICE Implementation
Section titled “STUN/TURN/ICE Implementation”STUN (Session Traversal Utilities for NAT)
Section titled “STUN (Session Traversal Utilities for NAT)”STUN is used to discover your public IP address and port as seen from the internet.
Implementation details (internal/p2p/stun.go):
- Supports RFC 5389 STUN Binding Requests
- Parses both
MAPPED-ADDRESSandXOR-MAPPED-ADDRESSattributes - Default servers: Google’s public STUN servers
- Configurable timeout (default: 5 seconds)
How it works:
sequenceDiagram
participant Client
participant STUN as STUN Server
Client->>STUN: Binding Request<br/>(Transaction ID)
Note over STUN: Server sees:<br/>Source IP: 203.0.113.50<br/>Source Port: 54321
STUN-->>Client: Binding Response<br/>(XOR-Mapped-Address: 203.0.113.50:54321)
Note over Client: Now knows public endpoint
Default STUN Servers:
mesh: stun: servers: - "stun:stun.l.google.com:19302" - "stun:stun1.l.google.com:19302" - "stun:stun2.l.google.com:19302" - "stun:stun3.l.google.com:19302" - "stun:stun4.l.google.com:19302" timeout: 5sTURN (Traversal Using Relays around NAT)
Section titled “TURN (Traversal Using Relays around NAT)”TURN provides relay services when direct connections are impossible (e.g., symmetric NAT).
Implementation details (internal/p2p/turn.go):
- Supports RFC 5766 TURN protocol
- Long-term credentials with HMAC-SHA1 authentication
- Channel binding for efficient data transfer
- Automatic allocation refresh
TURN Operations:
| Operation | Description |
|---|---|
Allocate |
Request a relay address from the server |
CreatePermission |
Allow a peer IP to send data through relay |
ChannelBind |
Bind a channel number to a peer for efficiency |
Refresh |
Keep allocation alive (default: 10 minutes) |
Send/Data |
Send and receive relayed data |
Configuration:
mesh: turn: enabled: true servers: - url: "turn:turn.example.com:3478" username: "user" password: "secret"ICE (Interactive Connectivity Establishment)
Section titled “ICE (Interactive Connectivity Establishment)”ICE coordinates STUN and TURN to find the best connection path.
Implementation details (internal/p2p/ice.go):
Candidate Types:
| Type | Priority | Description |
|---|---|---|
host |
126 | Local interface addresses |
srflx |
100 | Server-reflexive (via STUN) |
prflx |
110 | Peer-reflexive (discovered during checks) |
relay |
0 | TURN relay addresses |
ICE Candidate Gathering:
flowchart LR
subgraph Gathering["Candidate Gathering"]
Host[Gather Host<br/>Candidates]
SRFLX[Gather Server-Reflexive<br/>via STUN]
Relay[Gather Relay<br/>via TURN]
end
subgraph Candidates["Local Candidates"]
C1["192.168.1.100:5000<br/>(host)"]
C2["203.0.113.50:54321<br/>(srflx)"]
C3["198.51.100.10:49152<br/>(relay)"]
end
Host --> C1
SRFLX --> C2
Relay --> C3
Connectivity Checks:
ICE pairs local and remote candidates, then tests connectivity in priority order:
// Priority calculation (RFC 8445)priority = (typePref << 24) | (localPref << 8) | (256 - componentID)
// Pair prioritypairPriority = (1 << 32) * min(localPri, remotePri) + 2 * max(localPri, remotePri)NAT Traversal
Section titled “NAT Traversal”NAT Types
Section titled “NAT Types”Implementation details (internal/p2p/nat.go):
| NAT Type | Mapping | Filtering | Direct Connection |
|---|---|---|---|
| None | N/A | N/A | Always works |
| Full Cone | Endpoint-independent | Endpoint-independent | Always works |
| Restricted Cone | Endpoint-independent | Address-dependent | Works with hole-punching |
| Port Restricted | Endpoint-independent | Address+port-dependent | Works with hole-punching |
| Symmetric | Endpoint-dependent | Address+port-dependent | Requires TURN relay |
NAT Detection
Section titled “NAT Detection”flowchart TD
Start[Start Detection] --> STUN1[Send to STUN Server 1]
STUN1 --> Addr1[Get Mapped Address 1]
Addr1 --> STUN2[Send to STUN Server 2]
STUN2 --> Addr2[Get Mapped Address 2]
Addr2 --> Compare{Same Mapping?}
Compare -->|Yes| Friendly[Cone NAT<br/>Direct possible]
Compare -->|No| Symmetric[Symmetric NAT<br/>Relay required]
style Friendly fill:#50c878,stroke:#3a9d5f,color:#fff
style Symmetric fill:#ff6b6b,stroke:#cc5555,color:#fff
Traversal Strategy Selection
Section titled “Traversal Strategy Selection”// Recommended strategy based on NAT typesfunc RecommendedTraversalStrategy(nat1, nat2 NATType) string { if nat1 == NATTypeNone || nat2 == NATTypeNone { return "direct" } if nat1 == NATTypeSymmetric || nat2 == NATTypeSymmetric { if nat1 == NATTypeFullCone || nat2 == NATTypeFullCone { return "direct_to_full_cone" } return "relay" // Both need TURN } return "hole_punch" // Standard NAT traversal}Peer Discovery
Section titled “Peer Discovery”Discovery Server
Section titled “Discovery Server”The discovery server coordinates peer registration and endpoint exchange.
Implementation details (internal/mesh/discovery.go):
Registration Flow:
sequenceDiagram
participant Peer
participant Discovery as Discovery Server
participant WebSocket as WS Events
Peer->>Discovery: POST /api/v1/mesh/networks/{id}/peers<br/>{id, name, public_key, endpoints}
Discovery-->>Peer: {success: true, virtual_ip, peers: [...]}
Peer->>WebSocket: Connect to /events
loop Heartbeat (every 30s)
Peer->>Discovery: POST .../peers/{id}/heartbeat
end
Note over Discovery: New peer joins
Discovery-->>WebSocket: {type: "join", peer: {...}}
Note over Discovery: Peer leaves
Discovery-->>WebSocket: {type: "leave", peer: {...}}
API Endpoints:
| Method | Endpoint | Description |
|---|---|---|
POST |
/api/v1/mesh/networks/{id}/peers |
Register peer |
GET |
/api/v1/mesh/networks/{id}/peers |
List all peers |
PATCH |
/api/v1/mesh/networks/{id}/peers/{peer} |
Update endpoints |
DELETE |
/api/v1/mesh/networks/{id}/peers/{peer} |
Deregister |
POST |
.../peers/{peer}/heartbeat |
Keep alive |
WS |
/api/v1/mesh/networks/{id}/events |
Real-time events |
Server-Side Coordinator Configuration
Section titled “Server-Side Coordinator Configuration”The coordinator lives in the server binary and is configured through the
mesh block of the server config:
mesh: # Mount the /api/v1/mesh coordinator routes. Default: true. # Set to false to remove the endpoints entirely. enabled: true
# Persist networks and peers so they survive a server restart. # When unset the coordinator keeps state in memory only and every network # and peer registration is lost on restart. state_path: "/var/lib/bifrost/mesh-state.json"| Option | Type | Default | Description |
|---|---|---|---|
enabled |
bool | true |
Mount the coordinator REST/WebSocket routes. When false, all /api/v1/mesh/* paths return 404. |
state_path |
string | (empty) | File the coordinator persists networks and peers to. Empty means in-memory only. |
Notes on persistence:
- The state file is written atomically (temp file plus rename) with mode
0600and holds network IDs/CIDRs plus each peer’s ID, name, public key, virtual IP, endpoints and metadata. It contains no private keys. - Peer virtual IPs are re-pinned on startup, so restarting the server does not renumber a running mesh.
- Entries that can no longer be parsed (an invalid CIDR or virtual IP) are skipped with a warning rather than aborting startup.
- A write failure is logged but does not fail the request that triggered it: an in-memory registration that already succeeded is not rolled back.
meshchanges require a server restart.
Peer Information
Section titled “Peer Information”Each peer advertises:
{ "id": "peer-abc123", "name": "laptop-home", "public_key": "base64-encoded-ed25519-pubkey", "virtual_ip": "10.100.0.5", "endpoints": [ {"address": "192.168.1.100", "port": 51820, "type": "local", "priority": 100}, {"address": "203.0.113.50", "port": 54321, "type": "reflexive", "priority": 50}, {"address": "198.51.100.10", "port": 49152, "type": "relay", "priority": 10} ]}Relay Networking
Section titled “Relay Networking”TURN Relay
Section titled “TURN Relay”When direct connections fail, traffic is relayed through a TURN server.
Implementation details (internal/p2p/relay.go):
flowchart LR
subgraph PeerA["Peer A (Symmetric NAT)"]
AppA[Application]
RA[Relay Manager]
end
subgraph TURN["TURN Server"]
Alloc[Allocations]
Perm[Permissions]
Chan[Channels]
end
subgraph PeerB["Peer B (Symmetric NAT)"]
RB[Relay Manager]
AppB[Application]
end
AppA --> RA
RA <-->|Channel Data| TURN
TURN <-->|Channel Data| RB
RB --> AppB
style TURN fill:#ff6b6b,stroke:#cc5555,color:#fff
Channel Data Format:
+------+------+-------------------+| Chan | Len | Data Payload || (2) | (2) | (variable) |+------+------+-------------------+Peer Relay (Multi-Hop)
Section titled “Peer Relay (Multi-Hop)”When TURN is unavailable, traffic can be relayed through other connected peers.
Implementation details (internal/p2p/relay.go):
flowchart LR
A[Peer A] <-->|Direct| B[Peer B]
B <-->|Direct| C[Peer C]
A -.->|Via B| C
style B fill:#7b68ee,stroke:#5a4fcf,color:#fff
Relay Message Format:
+------+----------+-----------+| Type | Dest Len | Dest ID | Payload| (1) | (1) | (var) | (var)+------+----------+-----------+Configuration:
mesh: connection: direct_connect: true # Try direct first relay_enabled: true # Enable TURN relay relay_via_peers: false # Multi-hop peer relaying is NOT implemented; setting true is rejected at startup connect_timeout: 30s keep_alive_interval: 25sNote:
relay_via_peers(multi-hop relaying through other mesh peers) is not yet implemented on the data plane. The server rejects the config at startup if it is set totrue. Leave itfalse.
Encryption
Section titled “Encryption”Key Exchange
Section titled “Key Exchange”Implementation details (internal/p2p/crypto.go):
All P2P connections use:
- Key Generation: X25519 (Curve25519) static key pairs, plus a fresh ephemeral key pair per handshake
- Key Exchange: static-static X25519 for peer authentication, ephemeral-ephemeral X25519 for per-session key material and forward secrecy
- Handshake Authentication: HMAC-SHA256 over every handshake message, keyed from the static-static shared secret
- Encryption: ChaCha20-Poly1305 AEAD
- Key Derivation: HKDF-SHA256, separate send/receive keys, bound to the full handshake transcript
- Replay Protection: strictly increasing handshake timestamps, plus a sliding-window bitmap over data-frame nonces
Handshake Protocol:
Both handshake messages are a fixed 105 bytes:
[0] message type (0x01 init, 0x02 response)[1:33] sender's static X25519 public key[33:65] sender's per-session ephemeral X25519 public key[65:73] handshake timestamp (chosen by the initiator, echoed by the responder)[73:105] HMAC-SHA256 authenticator over bytes [0:73]sequenceDiagram
participant A as Initiator
participant B as Responder
Note over A: Generate ephemeral keypair<br/>Compute static-static secret<br/>Derive MAC key
A->>B: Handshake Init<br/>[0x01 | StaticPub | EphPub | Timestamp | MAC]
Note over B: Verify MAC (proves A holds<br/>its static private key)<br/>Reject replayed timestamp<br/>Generate ephemeral keypair
B-->>A: Handshake Response<br/>[0x02 | StaticPub | EphPub | EchoedTimestamp | MAC]
Note over A: Verify MAC and timestamp echo
Note over A,B: Compute ephemeral-ephemeral secret<br/>Derive send/receive keys from transcript
A->>B: Encrypted Data<br/>[0x03 | Nonce | Ciphertext | Tag]
B-->>A: Encrypted Data<br/>[0x03 | Nonce | Ciphertext | Tag]
Key Derivation:
staticShared = X25519(localStaticPriv, remoteStaticPub) // authenticates the peerephemeralShared = X25519(localEphPriv, remoteEphPub) // fresh every handshake
// Both static public keys, both ephemeral public keys (initiator first) and the// handshake timestamp. Binding the transcript means the two ends derive matching// keys only if they agree on every handshake input.transcript = initStaticPub || respStaticPub || initEphPub || respEphPub || timestamp
sendKey = HKDF-SHA256(ikm: staticShared, salt: ephemeralShared, info: "send" || transcript)recvKey = HKDF-SHA256(ikm: staticShared, salt: ephemeralShared, info: "recv" || transcript)
// Direction determined by public key comparisonif localPubKey > remotePubKey { sendKey, recvKey = recvKey, sendKey}Because ephemeralShared is fresh for every handshake, session keys are unique
per connection even though staticShared is fixed for a peer pair. This is what
makes it safe for the frame nonce counter to restart at 0 on each reconnect: a
(key, nonce) pair is never reused. Discarding the ephemeral private keys after
the handshake also gives each session forward secrecy — compromising a static
private key later does not decrypt recorded traffic.
Peer Authorization
Section titled “Peer Authorization”Inbound sessions are authorized before the handshake is accepted, and fail closed. A peer must clear all of the following:
- Known to discovery. Only public keys this node learned from the discovery
server can open an inbound session. A host that merely knows a peer’s public
key — which is not a secret, since discovery distributes it — is refused and
logged as
rejecting handshake from unknown peer. - On the allowlist, when
security.allowed_peersis set. See below. - Proves possession of the static private key, via the handshake
authenticator. A spoofed identity is refused and logged as
rejecting unauthenticated handshake. - Presents a fresh handshake timestamp. A replayed initiation is refused and
logged as
rejecting replayed handshake initiation. Without this check an attacker could resend a captured handshake, take over the victim’s connection slot, and blackhole it.
Authorization is revoked when a peer leaves the mesh, so a departed peer’s key cannot be reused to open new sessions.
Only frames from a peer that cleared all four checks are decrypted and written to the local TUN/TAP device.
allowed_peers
Section titled “allowed_peers”security.allowed_peers narrows trust from “whatever the discovery server
announces” to an explicit set of base64 public keys. An empty list means “no
explicit allowlist”, not “allow anybody”: peers still have to be announced by
discovery and still have to authenticate. Set it when you do not fully trust the
discovery server — a compromised coordinator can otherwise announce a peer of its
choosing.
security: allowed_peers: - "Uy8x5m2Nc0h1a2b3c4d5e6f7g8h9i0j1k2l3m4n5o6p=" - "9dJk3Lm4Np5Qr6St7Uv8Wx9Yz0Ab1Cd2Ef3Gh4Ij5K="Peers outside the list are ignored at discovery time and logged as
ignoring peer not in allowed_peers list. At startup the node logs which stance
is in effect (mesh peer authorization: ...).
Routing Protocol
Section titled “Routing Protocol”Distance-Vector Routing
Section titled “Distance-Vector Routing”Implementation details (internal/mesh/protocol.go, internal/mesh/router.go):
The mesh uses a distance-vector routing protocol similar to RIP.
Message Types:
| Type | Purpose |
|---|---|
RouteAnnounce |
Share known routes with neighbors |
RouteRequest |
Request routes from neighbors |
RouteWithdraw |
Notify route is no longer available |
Hello |
Periodic keepalive |
HelloAck |
RTT measurement |
LinkState |
Link state updates |
Route Metric Calculation:
Metric = Latency(ms) + (HopCount * 100)Split Horizon:
Routes are not announced back to the peer they were learned from, preventing routing loops:
if config.SplitHorizon && route.NextHop == peerID { continue // Don't announce back}Route Table
Section titled “Route Table”flowchart TB
subgraph RouteTable["Route Table"]
R1["Peer B: Direct, metric=50"]
R2["Peer C: via B, metric=150"]
R3["Peer D: via C, metric=250"]
end
subgraph Selection["Best Route Selection"]
Sort[Sort by Metric]
Best[Select Lowest]
end
RouteTable --> Sort --> Best
Configuration Examples
Section titled “Configuration Examples”Minimal Configuration
Section titled “Minimal Configuration”mesh: enabled: true network_id: "my-network" network_cidr: "10.100.0.0/16" discovery: server: "bifrost.example.com:7080"Full Configuration
Section titled “Full Configuration”mesh: enabled: true network_id: "corporate-vpn" network_cidr: "10.100.0.0/16" peer_name: "laptop-john"
device: type: tap # Layer 2 networking name: "mesh0" mtu: 1400 mac_address: "" # Auto-generated
discovery: server: "bifrost.example.com:7080" heartbeat_interval: 30s peer_timeout: 90s token: "${MESH_TOKEN}"
stun: servers: - "stun:stun.l.google.com:19302" - "stun:stun1.l.google.com:19302" - "stun:stun.cloudflare.com:3478" timeout: 5s
turn: enabled: true servers: - url: "turn:turn.example.com:3478" username: "${TURN_USER}" password: "${TURN_PASS}" - url: "turns:turn.example.com:5349" # TLS username: "${TURN_USER}" password: "${TURN_PASS}"
connection: direct_connect: true relay_enabled: true relay_via_peers: false # Not implemented; must be false (true is rejected at startup) connect_timeout: 30s keep_alive_interval: 25s
security: private_key: "" # Auto-generated if empty require_encryption: true # Always on; false is ignored with a warning allowed_peers: [] # Empty = no explicit allowlist (discovery-announced # peers only, each still authenticated). See # "Peer Authorization" above.Self-Hosted TURN Server
Section titled “Self-Hosted TURN Server”Using coturn:
# /etc/turnserver.conflistening-port=3478tls-listening-port=5349realm=turn.example.comserver-name=turn.example.com
# Authenticationlt-cred-mechuser=meshuser:meshpass
# Certificates for TURNScert=/etc/letsencrypt/live/turn.example.com/fullchain.pempkey=/etc/letsencrypt/live/turn.example.com/privkey.pemTroubleshooting
Section titled “Troubleshooting”Connection Issues
Section titled “Connection Issues”Problem: Peers discovered but not connecting
Section titled “Problem: Peers discovered but not connecting”Diagnosis:
# Check NAT type detectioncurl http://localhost:7082/api/v1/p2p/nat# Response: {"type": "symmetric", "mapped_address": "..."}
# Check discovered endpointscurl http://localhost:7082/api/v1/mesh/networks/my-network/peersSolutions:
- Symmetric NAT detected: Ensure TURN is configured
- No reflexive candidates: Check STUN server connectivity
- Firewall blocking: Allow UDP on ephemeral ports (32768-65535)
Problem: Connection times out
Section titled “Problem: Connection times out”Diagnosis:
# Test STUN connectivitync -u stun.l.google.com 19302
# Check local firewallsudo iptables -L -n | grep -i dropSolutions:
- Increase
connect_timeout - Add more STUN servers
- Check corporate firewall/proxy
Problem: Handshake is rejected
Section titled “Problem: Handshake is rejected”The inbound handshake path fails closed, so a peer that cannot be authorized is dropped rather than connected. The log line names the reason:
| Log message | Meaning | Fix |
|---|---|---|
rejecting handshake from unknown peer |
The peer’s public key was never announced by discovery, or was revoked when the peer left | Confirm both peers use the same network_id and discovery server, and that the peer is currently registered |
ignoring peer not in allowed_peers list |
security.allowed_peers is set and does not contain the peer’s key |
Add the peer’s base64 public key to the list, or clear the list |
rejecting unauthenticated handshake |
The sender does not hold the static private key for the public key it claimed — a spoofing attempt, or mismatched private_key config |
Verify the peer’s security.private_key matches the key it publishes via discovery |
rejecting replayed handshake initiation |
A handshake initiation was seen twice, which is a replay | Expected when traffic is being replayed; otherwise check for a middlebox duplicating UDP |
invalid handshake init: unexpected length (incompatible peer version?) |
The peer speaks an older, incompatible handshake format | Upgrade all mesh peers to the same version |
TURN Relay Issues
Section titled “TURN Relay Issues”Problem: TURN allocation fails
Section titled “Problem: TURN allocation fails”Diagnosis:
# Test TURN serverturnutils_stunclient -p 3478 turn.example.comSolutions:
- Verify credentials are correct
- Check server is reachable on port 3478/5349
- Ensure realm matches configuration
Problem: High latency via relay
Section titled “Problem: High latency via relay”Expected behavior: Relay adds latency due to extra hop.
Mitigation:
- Use geographically close TURN servers
- Enable peer relaying for shorter paths
- Consider multiple TURN servers
Routing Issues
Section titled “Routing Issues”Problem: Packets not reaching destination
Section titled “Problem: Packets not reaching destination”Diagnosis:
# Check route tablecurl http://localhost:7082/api/v1/mesh/routes
# Check peer connectionscurl http://localhost:7082/api/v1/p2p/connectionsSolutions:
- Verify peer is actually connected
- Check MTU settings (reduce if fragmentation)
- Ensure routing protocol is running
Problem: Routing loops
Section titled “Problem: Routing loops”Symptoms: Packets bounce between peers, high CPU usage.
Solutions:
- Enable split horizon (default)
- Check TTL is being decremented
- Verify sequence numbers prevent duplicate processing
Device Issues
Section titled “Device Issues”Problem: TUN/TAP device not created
Section titled “Problem: TUN/TAP device not created”Linux:
# Check if tun module is loadedlsmod | grep tun
# Load if missingsudo modprobe tun
# Check permissionsls -la /dev/net/tun# Should be: crw-rw-rw- 1 root root 10, 200macOS:
# Install tuntaposxbrew install --cask tuntap
# Or use system extension (macOS 10.15+)Windows:
# Install TAP-Windows adapter# Download from OpenVPN or WireGuardProblem: Interface has no IP
Section titled “Problem: Interface has no IP”# Linuxip addr add 10.100.0.5/16 dev mesh0ip link set mesh0 up
# macOSsudo ifconfig mesh0 10.100.0.5 10.100.0.1 upPerformance Tuning
Section titled “Performance Tuning”High CPU Usage
Section titled “High CPU Usage”- Use TUN instead of TAP (less overhead)
- Reduce keepalive frequency
- Limit broadcast TTL
Packet Loss
Section titled “Packet Loss”- Check MTU (try 1280 for maximum compatibility)
- Verify UDP buffer sizes
- Check for network congestion
# Linux: Increase UDP bufferssudo sysctl -w net.core.rmem_max=26214400sudo sysctl -w net.core.wmem_max=26214400Verification Commands
Section titled “Verification Commands”Check Mesh Status
Section titled “Check Mesh Status”# Node statuscurl http://localhost:7082/api/v1/mesh/status
# Peer listcurl http://localhost:7082/api/v1/mesh/networks/my-network/peers
# P2P statisticscurl http://localhost:7082/api/v1/p2p/statsNetwork Diagnostics
Section titled “Network Diagnostics”# Linuxip link show mesh0ip addr show mesh0ip route show dev mesh0
# macOSifconfig mesh0netstat -rn | grep mesh0
# Windowsnetsh interface show interface "mesh0"route printPing Test
Section titled “Ping Test”# Ping another peer's virtual IPping 10.100.0.2
# With verbose outputping -c 5 10.100.0.2Platform Support
Section titled “Platform Support”| Platform | TUN | TAP | Notes |
|---|---|---|---|
| Linux | Full | Full | Native kernel support |
| macOS | Full | Full | Requires tuntaposx or Network Extension |
| Windows | Full | Full | Requires wintun or TAP-Windows driver |
| FreeBSD | Full | Full | Native support |
| OpenWrt | Full | Partial | May need additional packages |
API Reference
Section titled “API Reference”Mesh Node Stats
Section titled “Mesh Node Stats”type NodeStats struct { Status NodeStatus `json:"status"` PeerCount int `json:"peer_count"` ConnectedPeers int `json:"connected_peers"` DirectConnections int `json:"direct_connections"` RelayedConnections int `json:"relayed_connections"` BytesSent int64 `json:"bytes_sent"` BytesReceived int64 `json:"bytes_received"` PacketsSent int64 `json:"packets_sent"` PacketsReceived int64 `json:"packets_received"` Uptime time.Duration `json:"uptime"`}P2P Manager Stats
Section titled “P2P Manager Stats”type Stats struct { ActiveConnections int DirectConnections int RelayedConnections int NATType NATType LocalEndpoints []netip.AddrPort}NAT Info
Section titled “NAT Info”type NATInfo struct { Type NATType `json:"type"` MappedAddress netip.AddrPort `json:"mapped_address"` LocalAddress netip.AddrPort `json:"local_address"` IsBehindNAT bool `json:"is_behind_nat"` Hairpin bool `json:"hairpin"` DetectedAt time.Time `json:"detected_at"`}