Network

A Kademlia DHT + gossip discovery overlay on top of the libp2p NAT-traversal stack. On this real snapshot the swarm runs in-process over loopback QUIC, so there is no geography, NAT or WAN RTT to measure — exercising hole-punching, relays and AutoNAT needs a multi-host WAN deployment.

The peer-to-peer overlay that lets machines discover and reach each other directly — even from behind home or office routers — with no central server. This snapshot runs in-process over loopback, so geography and NAT are not applicable here.

Impact: The grid keeps working as nodes join and leave, and there is no single point that can fail, censor, or be taken down.

Nodes in swarm

loopback peers

Online

9 known

L2 nodes

TEE attested

Transport

QUIC

loopback

Discovery

static

overlay mode

Node communications (circular)snapshot

Message graph from the snapshot — requesters dispatch to workers (blue); workers that agreed on a job's quorum hash are linked (green). Start the live backend for realtime updates.

nodes

dispatch

quorum

loading plot…

requesterhonest workerfailing nodecheating node

Closure posture

public

The single [security].mode switch between the open public grid and a fully closed company / enterprise grid.

mode: public
group enforcement: soft
networks: default

allowlist mTLS — outsiders refused at the transport layer

cryptographic group tokens (never soft declared labels)

fail-closed discovery — drop unknown-labeled peers

default-deny requester roster — serve only roster members

PUBLIC (this run): zero-config grid. Setting mode = private requires an allowlist roster, token group enforcement, and an explicit non-default network — else the node refuses to start. The requester↔TLS-peer identity binding (an offer's requester_id must equal the authenticated mTLS peer) is always on in both modes.

Smart routing & failover

A pre-flight, metadata-only size estimate picks local-vs-remote automatically, and over-capacity jobs reroute to bigger hosts instead of failing.

prefer: auto
local size threshold: 256 MB
local RAM fraction: 60%
local execution: enabled
re-dispatch retries: unlimited

Size-based routing: a job estimated to fit runs FREE in the node's own locked-down DuckDB; one that exceeds the local threshold/headroom is dispatched to the grid.

Robust failover: a “too big” / OOM job reroutes to higher-capacity hosts (excluding only failed nodes); only when none remain does it return ExceedsCapacity.

Swarm

The real worker nodes from this run. Each advertises a signed capability record; all are wired over a single loopback transport (no geography on one host).

Node	Attestation	Capability record	Trust	Transport
frost-owl	L2 · TEE	32 thr · 64.0 GB · max 12	0.92	loopback
harbor-vole	L2 · TEE	40 thr · 96.0 GB · max 10	0.92	loopback
tidal-fox	L2 · TEE	16 thr · 32.0 GB · max 6	0.91	loopback
marsh-otter	L2 · TEE	24 thr · 48.0 GB · max 8	0.92	loopback
amber-mole	L1 · TPM	12 thr · 24.0 GB · max 4	0.90	loopback
pine-marten	L1 · TPM	8 thr · 16.0 GB · max 4	0.90	loopback
slate-heron	L0 · anon	4 thr · 8.0 GB · max 2	0.81	loopback
rust-shrike	L0 · anon	4 thr · 8.0 GB · max 2	0.00	loopback
cobalt-stoat	L0 · anon	2 thr · 4.0 GB · max 1	0.07	loopback

Discovery & NAT traversal

The libp2p stack as configured in the live [discovery] / [discovery.nat]sections of this run's GridConfig. libp2p climbs from cheapest to most expensive path; relays are a last resort, never the default.

Technique	What it does	State
identify	exchange peer keys, listen addrs & supported protocols	always on
AutoNAT	peers probe you to learn your external addr & reachability	enabled
DCUtR hole punching	coordinated simultaneous-open over QUIC/UDP → direct path	enabled
Circuit Relay v2 (client)	reserve slots on volunteer relays when behind symmetric NAT · max_relays 3	enabled
AutoRelay (act as relay)	volunteer this node as a relay for others	off
mDNS	zero-config peer discovery on the local LAN · every 300s	enabled
DHT routing (Kademlia)	O(log n) FIND_NODE · repl 20 · par 3	enabled

Bootstrap caveat

A swarm needs ≥1 reachable entry point to join — but a bootstrap peer is an ordinary node. It holds no job state, never sits in the data path, and is freely replaceable: swap the address and the swarm is unchanged. (This run's discovery.bootstrap is empty — 0 entries — since every node is already in-process.)

Single-host caveat

Hole-punching (DCUtR), Circuit Relay and AutoNAT are configured and compiled in, but can only actually be exercised across real, separate networks. On this loopback run there are no NATs to traverse, so these paths are never triggered.

Capability record (gossiped)

signed, gossiped, locally filtered

{

node_id: b3:9cf6d78b60eb40600c0780a69bd99a0062d745e8ca6075736b2949cf3f7b078a
free_mem: 64.0 GB
free_cores: 32
max_jobs: 12
attestation_level: L2
price: 0 /unit
recent_receipts_root: b3:9cf6d78b60eb40600c0780a69bd99a0062d745e8ca6075736b2949cf3f7b078a

}

The real shape frost-owl publishes on topic duckdb-p2p/caps/1 — refreshed every 5s, TTL 30s, signed with each node's Ed25519 identity.

Discovery

no registry, no coordinator

How peers find work

Lookup is structured; matchmaking is local. No node ever sees the whole graph.

1Kademlia DHT. O(log n) peer lookup keyed by node id — replication factor 20, record TTL 3600s. Scales without a central index.
2Gossip / pubsub. Workers publish a signed capability record to topic duckdb-p2p/caps/1 (fanout 6) and refresh it as load changes.
3Local filtering. Requesters subscribe and filter the stream client-side by capacity, trust and attestation (sampling up to 16 candidates) before sending an offer.
4Bootstrap seeds. Used once, only to enter the swarm — then dropped from the hot path entirely.

Overlay config

real [discovery] values

mode: static
candidate sample: 16
kad replication: 20
kad query par: 3
gossip fanout: 6
cap TTL: 30s
max relays: 3

8 live capability records on the topic.