SeedInfer — Private AI on Verified Hardware

Faza 0 · NVFP4seedinfer/nemotron-lightning-1m1M ctx · 2M KVCUDA 13.3 · driver 580+

Become a Provider — earn per token

Serve seedinfer/nemotron-lightning-1m (NVIDIA Nemotron 3.5 Lightning 30B A3B NVFP4 — 30B/3B MoE+Mamba, 1M context) via your RTX 5090 32GB (GB202, Blackwell) — CUDA 13.3. Earn $0.02 / 1M input + $0.05 / 1M output — prefix caching 60s free / 5min max. Telemetry JSONL, fallback NIM/OpenRouter OK.

Input

$0.02 / 1M

prompt tokens

Output

$0.05 / 1M

completion tokens

Context

1M · 2M KV

NVFP4 W4A16+FP8 KV ~22-28GB (1M KV ~6GB)

One-liner install provider.tar.gz https://seedinfer.com/install.sh

Requirements

OSUbuntu 24.04+ (noble)

Driver580+ (CUDA 13.3+)

GPURTX 5090 32GB min

VRAM min32GB (16 hard min)

Docker24+ + nvidia-ctk

HF cache50GB+ (provider ~60GB with model)

Free space60GB+ (df -h)

NetworkUDP 41641

Ports47900:8000 + 47901:3001

tailscale + nvidia-container-toolkit are installed automatically by install.sh if missing. Host ports 47900:8000 + 47901:3001 can be overridden via env VLLM_PORT/AGENT_PORT.

Minimum 60GB free (vLLM ~28.8GB + NVFP4 ~30GB + cache). Check df -h · cleanup: docker system prune -a · free ports 47900/47901 (overridable via env VLLM_PORT/AGENT_PORT) — details in /docs.

One-liner — terminal Linuxcurl | bash

One command: curl -fsSL https://seedinfer.com/install.sh | bash — auto-fetches authkey from /api/v1/auth/request if --authkey is missing, nvidia-smi check (47900/47901 free, VLLM_PORT/AGENT_PORT) → Docker + nvidia-ctk + tailscale → tailscale up → prebuild docker pull ghcr.io/seedinfer/provider:cuda13.3-nvfp4 || curl https://seedinfer.com/provider-image.tar.gz | docker load (Pi) || docker compose up → heartbeat → verified.

Authkeytag:provider · 24h · reusable

Click Generate invite to create a key — valid for 24h, one-time use (tag:provider). Copy it into the one-liner below. Details in /docs.

Recommended — one command (auto-authkey + prebuild)

curl -fsSL https://seedinfer.com/install.sh | bash

No parameters — install.sh automatically fetches an authkey from https://seedinfer.com/api/v1/auth/request (tag:provider, 24h) and tries docker pull ghcr.io/seedinfer/provider:cuda13.3-nvfp4 → fallback curl https://seedinfer.com/provider-image.tar.gz | docker load (Pi) → fallback docker compose build (~28GB). Orange Pi 4 Pro does not build CUDA — it only hosts the tar/registry.

Gateway https://seedinfer.comTailnet https://tailnet.seedinfer.comPrebuild ghcr.io → Pi tar → build

Advanced — custom authkey / model / gatewayexpand

1 · With key from Generate invite

curl -fsSL https://seedinfer.com/install.sh | bash -s -- --authkey YOUR_AUTHKEY

Use the key from Generate invite above — backward compatible. Same effect as recommended, but with an explicit --authkey.

2 · Auto-fetch key (jq) — scripts

curl -fsSL https://seedinfer.com/install.sh | bash -s -- --authkey $(curl -s https://seedinfer.com/api/v1/auth/request | jq -r .authkey)

Manual auto-fetch (now default in install.sh — you don't need to pass jq).

3 · Full options + env prebuild

curl -fsSL https://seedinfer.com/install.sh | bash -s -- --authkey YOUR_AUTHKEY --model seedinfer/nemotron-lightning-1m --gateway https://seedinfer.com --hostname provider-5090

Custom model/gateway/hostname. ENV override: SEEDINFER_PREBUILD_IMAGE=ghcr.io/seedinfer/provider:cuda13.3-nvfp4 SEEDINFER_PREBUILD_URL=https://seedinfer.com/provider-image.tar.gz SEEDINFER_SKIP_PREBUILD=1 wymusza build.

https://seedinfer.com/install.sh·/api/install alias·/api/v1/auth/request· dashboard.seedinfer.com

Steps — from curl to fleet~5 min + 30GB download

1
curl install.sh
curl -fsSL https://seedinfer.com/install.sh | bash (auto-authkey)
Auto-fetches authkey from /api/v1/auth/request if --authkey is missing, checks nvidia-smi (CUDA 13.3+, driver 580+, VRAM 32GB min), HF model, installs Docker + nvidia-container-toolkit + tailscale if missing.
```
nvidia-smi VRAM OK (>=32GB) HF OK driver 580+ authkey auto-fetched docker pull ghcr.io → Pi tar → build
```
2
tailscale (domyślnie kontener)
Domyślnie: kontener (bezpieczne) — jeśli host już w tailscale.com (100.94.x.x tail*.ts.net) → install.sh auto-wykrywa (tailscale status --json CurrentTailnet.BaseDomain/Self.ControlURL/MagicDNSSuffix + BackendState Running) i uruchamia tailscale-seedinfer jako docker run -d --name tailscale-seedinfer --restart unless-stopped --cap-add=NET_ADMIN --cap-add=NET_RAW --device /dev/net/tun -v tailscale-seedinfer-state:/tailscale -e TS_AUTHKEY -e TS_HOSTNAME -e TS_LOGIN_SERVER -e TS_EXTRA_ARGS="--advertise-tags=tag:provider --accept-routes" tailscale/tailscale:latest (healthcheck, volume, network seedinfer-tailnet). Współistnienie: host 100.94.x.x (dom, tailscale.com) + kontener 100.64.x.x (Headscale) — nie rozłącza domowego tailnetu. Provider agent używa kontenera (DNS 100.64.x.x, gateway.seedinfer.ts.net).
Alternatywa compose: docker compose --profile tailscale up -d (sidecar tailscale w provider/docker-compose.yml, volume tailscale-seedinfer-state:/tailscale, healthcheck). Opt-in host: --force-host-tailscale (przełącza hosta --reset, rozłączy tailscale.com) lub TAILSCALE_USE_CONTAINER=0 (advanced). Check: docker exec tailscale-seedinfer tailscale status (kontener 100.64.x.x) + tailscale status (host 100.94.x.x — nienaruszony).
```
docker exec tailscale-seedinfer tailscale status  # kontener 100.64.x.x tailscale status  # host 100.94.x.x (nienaruszony) docker exec tailscale-seedinfer tailscale ip -4  # 100.64.x.x ping -c2 gateway.seedinfer.ts.net
```
3
docker pull / load → up
Klonuje provider/ do /opt/seedinfer-provider, tworzy .env (VLLM_MODEL=nvidia/...NVFP4), prebuild docker pull ghcr.io/seedinfer/provider:cuda13.3-nvfp4 || curl https://seedinfer.com/provider-image.tar.gz | docker load (Pi) || docker compose up -d --build → vLLM auto-download ~30GB do ./models/cache.
```
docker pull ghcr.io/...:cuda13.3-nvfp4 || curl https://seedinfer.com/provider-image.tar.gz | docker load || docker compose up --build
```
4
health → verified → fleet
curl http://127.0.0.1:47901/health + /v1/models → heartbeat co 30s do /api/v1/providers/heartbeat → pending → verifying → verified (fleet zielony, opacity 100).
```
curl https://seedinfer.com/api/v1/providers | jq
```

Health check

curl -fsS http://127.0.0.1:47901/health | jq
# {"status":"ok","provider_id":"...","vllm_health":{"status":"ok"},"gpu":{"count":1}}

curl -fsS http://127.0.0.1:47900/v1/models | jq
curl http://127.0.0.1:47901/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"seedinfer/nemotron-lightning-1m","messages":[{"role":"user","content":"Hello"}],"max_tokens":32}'

Heartbeat + verify

# heartbeat co 30s (agent/main.py)
POST https://seedinfer.com/api/v1/providers/heartbeat
# payload Provider + gpu + host + vllm_health

# auto-verify po 2 heartbeat (~60s)
GET https://seedinfer.com/api/v1/providers
# verification: pending -> verifying -> verified

# manual verify
curl -X POST https://seedinfer.com/api/v1/providers/verify \
  -H "Content-Type: application/json" \
  -d '{"provider_id":"provider-5090-xxx"}' | jq

Fleet UI

/providers — pending/verifying opacity 60, failed red, verified green (opacity 100) = official node.
GET /api/v1/providers?verified=1 — verified only.
Pi gateway decides verified (not Headscale ACL).

Badge: pending 🟡 verif. 60% → verified 🟢 100% → failed 🔴

Models — NVFP4 plug-and-play (Phase 0)

ID (MODEL)	VLLM_MODEL (HF repo)	VRAM	Pricing	On-disk	Ctx	Status
seedinfer/nemotron-lightning-1m	nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4	16-22GB	$0.02 / $0.05	~20-30GB	1M (2M KV)	active
gpt-oss-20b	alias → seedinfer/nemotron-lightning-1m	—	same	—	1M	alias
qwen3.6-35b-a3b	Qwen/Qwen3.6-35B-A3B	—	$0.06 / $0.50	—	131k	is coming
gemma-4-26b-a4b	modal —	—	on Modal	—	—	is coming

NVFP4 — vLLM nightly

W4A16 + FP8 KV via ModelOpt → ~16-22GB VRAM (vs 66GB BF16). On-disk ~20-30GB. pip install --pre vllm --extra-index-url https://wheels.vllm.ai/nightly + FROM nvidia/cuda:13.3.0-cudnn-devel-ubuntu24.04 (fallback 13.3.1/13.2.1, legacy 12.4.1 via PTX JIT). Auto-detect — do not pass --quantization (log: auto). If it fails: VLLM_QUANTIZATION=modelopt_fp4 + --kv-cache-dtype fp8.

--model nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 \
--served-model-name seedinfer/nemotron-lightning-1m \
--max-model-len 1048576 --enable-prefix-caching --enable-chunked-prefill \
--dtype bfloat16 --gpu-memory-utilization 0.90 --trust-remote-code
# jinja chat_template from HF auto (tokenizer_config.json)

Endpoints + pricing

/api/v1/models /api/v1/pricing /api/v1/fallback/status /api/v1/telemetry

Fallback: local (Headscale 100.64.x.x:47901) → NIM → OpenRouter → Modal (Modal warmup parallel po local fail). X-SeedInfer-Upstream header.

Telemetry: JSONL /mnt/nvme/telemetry/telemetry.jsonl → GET /api/v1/telemetry · fallback chain log.

On OOM: VLLM_MAX_MODEL_LEN=32768 + VLLM_GPU_MEMORY_UTILIZATION=0.80

GPU Matrix — minimum RTX 5090 32GB (CUDA 13.3)Blackwell native

Minimum RTX 5090 32GB (GB202, Blackwell sm_120, 21760 CUDA, 680 Tensor 5th gen, 32GB GDDR7 ~1.8 TB/s). NVFP4 30B ~16-22GB + ~6GB KV for 1M ctx = ~22-28GB → 32GB gives headroom with --gpu-memory-utilization 0.90 + --max-model-len 1048576. vLLM nightly cu12 wheels run on CUDA 13.3 via PTX JIT (forward-compat, driver 580+).

GPU	Arch	VRAM	BW	NVFP4 1M	Est. tput*	Status
RTX 5090 32GB	GB202 sm_120	32GB GDDR7	~1.8 TB/s	✅ ~22-28GB	~120-180 tok/s	minimum
A100 40GB	GA100 sm_80	40GB HBM2e	1.6 TB/s	✅ W4A16	~60-90	welcome
A100 80GB	GA100	80GB HBM2e	2.0 TB/s	✅	~70-100	welcome
H100 80GB	H100 sm_90	80GB HBM3	3.0 TB/s	✅	~150-220	welcome
L40S 48GB	AD102 sm_89	48GB GDDR6	864 GB/s	✅	~80-120	welcome
RTX 6000 Ada 48GB	AD102	48GB GDDR6	960 GB/s	✅	~80-120	welcome
RTX 6000 Pro Blackwell	GB202	96GB GDDR7	~1.8 TB/s+	✅ 96GB	~130-190	welcome
RTX 4500 Blackwell 32GB	GB203	32GB GDDR7	~1.0 TB/s	✅	~90-130	welcome
RTX 5000 Blackwell	GB203	32-48GB GDDR7	~1.2 TB/s	✅	~110-160	welcome
RTX 3090 24GB ⏳	GA102 sm_86	24GB GDDR6X	936 GB/s	⚠️ 24GB tight	~50-80	plan
RTX 4090 24GB ⏳	AD102 sm_89	24GB GDDR6X	1.0 TB/s	⚠️ tight	~70-100	plan

* Est. tput single-user prefill+decode for Nemotron 30B NVFP4 (W4A16+FP8 KV), batch 1, 1k in / 256 out, no prefix cache. Real throughput depends on humming/mamba backend + KV hit.
Eventually 3090/4090 (24GB) — planned as "community" tier with auto-downscale VLLM_MAX_MODEL_LEN=131072 and VLLM_GPU_MEMORY_UTILIZATION=0.85. Currently welcome for testing, but the official minimum is 32GB.
CUDA 13.3 + driver 580+ required for Blackwell GB202 native (sm_120). Modal/legacy CUDA 12.4 via PTX JIT works without rebuild.

Kontener do pobrania

provider.tar.gz

Full provider/ pack — Dockerfile.cuda + compose + agent

https://seedinfer.com/provider.tar.gz →

install.sh

One-liner plug-and-play ~279 linii

https://seedinfer.com/install.sh

Control plane

Headscale tailnet.seedinfer.com (WireGuard) · Gateway https://seedinfer.com

docs.seedinfer.com /docs →

Instrukcje dla dostawcy

# Provider pack — pobierz i zweryfikuj:
curl -fsSL https://seedinfer.com/provider.tar.gz -o provider.tar.gz && tar tzf provider.tar.gz | head
curl -fsSL https://seedinfer.com/install.sh | head -n 20

# Docs:
# https://seedinfer.com/docs  lub  https://docs.seedinfer.com

# Authkey:
curl -fsSL https://seedinfer.com/api/v1/auth/request | jq
# -> {authkey, expires: "24h", login_server: "https://tailnet.seedinfer.com"}

Details in /docs — requirements, ports, troubleshooting and env.

Weryfikacja po instalacji

# Lokalnie:
curl -fsS http://127.0.0.1:47901/health | jq
curl -fsS http://127.0.0.1:47900/v1/models | jq
docker ps | grep seedinfer-provider

# Gateway:
curl -fsS https://seedinfer.com/api/v1/providers | jq '.data[] | {id,status,verification}'
curl -fsS https://seedinfer.com/api/stats | jq '.active_providers'
# Heartbeat log Pi:
tail -f /mnt/nvme/telemetry/telemetry.jsonl | jq

Manual Docker build

git clone https://github.com/seedinfer/seedinfer.com.git
cd seedinfer.com
cp provider/.env.example provider/.env
# edytuj TAILSCALE_AUTHKEY
docker compose -f provider/docker-compose.yml up -d --build
docker build -f provider/Dockerfile.cuda -t seedinfer/provider:cuda-0.1.0 .

SeedInfer.com · Provider pack Faza 0 CUDA — /install.sh · /provider.tar.gz · /api/v1/auth/request · /api/v1/providers · docs.seedinfer.com /docs