System Overview
This document provides a comprehensive overview of the NCN Network v2 architecture, including component responsibilities, interactions, and design decisions.
High-Level Architecture
NCN Network v2 is a decentralized AI inference network with four main component types:
┌──────────────────────────────────────────────────────────────────────────┐
│ NCN Network v2 │
│ │
│ ┌─────────────┐ │
│ │ Clients │ Submit inference requests, pay for computation │
│ └──────┬──────┘ │
│ │ HTTP/gRPC/WebSocket │
│ ▼ │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ Gateway Layer │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │
│ │ │ Gateway 1 │ │ Gateway 2 │ │ Gateway N │ ... │ │
│ │ │ (Subnet 1) │ │ (Subnet 2) │ │ (Subnet N) │ │ │
│ │ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │ │
│ └─────────┼────────────────┼────────────────┼──────────────────────┘ │
│ │ gRPC │ │ │
│ ▼ ▼ ▼ │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ Compute Layer │ │
│ │ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │ │
│ │ │ Compute 1 │ │ Compute 2 │ │ Compute M │ ... │ │
│ │ │ (GPU/CPU) │ │ (GPU/CPU) │ │ (GPU/CPU) │ │ │
│ │ └─────────────┘ └─────────────┘ └─────────────┘ │ │
│ └──────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ Coordination Layer │ │
│ │ ┌───────────────────────────────────────────────────────────┐ │ │
│ │ │ P2P Registry Network │ │ │
│ │ │ • Node Discovery (Kademlia DHT) │ │ │
│ │ │ • Validator Consensus │ │ │
│ │ │ • Subnet Management │ │ │
│ │ │ • Mempool for pending requests │ │ │
│ │ └───────────────────────────────────────────────────────────┘ │ │
│ └──────────────────────────────────────────────────────────────────┘ │
│ │
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ Blockchain Layer │ │
│ │ ┌─────────────────┐ ┌─────────────────┐ │ │
│ │ │ InferencePayment│ │ValidatorRegistry│ │ │
│ │ │ Contract │ │ Contract │ │ │
│ │ └─────────────────┘ └─────────────────┘ │ │
│ │ Arbitrum / Ethereum │ │
│ └──────────────────────────────────────────────────────────────────┘ │
│ │
└──────────────────────────────────────────────────────────────────────────┘Component Deep Dive
Gateway Node
Purpose: Routes inference requests between clients and compute nodes, manages payments.
Key Responsibilities:
Receive client requests (gRPC, HTTP, WebSocket)
Reserve available compute nodes
Request preprocessing validation from P2P Registry
Verify client payments on blockchain
Dispatch tasks to compute nodes
Request completion validation
Trigger payment distribution
Return results to clients
Internal Architecture:
State Management:
Reservation Manager: Tracks compute node reservations
Payment Processor: Handles payment requests and verification
State Manager: Persists request lifecycle state
Finalization Handler: Manages completion and payment distribution
Compute Node
Purpose: Executes AI models in secure sandboxed environments.
Key Responsibilities:
Register with Gateway
Sync models from subnet
Receive inference tasks
Execute models in sandbox
Sign computation results
Return results to Gateway
Internal Architecture:
Sandbox Security Layers:
Syscall filtering
seccomp
Blocks dangerous syscalls
Process isolation
PID namespace
Hides other processes
Network isolation
Network namespace
No external network
Filesystem isolation
Landlock
Read-only access to allowed paths
Resource limits
rlimit
CPU, memory, time bounds
P2P Registry
Purpose: Decentralized coordination service for node discovery, validation, and subnet management.
Key Responsibilities:
Node registration and heartbeat tracking
Model registry
Subnet creation and management
Validator pool management
Preprocessing validation
Completion validation
P2P coordination with other registry nodes
Internal Architecture:
Smart Contracts
Purpose: On-chain payment coordination and validator registry.
Contracts:
InferencePayment
0x4361115359E5C0a25c9b2f8Bb71184F010b768ea
Payment handling
NCNToken
0x38E2565e8905BeAf83C34b266592465C22A2f108
ERC20 payment token
ValidatorRegistry
(TBD)
On-chain validator management
InferencePayment Contract Flow:
Data Flow
Complete Request Lifecycle
Scalability Design
Horizontal Scaling
Gateway
Multiple gateways, each manages subnets
Compute
Add more compute nodes to subnet
Registry
P2P network, distributed state
Performance Targets
Inference latency
< 5 seconds
Throughput
100+ requests/second
Uptime
99.9%
Payment success
100%
Bottleneck Mitigation
Single gateway
Multiple gateways per subnet
Validator consensus
Parallel signature collection
Blockchain confirmation
Optimistic execution
Model loading
Model caching with TTL
Fault Tolerance
Node Failures
Gateway crash
Client reconnects to another gateway
Compute crash
Gateway releases reservation, retries
Registry crash
Other registry nodes continue
Payment Safety
Compute failure
Expiry-based refund
Gateway failure
On-chain expiry refund
Validator unavailable
M-of-N allows failures
Next Steps
Payment Flow - Detailed payment mechanics
Validation & Consensus - Validator system
Security Model - Security architecture
Last updated