P2P Registry

The P2P Registry is the decentralized coordination service for NCN Network v2. It handles node discovery, validator consensus, and subnet management using a libp2p-based peer-to-peer network.

---

Overview

┌─────────────────────────────────────────────────────────────────────────────┐
│                            P2P Registry Node                                 │
│                                                                             │
│   External Interfaces                                                       │
│   ┌─────────────────────┐   ┌─────────────────────┐                        │
│   │    gRPC Server      │   │     P2P Node        │                        │
│   │    (Port 50050)     │   │    (Port 8828)      │                        │
│   │                     │   │                     │                        │
│   │  • Node registration│   │  • Kademlia DHT     │                        │
│   │  • Validation RPCs  │   │  • Peer discovery   │                        │
│   │  • Subnet mgmt      │   │  • Data replication │                        │
│   └─────────────────────┘   └─────────────────────┘                        │
│                                                                             │
│   Core Services                                                             │
│   ┌─────────────────────────────────────────────────────────────────────┐   │
│   │  ┌───────────────┐  ┌───────────────┐  ┌───────────────┐           │   │
│   │  │ Preprocessing │  │  Completion   │  │    Subnet     │           │   │
│   │  │   Service     │  │   Service     │  │   Manager     │           │   │
│   │  └───────────────┘  └───────────────┘  └───────────────┘           │   │
│   │  ┌───────────────┐  ┌───────────────┐  ┌───────────────┐           │   │
│   │  │   Validator   │  │    Mempool    │  │  DHT Storage  │           │   │
│   │  │     Pool      │  │               │  │               │           │   │
│   │  └───────────────┘  └───────────────┘  └───────────────┘           │   │
│   └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
│   Blockchain Sync                                                           │
│   ┌─────────────────────────────────────────────────────────────────────┐   │
│   │  Validator Registry Contract Sync (periodic)                         │   │
│   └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

---

Features

• Node Discovery: Kademlia DHT-based peer discovery

• Validator Consensus: M-of-N signature collection

• Subnet Management: Two-phase subnet creation with fee estimation

• State Persistence: DHT-based distributed storage

• Blockchain Sync: Automatic validator registry synchronization

• Mempool: Temporary storage for pending validations

---

Quick Start

Build

cargo build --release -p p2p_registry_node

Run

cargo run --release --bin p2p_registry_node -- \
  --p2p-listen-addr /ip4/0.0.0.0/tcp/8828 \
  --grpc-listen-addr 127.0.0.1:50050

Expected Output:

Starting P2P Registry Node...
P2P listening on /ip4/0.0.0.0/tcp/8828
Local peer ID: 12D3KooWxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
gRPC server listening on 127.0.0.1:50050
✓ Validator pool initialized (0 validators)
✓ Mempool initialized
✓ Subnet manager initialized
✓ P2P Registry ready

---

Architecture

Internal Modules

Module	File	Purpose
Main Service	src/main.rs	Service orchestration
P2P Registry	src/p2p_registry.rs	libp2p networking
Validator Pool	src/validator_pool.rs	Validator management
Mempool	src/mempool.rs	Pending request storage
Preprocessing	src/preprocessing.rs	Request validation
Completion	src/completion.rs	Result validation
Subnet Manager	src/subnet_manager.rs	Subnet CRUD
DHT Persistence	src/dht_persistence.rs	Distributed storage
Validator Signer	src/validator_signer.rs	Signature generation
Validator Sync	src/validator_sync_service.rs	Blockchain sync

Service Dependencies

┌─────────────────────────────────────────────────────────────────┐
│                        P2P Registry                              │
│                                                                 │
│   ┌───────────────┐                                             │
│   │  gRPC Server  │                                             │
│   └───────┬───────┘                                             │
│           │                                                     │
│           ▼                                                     │
│   ┌───────────────────────────────────────────────────────┐     │
│   │                  Request Handler                       │     │
│   │  ┌─────────────────────────────────────────────────┐  │     │
│   │  │ RegisterNode │ DiscoverNodes │ Validation RPCs │  │     │
│   │  └─────────────────────────────────────────────────┘  │     │
│   └───────────────────────────┬───────────────────────────┘     │
│                               │                                 │
│           ┌───────────────────┼───────────────────┐             │
│           ▼                   ▼                   ▼             │
│   ┌───────────────┐   ┌───────────────┐   ┌───────────────┐     │
│   │ Validator Pool│   │   Mempool     │   │Subnet Manager │     │
│   │               │   │               │   │               │     │
│   │ • Selection   │   │ • Pending reqs│   │ • Create      │     │
│   │ • Reputation  │   │ • TTL cleanup │   │ • Get/List    │     │
│   │ • Slashing    │   │ • Statistics  │   │ • Validation  │     │
│   └───────────────┘   └───────────────┘   └───────────────┘     │
│           │                   │                   │             │
│           └───────────────────┼───────────────────┘             │
│                               ▼                                 │
│                       ┌───────────────┐                         │
│                       │  DHT Storage  │                         │
│                       │  (Kademlia)   │                         │
│                       └───────────────┘                         │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

---

gRPC API

P2PRegistryService

Node Management

RPC	Request	Response	Description
RegisterNode	NodeInfo	Ack	Register a node
UpdateNodeHeartbeat	UpdateNodeHeartbeatRequest	Ack	Update heartbeat
DeregisterNode	WalletAddress	Ack	Remove node
DiscoverNodes	DiscoverNodesRequest	stream NodeInfo	Find nodes

Validation

RPC	Request	Response	Description
RequestPreprocessingValidation	InferenceRequest	PreprocessingValidation	Get preprocessing validation
RequestCompletionValidation	InferenceResponse	CompletionValidation	Get completion validation

Subnet Management

RPC	Request	Response	Description
EstimateSubnetFee	EstimateSubnetFeeRequest	EstimateSubnetFeeResponse	Estimate creation fee
CreateSubnet	CreateSubnetRequest	CreateSubnetResponse	Create subnet
GetSubnet	GetSubnetRequest	GetSubnetResponse	Get subnet info
ListSubnets	Empty	stream SubnetMetadata	List all subnets

Validator Management

RPC	Request	Response	Description
RegisterValidator	RegisterValidatorRequest	RegisterValidatorResponse	Register validator
UnregisterValidator	UnregisterValidatorRequest	Ack	Unregister
GetValidatorInfo	ValidatorAddressRequest	ValidatorInfoResponse	Get validator info
ListValidators	ListValidatorsRequest	stream ValidatorInfoMessage	List validators
UpdateValidatorReputation	UpdateReputationRequest	Ack	Update reputation

See gRPC API Reference for detailed documentation.

---

P2P Networking

Kademlia DHT

The registry uses libp2p's Kademlia DHT for:

• Peer discovery

• Distributed storage

• Network resilience

Peer Discovery

Node A                    DHT                    Node B
  │                        │                        │
  │  1. Bootstrap          │                        │
  │───────────────────────▶│                        │
  │                        │                        │
  │  2. FIND_NODE          │                        │
  │───────────────────────▶│                        │
  │                        │                        │
  │  3. K closest peers    │                        │
  │◀───────────────────────│                        │
  │                        │                        │
  │  4. Connect to peers   │                        │
  │─────────────────────────────────────────────────▶│
  │                        │                        │

Bootstrap Nodes

# Connect to bootstrap nodes
--bootstrap /ip4/192.168.1.100/tcp/8828/p2p/12D3KooW...
--bootstrap /ip4/192.168.1.101/tcp/8828/p2p/12D3KooX...

---

Validator Pool

Features

• Registration: Add/remove validators

• Selection: Multiple strategies (random, reputation, stake)

• Reputation: Track validation success/failure

• Slashing: Penalize misbehavior

• Rotation: Periodic validator rotation

Configuration

# Validator pool settings
MIN_VALIDATORS_FOR_CONSENSUS=3
MAX_VALIDATORS_TO_SELECT=5
MIN_REPUTATION_THRESHOLD=20
VALIDATOR_ROTATION_INTERVAL_SECS=3600

See Validator Pool for details.

---

Mempool

Structure

pub struct Mempool {
    pending_requests: HashMap<String, MempoolEntry<InferenceRequest>>,
    preprocessing_validations: HashMap<String, MempoolEntry<PreprocessingValidation>>,
    completion_validations: HashMap<String, MempoolEntry<CompletionValidation>>,
}

TTL Configuration

Entry Type	Default TTL
Pending requests	10 minutes
Preprocessing validations	15 minutes
Completion validations	5 minutes

Cleanup

Automatic cleanup runs every 60 seconds:

// Cleanup task
loop {
    tokio::time::sleep(Duration::from_secs(60)).await;
    let stats = mempool.lock().await.cleanup_expired();
    log::info!("Mempool cleanup: {:?}", stats);
}

---

Subnet Manager

Subnet Creation Flow

1. EstimateSubnetFee
   ├── Calculate fee based on models
   ├── Generate fee_quote_id (valid 10 min)
   └── Return fee + contract address

2. Client pays on-chain

3. CreateSubnet
   ├── Verify payment_tx on blockchain
   ├── Verify config_hash matches quote
   ├── Collect validator signatures
   ├── Store subnet metadata
   └── Return subnet_id

SubnetMetadata

message SubnetMetadata {
  uint64 subnet_id = 1;
  string gateway_address = 2;
  int64 created_at = 3;
  repeated ModelDescriptor models = 4;
  repeated ValidatorSignature validated_by = 5;
  string network_fee_wei = 6;
  string payment_tx = 7;
  string status = 9;  // pending, active, deprecated
}

---

Configuration

Command Line Arguments

Argument	Required	Default	Description
--p2p-listen-addr	No	/ip4/0.0.0.0/tcp/8828	P2P listen address
--grpc-listen-addr	No	127.0.0.1:50050	gRPC listen address
--bootstrap	No	-	Bootstrap node addresses
--data-dir	No	./data	Data persistence directory

Environment Variables

Variable	Required	Default	Description
VALIDATOR_PRIVATE_KEY	Yes*	-	Validator signing key
VALIDATOR_SYNC_ENABLED	No	true	Enable blockchain sync
VALIDATOR_SYNC_INTERVAL	No	60	Sync interval (seconds)
VALIDATOR_REGISTRY_CONTRACT	No	-	On-chain validator registry
FORKNET_RPC_URL	No	-	Blockchain RPC URL
CHAIN_ID	No	828	Blockchain chain ID

*Auto-generated if not provided (development only)

See Configuration Reference for complete details.

---

Background Tasks

Automatic Tasks

Task	Interval	Purpose
Mempool cleanup	60s	Remove expired entries
Validator rotation	1 hour	Rotate validators
Statistics logging	5 min	Log pool/mempool stats
Blockchain sync	60s	Sync validator registry

Task Implementation

// Mempool cleanup task
tokio::spawn(async move {
    loop {
        tokio::time::sleep(Duration::from_secs(60)).await;
        let stats = mempool.lock().await.cleanup_expired();
        log::info!("Mempool cleanup: removed {} expired entries", stats.total_removed);
    }
});

// Validator rotation task
tokio::spawn(async move {
    loop {
        tokio::time::sleep(Duration::from_secs(3600)).await;
        let rotated = validator_pool.lock().await.rotate_validators();
        log::info!("Validator rotation: {} validators affected", rotated);
    }
});

---

Monitoring

Metrics

Metric	Type	Description
registry_nodes_registered	Gauge	Registered nodes
registry_validators_active	Gauge	Active validators
registry_mempool_size	Gauge	Mempool entries
registry_subnets_total	Counter	Total subnets
registry_validations_total	Counter	Total validations
registry_p2p_peers	Gauge	Connected peers

Health Endpoint

curl http://127.0.0.1:50050/health

# Response
{
  "status": "healthy",
  "p2p": {
    "peer_id": "12D3KooW...",
    "connected_peers": 5
  },
  "validators": {
    "active": 10,
    "total": 15
  },
  "mempool": {
    "pending_requests": 3,
    "preprocessing": 2,
    "completion": 1
  }
}

---

Troubleshooting

Common Issues

"No validators available"

Error: Insufficient validators for consensus

• Check validators are registered and active

• Verify VALIDATOR_SYNC_ENABLED=true

• Check blockchain RPC connectivity

"P2P bootstrap failed"

Error: Failed to connect to bootstrap nodes

• Verify bootstrap node addresses

• Check network connectivity

• Ensure ports are open (8828)

"Mempool full"

Warning: Mempool size exceeds threshold

• Increase cleanup frequency

• Check for stuck requests

• Verify downstream services (gateways)

See Registry Troubleshooting for more help.

---

Related Documentation

• Architecture - Internal architecture details

• DHT Persistence - Distributed storage

• Validator Pool - Validator management

• gRPC API - API reference

• Configuration - Configuration options