P2P Registry

The P2P Registry is the decentralized coordination service for NCN Network v2. It handles node discovery, validator consensus, and subnet management using a libp2p-based peer-to-peer network.

Overview

┌─────────────────────────────────────────────────────────────────────────────┐
│                            P2P Registry Node                                 │
│                                                                             │
│   External Interfaces                                                       │
│   ┌─────────────────────┐   ┌─────────────────────┐                        │
│   │    gRPC Server      │   │     P2P Node        │                        │
│   │    (Port 50050)     │   │    (Port 8828)      │                        │
│   │                     │   │                     │                        │
│   │  • Node registration│   │  • Kademlia DHT     │                        │
│   │  • Validation RPCs  │   │  • Peer discovery   │                        │
│   │  • Subnet mgmt      │   │  • Data replication │                        │
│   └─────────────────────┘   └─────────────────────┘                        │
│                                                                             │
│   Core Services                                                             │
│   ┌─────────────────────────────────────────────────────────────────────┐   │
│   │  ┌───────────────┐  ┌───────────────┐  ┌───────────────┐           │   │
│   │  │ Preprocessing │  │  Completion   │  │    Subnet     │           │   │
│   │  │   Service     │  │   Service     │  │   Manager     │           │   │
│   │  └───────────────┘  └───────────────┘  └───────────────┘           │   │
│   │  ┌───────────────┐  ┌───────────────┐  ┌───────────────┐           │   │
│   │  │   Validator   │  │    Mempool    │  │  DHT Storage  │           │   │
│   │  │     Pool      │  │               │  │               │           │   │
│   │  └───────────────┘  └───────────────┘  └───────────────┘           │   │
│   └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
│   Blockchain Sync                                                           │
│   ┌─────────────────────────────────────────────────────────────────────┐   │
│   │  Validator Registry Contract Sync (periodic)                         │   │
│   └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

Features

Node Discovery: Kademlia DHT-based peer discovery
Validator Consensus: M-of-N signature collection
Subnet Management: Two-phase subnet creation with fee estimation
State Persistence: DHT-based distributed storage
Blockchain Sync: Automatic validator registry synchronization
Mempool: Temporary storage for pending validations

Quick Start

Build

cargo build --release -p p2p_registry_node

Run

cargo run --release --bin p2p_registry_node -- \
  --p2p-listen-addr /ip4/0.0.0.0/tcp/8828 \
  --grpc-listen-addr 127.0.0.1:50050

Expected Output:

Starting P2P Registry Node...
P2P listening on /ip4/0.0.0.0/tcp/8828
Local peer ID: 12D3KooWxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
gRPC server listening on 127.0.0.1:50050
✓ Validator pool initialized (0 validators)
✓ Mempool initialized
✓ Subnet manager initialized
✓ P2P Registry ready

Architecture

Internal Modules

Module

File

Purpose

Main Service

src/main.rs

Service orchestration

P2P Registry

src/p2p_registry.rs

libp2p networking

Validator Pool

src/validator_pool.rs

Validator management

Mempool

src/mempool.rs

Pending request storage

Preprocessing

src/preprocessing.rs

Request validation

Completion

src/completion.rs

Result validation

Subnet Manager

src/subnet_manager.rs

Subnet CRUD

DHT Persistence

src/dht_persistence.rs

Distributed storage

Validator Signer

src/validator_signer.rs

Signature generation

Validator Sync

src/validator_sync_service.rs

Blockchain sync

Service Dependencies

┌─────────────────────────────────────────────────────────────────┐
│                        P2P Registry                              │
│                                                                 │
│   ┌───────────────┐                                             │
│   │  gRPC Server  │                                             │
│   └───────┬───────┘                                             │
│           │                                                     │
│           ▼                                                     │
│   ┌───────────────────────────────────────────────────────┐     │
│   │                  Request Handler                       │     │
│   │  ┌─────────────────────────────────────────────────┐  │     │
│   │  │ RegisterNode │ DiscoverNodes │ Validation RPCs │  │     │
│   │  └─────────────────────────────────────────────────┘  │     │
│   └───────────────────────────┬───────────────────────────┘     │
│                               │                                 │
│           ┌───────────────────┼───────────────────┐             │
│           ▼                   ▼                   ▼             │
│   ┌───────────────┐   ┌───────────────┐   ┌───────────────┐     │
│   │ Validator Pool│   │   Mempool     │   │Subnet Manager │     │
│   │               │   │               │   │               │     │
│   │ • Selection   │   │ • Pending reqs│   │ • Create      │     │
│   │ • Reputation  │   │ • TTL cleanup │   │ • Get/List    │     │
│   │ • Slashing    │   │ • Statistics  │   │ • Validation  │     │
│   └───────────────┘   └───────────────┘   └───────────────┘     │
│           │                   │                   │             │
│           └───────────────────┼───────────────────┘             │
│                               ▼                                 │
│                       ┌───────────────┐                         │
│                       │  DHT Storage  │                         │
│                       │  (Kademlia)   │                         │
│                       └───────────────┘                         │
│                                                                 │
└─────────────────────────────────────────────────────────────────┘

gRPC API

P2PRegistryService

Node Management

RPC

Request

Response

Description

RegisterNode

NodeInfo

Ack

UpdateNodeHeartbeat

UpdateNodeHeartbeatRequest

Ack

Update heartbeat

DeregisterNode

WalletAddress

Ack

Remove node

DiscoverNodes

DiscoverNodesRequest

stream NodeInfo

Find nodes

Validation

RPC

Request

Response

Description

RequestPreprocessingValidation

InferenceRequest

PreprocessingValidation

Get preprocessing validation

RequestCompletionValidation

InferenceResponse

CompletionValidation

Get completion validation

Subnet Management

RPC

Request

Response

Description

EstimateSubnetFee

EstimateSubnetFeeRequest

EstimateSubnetFeeResponse

Estimate creation fee

CreateSubnet

CreateSubnetRequest

CreateSubnetResponse

Create subnet

GetSubnet

GetSubnetRequest

GetSubnetResponse

Get subnet info

ListSubnets

Empty

stream SubnetMetadata

List all subnets

Validator Management

RPC

Request

Response

Description

RegisterValidator

RegisterValidatorRequest

RegisterValidatorResponse

UnregisterValidator

UnregisterValidatorRequest

Ack

Unregister

GetValidatorInfo

ValidatorAddressRequest

ValidatorInfoResponse

Get validator info

ListValidators

ListValidatorsRequest

stream ValidatorInfoMessage

List validators

UpdateValidatorReputation

UpdateReputationRequest

Ack

Update reputation

See gRPC API Reference for detailed documentation.

P2P Networking

Kademlia DHT

The registry uses libp2p's Kademlia DHT for:

Peer discovery
Distributed storage
Network resilience

Peer Discovery

Node A                    DHT                    Node B
  │                        │                        │
  │  1. Bootstrap          │                        │
  │───────────────────────▶│                        │
  │                        │                        │
  │  2. FIND_NODE          │                        │
  │───────────────────────▶│                        │
  │                        │                        │
  │  3. K closest peers    │                        │
  │◀───────────────────────│                        │
  │                        │                        │
  │  4. Connect to peers   │                        │
  │─────────────────────────────────────────────────▶│
  │                        │                        │

Bootstrap Nodes

# Connect to bootstrap nodes
--bootstrap /ip4/192.168.1.100/tcp/8828/p2p/12D3KooW...
--bootstrap /ip4/192.168.1.101/tcp/8828/p2p/12D3KooX...

Validator Pool

Features

Registration: Add/remove validators
Selection: Multiple strategies (random, reputation, stake)
Reputation: Track validation success/failure
Slashing: Penalize misbehavior
Rotation: Periodic validator rotation

Configuration

# Validator pool settings
MIN_VALIDATORS_FOR_CONSENSUS=3
MAX_VALIDATORS_TO_SELECT=5
MIN_REPUTATION_THRESHOLD=20
VALIDATOR_ROTATION_INTERVAL_SECS=3600

See Validator Pool for details.

Mempool

Structure

pub struct Mempool {
    pending_requests: HashMap<String, MempoolEntry<InferenceRequest>>,
    preprocessing_validations: HashMap<String, MempoolEntry<PreprocessingValidation>>,
    completion_validations: HashMap<String, MempoolEntry<CompletionValidation>>,
}

TTL Configuration

Entry Type

Default TTL

Pending requests

10 minutes

Preprocessing validations

15 minutes

Completion validations

5 minutes

Cleanup

Automatic cleanup runs every 60 seconds:

// Cleanup task
loop {
    tokio::time::sleep(Duration::from_secs(60)).await;
    let stats = mempool.lock().await.cleanup_expired();
    log::info!("Mempool cleanup: {:?}", stats);
}

Subnet Manager

Subnet Creation Flow

1. EstimateSubnetFee
   ├── Calculate fee based on models
   ├── Generate fee_quote_id (valid 10 min)
   └── Return fee + contract address

2. Client pays on-chain

3. CreateSubnet
   ├── Verify payment_tx on blockchain
   ├── Verify config_hash matches quote
   ├── Collect validator signatures
   ├── Store subnet metadata
   └── Return subnet_id

SubnetMetadata

message SubnetMetadata {
  uint64 subnet_id = 1;
  string gateway_address = 2;
  int64 created_at = 3;
  repeated ModelDescriptor models = 4;
  repeated ValidatorSignature validated_by = 5;
  string network_fee_wei = 6;
  string payment_tx = 7;
  string status = 9;  // pending, active, deprecated
}

Configuration

Command Line Arguments

Argument

Required

Default

Description

--p2p-listen-addr

/ip4/0.0.0.0/tcp/8828

P2P listen address

--grpc-listen-addr

127.0.0.1:50050

gRPC listen address

--bootstrap

Bootstrap node addresses

--data-dir

./data

Data persistence directory

Environment Variables

Variable

Required

Default

Description

VALIDATOR_PRIVATE_KEY

Yes*

Validator signing key

VALIDATOR_SYNC_ENABLED

true

Enable blockchain sync

VALIDATOR_SYNC_INTERVAL

60

Sync interval (seconds)

VALIDATOR_REGISTRY_CONTRACT

On-chain validator registry

FORKNET_RPC_URL

Blockchain RPC URL

CHAIN_ID

828

Blockchain chain ID

*Auto-generated if not provided (development only)

See Configuration Reference for complete details.

Background Tasks

Automatic Tasks

Task

Interval

Purpose

Mempool cleanup

60s

Remove expired entries

Validator rotation

1 hour

Rotate validators

Statistics logging

5 min

Log pool/mempool stats

Blockchain sync

60s

Sync validator registry

Task Implementation

// Mempool cleanup task
tokio::spawn(async move {
    loop {
        tokio::time::sleep(Duration::from_secs(60)).await;
        let stats = mempool.lock().await.cleanup_expired();
        log::info!("Mempool cleanup: removed {} expired entries", stats.total_removed);
    }
});

// Validator rotation task
tokio::spawn(async move {
    loop {
        tokio::time::sleep(Duration::from_secs(3600)).await;
        let rotated = validator_pool.lock().await.rotate_validators();
        log::info!("Validator rotation: {} validators affected", rotated);
    }
});

Monitoring

Metrics

Metric

Type

Description

registry_nodes_registered

Gauge

Registered nodes

registry_validators_active

Gauge

Active validators

registry_mempool_size

Gauge

Mempool entries

registry_subnets_total

Counter

Total subnets

registry_validations_total

Counter

Total validations

registry_p2p_peers

Gauge

Connected peers

Health Endpoint

curl http://127.0.0.1:50050/health

# Response
{
  "status": "healthy",
  "p2p": {
    "peer_id": "12D3KooW...",
    "connected_peers": 5
  },
  "validators": {
    "active": 10,
    "total": 15
  },
  "mempool": {
    "pending_requests": 3,
    "preprocessing": 2,
    "completion": 1
  }
}

Troubleshooting

Common Issues

"No validators available"

Error: Insufficient validators for consensus

Check validators are registered and active
Verify VALIDATOR_SYNC_ENABLED=true
Check blockchain RPC connectivity

"P2P bootstrap failed"

Error: Failed to connect to bootstrap nodes

Verify bootstrap node addresses
Check network connectivity
Ensure ports are open (8828)

"Mempool full"

Warning: Mempool size exceeds threshold

Increase cleanup frequency
Check for stuck requests
Verify downstream services (gateways)

See Registry Troubleshooting for more help.

Architecture - Internal architecture details
DHT Persistence - Distributed storage
Validator Pool - Validator management
gRPC API - API reference
Configuration - Configuration options

PreviousCompute Node NextSmart Contracts

Last updated 1 month ago

hashtagOverview

hashtagFeatures

hashtagQuick Start

hashtagBuild

hashtagRun

hashtagArchitecture

hashtagInternal Modules

hashtagService Dependencies

hashtaggRPC API

hashtagP2PRegistryService

hashtagNode Management

hashtagValidation

hashtagSubnet Management

hashtagValidator Management

hashtagP2P Networking

hashtagKademlia DHT

hashtagPeer Discovery

hashtagBootstrap Nodes

hashtagValidator Pool

hashtagFeatures

hashtagConfiguration

hashtagMempool

hashtagStructure

hashtagTTL Configuration

hashtagCleanup

hashtagSubnet Manager

hashtagSubnet Creation Flow

hashtagSubnetMetadata

hashtagConfiguration

hashtagCommand Line Arguments

hashtagEnvironment Variables

hashtagBackground Tasks

hashtagAutomatic Tasks

hashtagTask Implementation

hashtagMonitoring

hashtagMetrics

hashtagHealth Endpoint

hashtagTroubleshooting

hashtagCommon Issues

hashtagRelated Documentation

Overview

Features

Quick Start

Build

Run

Architecture

Internal Modules

Service Dependencies

gRPC API

P2PRegistryService

Node Management

Validation

Subnet Management

Validator Management

P2P Networking

Kademlia DHT

Peer Discovery

Bootstrap Nodes

Validator Pool

Features

Configuration

Mempool

Structure

TTL Configuration

Cleanup

Subnet Manager

Subnet Creation Flow

SubnetMetadata

Configuration

Command Line Arguments

Environment Variables

Background Tasks

Automatic Tasks

Task Implementation

Monitoring

Metrics

Health Endpoint

Troubleshooting

Common Issues

Related Documentation