# P2P Registry The P2P Registry is the decentralized coordination service for NCN Network v2. It handles node discovery, validator consensus, and subnet management using a libp2p-based peer-to-peer network. *** ## Overview ``` ┌─────────────────────────────────────────────────────────────────────────────┐ │ P2P Registry Node │ │ │ │ External Interfaces │ │ ┌─────────────────────┐ ┌─────────────────────┐ │ │ │ gRPC Server │ │ P2P Node │ │ │ │ (Port 50050) │ │ (Port 8828) │ │ │ │ │ │ │ │ │ │ • Node registration│ │ • Kademlia DHT │ │ │ │ • Validation RPCs │ │ • Peer discovery │ │ │ │ • Subnet mgmt │ │ • Data replication │ │ │ └─────────────────────┘ └─────────────────────┘ │ │ │ │ Core Services │ │ ┌─────────────────────────────────────────────────────────────────────┐ │ │ │ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │ │ │ │ │ Preprocessing │ │ Completion │ │ Subnet │ │ │ │ │ │ Service │ │ Service │ │ Manager │ │ │ │ │ └───────────────┘ └───────────────┘ └───────────────┘ │ │ │ │ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │ │ │ │ │ Validator │ │ Mempool │ │ DHT Storage │ │ │ │ │ │ Pool │ │ │ │ │ │ │ │ │ └───────────────┘ └───────────────┘ └───────────────┘ │ │ │ └─────────────────────────────────────────────────────────────────────┘ │ │ │ │ Blockchain Sync │ │ ┌─────────────────────────────────────────────────────────────────────┐ │ │ │ Validator Registry Contract Sync (periodic) │ │ │ └─────────────────────────────────────────────────────────────────────┘ │ │ │ └─────────────────────────────────────────────────────────────────────────────┘ ``` *** ## Features * **Node Discovery**: Kademlia DHT-based peer discovery * **Validator Consensus**: M-of-N signature collection * **Subnet Management**: Two-phase subnet creation with fee estimation * **State Persistence**: DHT-based distributed storage * **Blockchain Sync**: Automatic validator registry synchronization * **Mempool**: Temporary storage for pending validations *** ## Quick Start ### Build ```bash cargo build --release -p p2p_registry_node ``` ### Run ```bash cargo run --release --bin p2p_registry_node -- \ --p2p-listen-addr /ip4/0.0.0.0/tcp/8828 \ --grpc-listen-addr 127.0.0.1:50050 ``` **Expected Output**: ``` Starting P2P Registry Node... P2P listening on /ip4/0.0.0.0/tcp/8828 Local peer ID: 12D3KooWxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx gRPC server listening on 127.0.0.1:50050 ✓ Validator pool initialized (0 validators) ✓ Mempool initialized ✓ Subnet manager initialized ✓ P2P Registry ready ``` *** ## Architecture ### Internal Modules | Module | File | Purpose | | ---------------- | ------------------------------- | ----------------------- | | Main Service | `src/main.rs` | Service orchestration | | P2P Registry | `src/p2p_registry.rs` | libp2p networking | | Validator Pool | `src/validator_pool.rs` | Validator management | | Mempool | `src/mempool.rs` | Pending request storage | | Preprocessing | `src/preprocessing.rs` | Request validation | | Completion | `src/completion.rs` | Result validation | | Subnet Manager | `src/subnet_manager.rs` | Subnet CRUD | | DHT Persistence | `src/dht_persistence.rs` | Distributed storage | | Validator Signer | `src/validator_signer.rs` | Signature generation | | Validator Sync | `src/validator_sync_service.rs` | Blockchain sync | ### Service Dependencies ``` ┌─────────────────────────────────────────────────────────────────┐ │ P2P Registry │ │ │ │ ┌───────────────┐ │ │ │ gRPC Server │ │ │ └───────┬───────┘ │ │ │ │ │ ▼ │ │ ┌───────────────────────────────────────────────────────┐ │ │ │ Request Handler │ │ │ │ ┌─────────────────────────────────────────────────┐ │ │ │ │ │ RegisterNode │ DiscoverNodes │ Validation RPCs │ │ │ │ │ └─────────────────────────────────────────────────┘ │ │ │ └───────────────────────────┬───────────────────────────┘ │ │ │ │ │ ┌───────────────────┼───────────────────┐ │ │ ▼ ▼ ▼ │ │ ┌───────────────┐ ┌───────────────┐ ┌───────────────┐ │ │ │ Validator Pool│ │ Mempool │ │Subnet Manager │ │ │ │ │ │ │ │ │ │ │ │ • Selection │ │ • Pending reqs│ │ • Create │ │ │ │ • Reputation │ │ • TTL cleanup │ │ • Get/List │ │ │ │ • Slashing │ │ • Statistics │ │ • Validation │ │ │ └───────────────┘ └───────────────┘ └───────────────┘ │ │ │ │ │ │ │ └───────────────────┼───────────────────┘ │ │ ▼ │ │ ┌───────────────┐ │ │ │ DHT Storage │ │ │ │ (Kademlia) │ │ │ └───────────────┘ │ │ │ └─────────────────────────────────────────────────────────────────┘ ``` *** ## gRPC API ### P2PRegistryService #### Node Management | RPC | Request | Response | Description | | --------------------- | ---------------------------- | ----------------- | ---------------- | | `RegisterNode` | `NodeInfo` | `Ack` | Register a node | | `UpdateNodeHeartbeat` | `UpdateNodeHeartbeatRequest` | `Ack` | Update heartbeat | | `DeregisterNode` | `WalletAddress` | `Ack` | Remove node | | `DiscoverNodes` | `DiscoverNodesRequest` | `stream NodeInfo` | Find nodes | #### Validation | RPC | Request | Response | Description | | -------------------------------- | ------------------- | ------------------------- | ---------------------------- | | `RequestPreprocessingValidation` | `InferenceRequest` | `PreprocessingValidation` | Get preprocessing validation | | `RequestCompletionValidation` | `InferenceResponse` | `CompletionValidation` | Get completion validation | #### Subnet Management | RPC | Request | Response | Description | | ------------------- | -------------------------- | --------------------------- | --------------------- | | `EstimateSubnetFee` | `EstimateSubnetFeeRequest` | `EstimateSubnetFeeResponse` | Estimate creation fee | | `CreateSubnet` | `CreateSubnetRequest` | `CreateSubnetResponse` | Create subnet | | `GetSubnet` | `GetSubnetRequest` | `GetSubnetResponse` | Get subnet info | | `ListSubnets` | `Empty` | `stream SubnetMetadata` | List all subnets | #### Validator Management | RPC | Request | Response | Description | | --------------------------- | ---------------------------- | ----------------------------- | ------------------ | | `RegisterValidator` | `RegisterValidatorRequest` | `RegisterValidatorResponse` | Register validator | | `UnregisterValidator` | `UnregisterValidatorRequest` | `Ack` | Unregister | | `GetValidatorInfo` | `ValidatorAddressRequest` | `ValidatorInfoResponse` | Get validator info | | `ListValidators` | `ListValidatorsRequest` | `stream ValidatorInfoMessage` | List validators | | `UpdateValidatorReputation` | `UpdateReputationRequest` | `Ack` | Update reputation | See [gRPC API Reference](https://github.com/NeuroChainAi/docs-guides/blob/main/components/p2p-registry/grpc-api.md) for detailed documentation. *** ## P2P Networking ### Kademlia DHT The registry uses libp2p's Kademlia DHT for: * Peer discovery * Distributed storage * Network resilience ### Peer Discovery ``` Node A DHT Node B │ │ │ │ 1. Bootstrap │ │ │───────────────────────▶│ │ │ │ │ │ 2. FIND_NODE │ │ │───────────────────────▶│ │ │ │ │ │ 3. K closest peers │ │ │◀───────────────────────│ │ │ │ │ │ 4. Connect to peers │ │ │─────────────────────────────────────────────────▶│ │ │ │ ``` ### Bootstrap Nodes ```bash # Connect to bootstrap nodes --bootstrap /ip4/192.168.1.100/tcp/8828/p2p/12D3KooW... --bootstrap /ip4/192.168.1.101/tcp/8828/p2p/12D3KooX... ``` *** ## Validator Pool ### Features * **Registration**: Add/remove validators * **Selection**: Multiple strategies (random, reputation, stake) * **Reputation**: Track validation success/failure * **Slashing**: Penalize misbehavior * **Rotation**: Periodic validator rotation ### Configuration ```bash # Validator pool settings MIN_VALIDATORS_FOR_CONSENSUS=3 MAX_VALIDATORS_TO_SELECT=5 MIN_REPUTATION_THRESHOLD=20 VALIDATOR_ROTATION_INTERVAL_SECS=3600 ``` See [Validator Pool](https://github.com/NeuroChainAi/docs-guides/blob/main/components/p2p-registry/validator-pool.md) for details. *** ## Mempool ### Structure ```rust pub struct Mempool { pending_requests: HashMap>, preprocessing_validations: HashMap>, completion_validations: HashMap>, } ``` ### TTL Configuration | Entry Type | Default TTL | | ------------------------- | ----------- | | Pending requests | 10 minutes | | Preprocessing validations | 15 minutes | | Completion validations | 5 minutes | ### Cleanup Automatic cleanup runs every 60 seconds: ```rust // Cleanup task loop { tokio::time::sleep(Duration::from_secs(60)).await; let stats = mempool.lock().await.cleanup_expired(); log::info!("Mempool cleanup: {:?}", stats); } ``` *** ## Subnet Manager ### Subnet Creation Flow ``` 1. EstimateSubnetFee ├── Calculate fee based on models ├── Generate fee_quote_id (valid 10 min) └── Return fee + contract address 2. Client pays on-chain 3. CreateSubnet ├── Verify payment_tx on blockchain ├── Verify config_hash matches quote ├── Collect validator signatures ├── Store subnet metadata └── Return subnet_id ``` ### SubnetMetadata ```protobuf message SubnetMetadata { uint64 subnet_id = 1; string gateway_address = 2; int64 created_at = 3; repeated ModelDescriptor models = 4; repeated ValidatorSignature validated_by = 5; string network_fee_wei = 6; string payment_tx = 7; string status = 9; // pending, active, deprecated } ``` *** ## Configuration ### Command Line Arguments | Argument | Required | Default | Description | | -------------------- | -------- | ----------------------- | -------------------------- | | `--p2p-listen-addr` | No | `/ip4/0.0.0.0/tcp/8828` | P2P listen address | | `--grpc-listen-addr` | No | `127.0.0.1:50050` | gRPC listen address | | `--bootstrap` | No | - | Bootstrap node addresses | | `--data-dir` | No | `./data` | Data persistence directory | ### Environment Variables | Variable | Required | Default | Description | | ----------------------------- | -------- | ------- | --------------------------- | | `VALIDATOR_PRIVATE_KEY` | Yes\* | - | Validator signing key | | `VALIDATOR_SYNC_ENABLED` | No | `true` | Enable blockchain sync | | `VALIDATOR_SYNC_INTERVAL` | No | `60` | Sync interval (seconds) | | `VALIDATOR_REGISTRY_CONTRACT` | No | - | On-chain validator registry | | `FORKNET_RPC_URL` | No | - | Blockchain RPC URL | | `CHAIN_ID` | No | `828` | Blockchain chain ID | \*Auto-generated if not provided (development only) See [Configuration Reference](https://github.com/NeuroChainAi/docs-guides/blob/main/components/p2p-registry/configuration.md) for complete details. *** ## Background Tasks ### Automatic Tasks | Task | Interval | Purpose | | ------------------ | -------- | ----------------------- | | Mempool cleanup | 60s | Remove expired entries | | Validator rotation | 1 hour | Rotate validators | | Statistics logging | 5 min | Log pool/mempool stats | | Blockchain sync | 60s | Sync validator registry | ### Task Implementation ```rust // Mempool cleanup task tokio::spawn(async move { loop { tokio::time::sleep(Duration::from_secs(60)).await; let stats = mempool.lock().await.cleanup_expired(); log::info!("Mempool cleanup: removed {} expired entries", stats.total_removed); } }); // Validator rotation task tokio::spawn(async move { loop { tokio::time::sleep(Duration::from_secs(3600)).await; let rotated = validator_pool.lock().await.rotate_validators(); log::info!("Validator rotation: {} validators affected", rotated); } }); ``` *** ## Monitoring ### Metrics | Metric | Type | Description | | ---------------------------- | ------- | ----------------- | | `registry_nodes_registered` | Gauge | Registered nodes | | `registry_validators_active` | Gauge | Active validators | | `registry_mempool_size` | Gauge | Mempool entries | | `registry_subnets_total` | Counter | Total subnets | | `registry_validations_total` | Counter | Total validations | | `registry_p2p_peers` | Gauge | Connected peers | ### Health Endpoint ```bash curl http://127.0.0.1:50050/health # Response { "status": "healthy", "p2p": { "peer_id": "12D3KooW...", "connected_peers": 5 }, "validators": { "active": 10, "total": 15 }, "mempool": { "pending_requests": 3, "preprocessing": 2, "completion": 1 } } ``` *** ## Troubleshooting ### Common Issues **"No validators available"** ``` Error: Insufficient validators for consensus ``` * Check validators are registered and active * Verify `VALIDATOR_SYNC_ENABLED=true` * Check blockchain RPC connectivity **"P2P bootstrap failed"** ``` Error: Failed to connect to bootstrap nodes ``` * Verify bootstrap node addresses * Check network connectivity * Ensure ports are open (8828) **"Mempool full"** ``` Warning: Mempool size exceeds threshold ``` * Increase cleanup frequency * Check for stuck requests * Verify downstream services (gateways) See [Registry Troubleshooting](https://github.com/NeuroChainAi/docs-guides/blob/main/troubleshooting/registry-issues.md) for more help. *** ## Related Documentation * [Architecture](https://github.com/NeuroChainAi/docs-guides/blob/main/components/p2p-registry/architecture.md) - Internal architecture details * [DHT Persistence](https://github.com/NeuroChainAi/docs-guides/blob/main/components/p2p-registry/dht-persistence.md) - Distributed storage * [Validator Pool](https://github.com/NeuroChainAi/docs-guides/blob/main/components/p2p-registry/validator-pool.md) - Validator management * [gRPC API](https://github.com/NeuroChainAi/docs-guides/blob/main/components/p2p-registry/grpc-api.md) - API reference * [Configuration](https://github.com/NeuroChainAi/docs-guides/blob/main/components/p2p-registry/configuration.md) - Configuration options