Enable PDP

This guide walks you through setting up a PDP-enabled Filecoin Storage Provider using Lotus, YugabyteDB, and Curio

For a PDP-only node without PoRep/sealing (skiff), see the Curio-PDP runbook and Skiff binary.

ALPHA FEATURE - UNDER DEVELOPMENT

This documentation covers the PDP (Proof of Data Possession) feature, which is currently in alpha and under active development. This tool is intended for testing and experimental use only.

For production use and submitting real deals with live PDP Storage Providers, please use the Synapse SDK.

🚀 Prerequisites

Note: This guide is written specifically for Ubuntu 22.04. If you are using a different Linux distribution, refer to the relevant documentation for package installation and compatibility.

Before starting, make sure you have a user with sudo privileges. This section prepares your system for the PDP stack.

---

⚙️ Hardware requirements

• RAM: 32 GiB+

• CPU: 8 Core+

• Storage:

  • 1 TiB Fast storage (NVMe/SSD)

  • 10 TiB Long-term storage (HDD)

• GPU: Not required

• Connectivity: Public HTTPS endpoint (domain)

---

🧰 System Package Installation

sudo apt update && sudo apt upgrade -y && sudo apt install -y \
  mesa-opencl-icd ocl-icd-opencl-dev gcc git jq pkg-config curl clang \
  build-essential hwloc libhwloc-dev libarchive-dev wget ntp python-is-python3 aria2

---

🔨 Install Go (match go.mod)

Curio’s minimum Go version is set in the Curio repo at go.mod.

Example (current repo min is 1.26.2):

sudo rm -rf /usr/local/go
wget https://go.dev/dl/go1.26.2.linux-amd64.tar.gz
sudo tar -C /usr/local -xzf go1.26.2.linux-amd64.tar.gz
echo 'export PATH=$PATH:/usr/local/go/bin' >> ~/.bashrc
source ~/.bashrc
go version

You should see something like: go version go1.26.2 linux/amd64

---

🔧 Install Rust

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

When prompted, choose the option 1) Proceed with standard installation (default — just press Enter).

source $HOME/.cargo/env
rustc --version

You should see something like: rustc 1.86.0 (05f9846f8 2025-03-31)

---

🔐 Add Go and Rust to Secure Sudo Path

sudo tee /etc/sudoers.d/dev-paths <<EOF
Defaults secure_path="/usr/local/go/bin:$HOME/.cargo/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
EOF

---

---

PDP schema / migrations (important)

PDP requires database schema migrations.

How to confirm PDP schema exists:

• Connect to YSQL and verify expected tables exist in the curio schema.

ysqlsh -h "$CURIO_DB_HOST" -p "${CURIO_DB_PORT:-5433}" -U "$CURIO_DB_USER" -d "${CURIO_DB_NAME:-yugabyte}" -c "\dn+ curio"

Migration reference:

• The PDP schema is added via Curio’s HarmonyDB migrations (for example: harmony/harmonydb/sql/20240930-pdp.sql).

If you upgraded Curio binaries but PDP still fails:

• check Curio logs on startup for schema upgrade output

• confirm you are connected to the correct DB (CURIO_DB_NAME default yugabyte) and schema (curio)

---

Ports & domain names (PDP vs Market)

PDP commonly confuses operators because multiple HTTP-exposed services may exist in a Curio deployment.

Recommended patterns:

Pattern A: single domain, one Curio HTTP server

• One public domain (e.g. curio.example.com)

• One HTTP server handles multiple routes

• Reverse proxy optional (see HTTP server docs)

Pattern B: separate domains for separate services

• pdp.example.com and market.example.com

• Useful when you want different auth/proxying/caching policies

Checklist:

• Decide whether Curio terminates TLS or a reverse proxy does.

• Ensure inbound 80/443 is reachable for Let’s Encrypt (if used).

See:

• documentation/en/curio-market/curio-http-server.md

---

Troubleshooting: pdptool ping works locally but not remotely

Common causes:

• firewall blocks inbound traffic

• wrong domain/port

• TLS delegation mismatch

What to do:

• test from an external host

• confirm DNS and ports 80/443 (or your proxy) are correct

• include full logs + command used

⛓️ Installing and Running Lotus

🧠 Lotus is your gateway to the Filecoin network. It syncs the chain, manages wallets, and is required for Curio to interact with your node.

Lotus Documentation

Lotus Support Channels

🔧 Build Lotus Daemon

Clone and check out Lotus:

git clone https://github.com/filecoin-project/lotus.git
cd lotus
git checkout $(curl -s https://api.github.com/repos/filecoin-project/lotus/releases/latest | jq -r .tag_name)

Build and Install for Mainnet

make clean && make lotus
sudo make install-daemon
lotus --version

Build and Install for Calibration

make clean && make GOFLAGS="-tags=calibnet" lotus
sudo make install-daemon
lotus --version

You should see something like: lotus version 1.32.2+calibnet+git.ff88d8269

---

📦 Import a Snapshot and Start the Daemon

Download the Snapshot

Mainnet:

aria2c -x5 -o snapshot.car.zst https://forest-archive.chainsafe.dev/latest/mainnet/

Calibration:

aria2c -x5 -o snapshot.car.zst https://forest-archive.chainsafe.dev/latest/calibnet/

Import and Start the Daemon

lotus daemon --import-snapshot snapshot.car.zst --remove-existing-chain --halt-after-import
nohup lotus daemon > ~/lotus.log 2>&1 &

If you encounter errors related to EnableEthRPC or EnableIndexer, run the following command and restart Lotus

sed -i 's/^\( *\)#*EnableEthRPC = .*/\1EnableEthRPC = true/; s/^\( *\)#*EnableIndexer = .*/\1EnableIndexer = true/' ~/.lotus/config.toml

Monitor Sync Progress

lotus sync wait

To monitor continuously:

lotus sync wait --watch

Monitor Logs

tail -f ~/lotus.log

---

🐘 Running YugabyteDB

🧠 Curio uses YugabyteDB to store metadata about deals, sealing operations, and PDP submissions.

Yugabyte Documentation

Yugabyte Support Channels

🛠 Set ulimit configuration

Before starting Yugabyte, you must increase the default ulimit values to ensure system limits do not interfere with the database.

To do this:

🔁 Persist new limits across reboots

Add these lines to /etc/security/limits.conf:

echo "$(whoami) soft nofile 1048576" | sudo tee -a /etc/security/limits.conf
echo "$(whoami) hard nofile 1048576" | sudo tee -a /etc/security/limits.conf

This ensures the increased limits are automatically applied to future sessions.

⚡ Apply limit immediately (for current shell only)

ulimit -n 1048576

Verify:

ulimit -n

This should output 1048576.

⚙️ Install Yugabyte

wget https://software.yugabyte.com/releases/2.25.1.0/yugabyte-2.25.1.0-b381-linux-x86_64.tar.gz
tar xvfz yugabyte-2.25.1.0-b381-linux-x86_64.tar.gz
cd yugabyte-2.25.1.0
./bin/post_install.sh

🚀 Start the DB

./bin/yugabyted start \
  --advertise_address 127.0.0.1 \
  --master_flags rpc_bind_addresses=127.0.0.1 \
  --tserver_flags rpc_bind_addresses=127.0.0.1

If you encounter locale-related errors when starting Yugabyte for the first time, run:

sudo locale-gen en_US.UTF-8

Visit http://127.0.0.1:15433 to confirm successful installation. This is the YugabyteDB web UI — it should display the dashboard if the service is running correctly and all nodes are healthy.

You can also check your Yugabyte cluster details directly in the CLI with:

./bin/yugabyted status

---

🧱 Installing and Configuring Curio

🧠 Curio is the core PDP client that coordinates sealing, interacts with Lotus and submits PDP proofs.

Curio Documentation

Curio Support Channels

⚙️ System Configuration

Before you proceed with the installation, you should increase the UDP buffer size:

sudo sysctl -w net.core.rmem_max=2097152
sudo sysctl -w net.core.rmem_default=2097152

To make this change persistent across reboots:

echo 'net.core.rmem_max=2097152' | sudo tee -a /etc/sysctl.conf
echo 'net.core.rmem_default=2097152' | sudo tee -a /etc/sysctl.conf

🔬 Build Curio

Clone the repository and switch to the PDP branch:

git clone https://github.com/filecoin-project/curio.git
cd curio
git checkout pdpM3d

Curio is compiled for a specific Filecoin network at build time. Choose the appropriate build command below.

Mainnet

make clean build

Calibration

make clean calibnet

This step will take a few minutes to complete.

✅ Install and Verify Curio

Run the following to install the compiled binary:

sudo make install

This will place curio in /usr/local/bin

Verify the installation:

curio --version

Expected example output:

curio version 1.24.4+calibnet+git_f954c0a_2025-04-06T15:46:32-04:00

---

🔧 Guided Setup

Curio provides a utility to help you set up a new miner interactively. Run the following command:

curio guided-setup

1️⃣ Select Curio Installation Type

Use the arrow keys to navigate the guided setup menu and select "Setup non-Storage Provider cluster".

2️⃣ Enter Your YugabyteDB Connection Details

If you used the default installation steps from this guide, the following values should work:

• Host: 127.0.0.1

• Port: 5433

• Username: yugabyte

• Password: yugabyte

• Database: yugabyte

You can verify these settings by running the following command from the Yugabyte directory:

./bin/yugabyted status

After selecting "Continue to connect and update schema", Curio will automatically create the required tables and schema in the database.

3️⃣ Telemetry (Optional)

You'll be asked whether to share anonymised or signed telemetry with the Curio team to help improve the software.

Select your preference and continue.

4️⃣ Save Database Configuration

At the final step of the guided setup, you'll be prompted to choose where to save your database configuration file.

Use the arrow keys to select a location. A common default is:

/home/your-username/curio.env

Once selected, setup will complete, and the miner configuration will be stored.

5️⃣ Launch the Curio Web GUI

To explore the Curio interface visually, start the GUI layer:

curio run --layers=gui

Then, open your browser and go to:

http://127.0.0.1:4701

This will launch the Curio web GUI locally.

---

🧪 Enabling FWSS PDP

🧠 This section enables FWSS Proof of Data Possession (PDP) on your SP node using Curio. These steps guide you through running a standalone PDP service using Curio and pdptool.

PDP Support Channels

📦 Attach Storage Locations

With Curio running with the GUI layer:

curio run --layers=gui

Run the following commands in your Curio CLI to attach storage paths:

curio cli storage attach --init --seal /fast-storage/path
curio cli storage attach --init --store /long-term-storage/path

Your fast-storage path should point to high-performance storage media such as NVMe or SSD

---

🔧 Add a PDP Configuration Layer

Browse to the Configurations page of the Curio GUI.

Create a new layer named pdp and enable the following under Subsystems:

You may find it helpful to search for the setting names in your browser.

• ✅ EnableParkPiece

• ✅ EnablePDP

• ✅ EnableCommP

• ✅ EnableMoveStorage

In the HTTP section:

• ✅ Enable: true

• 🌐 DomainName: your domain (e.g., pdp.mydomain.com)

• 📡 ListenAddress: 0.0.0.0:443

Tip: You must point your domain's A record to your server's public IP address for Let's Encrypt to issue a certificate.

---

💰 Import your Filecoin Wallet Private Key:

There are several ways to obtain private keys for Ethereum addresses. In this guide, we will use a new delegated FIL wallet address.

Create a new delegated wallet:

lotus wallet new delegated

# Example output:
t410fuo4dghaeiqzokiqnxruzdr6e3cjktnxprrc56bi

You can display your Lotus wallets at any time by running:

lotus wallet list

Export & convert your new delegated wallet address private key:

lotus wallet export <your-delegated-wallet-address> | xxd -r -p | jq -r '.PrivateKey' | base64 -d | xxd -p -c 32

# Example output:
d4c2e3f9a716bb0e47fa91b2cf4a29870be3c5982fd6eafed71e8ac3f9c0b127

Browse to the PDP page of the Curio GUI and in the Owner Address section:

• Select Import Key

• Copy the previously generated private wallet key into the Private Key (Hex) field.

• Select Import Key

Your 0x wallet address - the delegated Ethereum address derived from your Filecoin delegated wallet private key - will be added to the Owner Address section of the Curio PDP page.

Make sure to send a small amount of FIL or tFIL (testnet FIL) to your 0x wallet - we recommend 8 FIL for Mainnet & 5 tFIL for Calibration to ensure uninterrupted PDP operation during initial setup and testing. Calibration test FIL faucet information.

Important: Secure your private key material. Don't expose or store it in plain text without protection.

---

🚀 Restart and Verify

Restart Curio with both layers:

curio run --layers=gui,pdp

If you encounter errors related to EnableEthRPC or EnableIndexer, run the following command and restart Lotus

sed -i 's/^\( *\)#*EnableEthRPC = .*/\1EnableEthRPC = true/; s/^\( *\)#*EnableIndexer = .*/\1EnableIndexer = true/' ~/.lotus/config.toml

If you encounter errors binding to port 443 when starting Curio with the pdp configuration layer, run:

sudo setcap 'cap_net_bind_service=+ep' /usr/local/bin/curio

Test the PDP service:

If pdptool is not installed, clone and build Curio:

git clone https://github.com/filecoin-project/curio.git
cd curio/cmd/pdptool
go build .

./pdptool ping --service-url https://your-domain.com --service-name public

Always use public for the --service-name flag

Expected output:

Ping successful: Service is reachable and JWT token is valid.

---

🎉 You're Done!

You've successfully launched a PDP-enabled Filecoin Storage Provider stack. Your system is now:

• ✅ Syncing with the Filecoin network via Lotus

• ✅ Recording deal and sector metadata in YugabyteDB

• ✅ Operating Curio to manage sealing and coordination

• ✅ Enabled Proof of Data Possession (PDP)

• ✅ Connected to your PDP-enabled storage provider

---

🔜 Next Steps

• ✔️ Register your FWSS node

• 🔗 Explore FWSS & PDP tools & resources at https://www.filecoin.services

• 💬 Join the community - Filecoin Slack - #fil-pdp