Benchmark
Real-world Conversion Benchmark
The following benchmark was performed on a production Cosmos node dataset to measure end-to-end LevelDB to PebbleDB conversion performance.
============================================================
CONVERSION METRICS SUMMARY
============================================================
Global Statistics:
Total duration: 4m11s
Total keys: 217895735
Total data read: 39.54 GiB
Total data written: 39.54 GiB
Avg throughput: 865721 keys/sec, 160.88 MB/sec
Write/Read ratio: 100.0%
Per-Database Statistics:
blockstore.db:
Keys: 57
Duration: 0s
Throughput: 2059 keys/sec, 24.07 MB/sec
Avg sizes: key=1226 B, value=11032 B
tx_index.db:
Keys: 15557
Duration: 0s
Throughput: 775361 keys/sec, 63.37 MB/sec
Avg sizes: key=9 B, value=77 B
application.db:
Keys: 217872868
Duration: 4m10s
Throughput: 871027 keys/sec, 160.94 MB/sec
Avg sizes: key=19 B, value=174 B
state.db:
Keys: 7253
Duration: 0s
Throughput: 32939 keys/sec, 1070.39 MB/sec
Avg sizes: key=3407 B, value=30667 B
============================================================
Conversion completed successfully.
New Pebble-backed data directory: pebbledb/data
This run duration: 4m11s
Total elapsed time: 5m10s (since first start)
Size summary:
Source (LevelDB) data: 23.32 GiB (25035258623 bytes)
Target (PebbleDB) data: 19.51 GiB (20945019250 bytes)
Size ratio: 83.7 %
Key Takeaways
- ~218M keys migrated in ~4 minutes
- Sustained throughput of ~160 MB/s
- Conversion speed dominated by
application.db - PebbleDB size reduction: -16.3% (23.32 GiB → 19.51 GiB)
- Zero data loss, 1:1 write/read parity
Test Environment
The benchmark was performed on a machine with an AMD Ryzen 9 8940HX CPU, 32 GiB of DDR5 RAM, and an NVMe disk using the Btrfs file system. The temporary folder was located on the NVMe disk, not in RAM.