Extending and Registering New Benchmarks

This guide shows how to add a new benchmark algorithm and make it discoverable by PGSI.

Option A: Create a Full Project Scaffold

Generate a benchmark project skeleton:

pgsi-analyzer startproject my-benchmarks --algorithms all

This creates a ready structure and registry file.

Option B: Add One Benchmark to Existing Project

Create and register a benchmark in an existing benchmark project:

pgsi-analyzer create benchmark \
  --name my_algo \
  --benchmarks-dir my-benchmarks

This command:

creates method folders (cpython, pypy, cython, ctypes, py_compile),
adds starter files (main.py and method-specific stubs),
updates pgsi_registry.json with the new benchmark entry.

Implement the Workload

Edit the generated files, especially:

my-benchmarks/my_algo/cpython/main.py
and equivalent method directories as needed.

Keep the measurement decorators so PGSI can collect outputs:

@measure_energy_to_csv(n=get_measurement_runs("my_algo"), csv_filename="my_algo_cpython")
def run_energy_benchmark():
    run_workload()

@measure_time_to_csv(n=get_measurement_runs("my_algo"), csv_filename="my_algo_cpython")
def run_time_benchmark():
    run_workload()

Validate Discovery

pgsi-analyzer benchmark list --algorithms --benchmarks-dir my-benchmarks

You should see my_algo in the output.

Run Only the New Benchmark

pgsi-analyzer benchmark run \
  --algorithms my_algo \
  --methods cpython \
  --runs 5 \
  --benchmarks-dir my-benchmarks \
  --output results_my_algo

Best Practices

Keep workloads deterministic where possible.
Start with cpython baseline, then add other methods.
Add enough runs to reduce random variance.
Keep input sizes representative of your real workload category.