Skip to content

rework benchmarks: script and output format

STEVAN Antoine requested to merge benchmarks into main

this is an attempt at making benchmarks easier to work with

  • benchmarks run will read benchmarks from NUON data and run them
  • benchmarks plot will plot benchmark results
  • the output format will be something
    • a directory whose name is the hash of the CPU spec and the Komodo commit hash
    • contains cpu.json with the CPU info
    • contains komodo.txt with the Komodo commit hash
    • contains NDJSON result files

💡 Note

results will typically be uploaded to https://gitlab.isae-supaero.fr/dragoon/komodo-benchmark-results

💡 Note

this MR goes alongside the <code data-sourcepos="16:32-16:63">komodo-benchmark-results@restart</code> branch

changelog

  • bump Nushell to 0.101.0
    • parallel $in => remove useless let input = $in when possible, e.g. still required when using the function's $in in a for loop
    • group-by changed => group-by x --to-table will now produce a table with columns x and items instead of group and items as in 0.95.0
  • add link to results repo
  • the Nushell benchmarks lib
    • rename --force to --no-confirm (-y)
    • add --append
    • reject columns that GPLT will complain about, e.g. $.points.k for the FRI plots
    • add --save to the FRI plot
  • move the "field" and "curve group" benchmarks from benchmarks/src/bin/operations/ to benchmarks/src/bin/
  • remove benchmarks/params/fri.nu because it's been uniformized with the other methods
  • rewrite the README
  • add main function to benchmarks/ that runs the benchmarks from a NUON record specification
  • simplify the output of FRI run

TODO

  • fix "atomic operations" (done in 4f69a1d6)
  • check that plotting still works

images

Field

complex_curve_group_operations complex_field_operations simple_curve_group_operations simple_field_operations

Linear algebra

linalg-inverse linalg-mul linalg-transpose

FEC

encoding decoding recoding combined ratio

ZK

setup commit

FRI

commits_single commits_single_normalized end_to_end evaluating proofs proofs_normalized proving verifying_single

Edited by STEVAN Antoine

Merge request reports