Parsimony
An open-source benchmark that compares how much code AI coding models add, measured in syntactic units.
SITE CAPTUREBased on information published on the service's website. Selection and review policy
WHAT IT DOES
What does it do?
Parsimony parses Python files before and after a patch and counts syntactic units such as statements, calls, and comparisons. It subtracts deleted units from added units and ranks models by their average net increase on solved tasks. It can also open result files from other benchmarks and display them in the same charts and tables.
KEY FEATURES
Key features
- Parses Python code before and after a patch to count syntactic units
- Subtracts deleted units from added units to calculate the net increase
- Ranks models by average net increase on solved tasks
- Displays result files from other benchmarks as charts and tables in the browser