FrontierHarness Eval
An evaluation service comparing quality, cost, and speed across agent harnesses using the same model
SITE CAPTUREBased on information published on the service's website. Selection and review policy
WHAT IT DOES
What does it do?
It tests 12 configurations of 9 harnesses on the same software engineering tasks, model, and runtime. Each of 360 trials began from the same newly restored checkpoint to reduce warm-cache bias. Users can also follow the repository's manual procedure to install and test their own harness and generate a report.
KEY FEATURES
Key features
- Comparison of median cost per successful task by harness
- Measurement of cost per task and time per successful task
- Comparison of cache hit rates for successful tasks
- Repeated evaluations under the same checkpoint and resource conditions
- Testing of a user's own harness and report generation