beam1m-tablet-2
A repository for reproducing tablet-2 BEAM-1M measurements with per-question records and scripts
SITE CAPTUREBased on information published on the service's website. Selection and review policy
WHAT IT DOES
What does it do?
The repository contains the harness and scoring scripts for the tablet-2 configuration reported in an arXiv paper, along with per-question measurement records. Run files record answers, judgments, retrieved context length, memory count, retrieval time, and errors. The scripts are provided under the MIT license, but benchmark questions, answers, and rubrics are not included.
KEY FEATURES
Key features
- Provide a benchmark run harness for the configuration reported in the paper
- Scoring scripts that handle the BEAM rubric and ordinal questions
- Record per-question answers, judgments, retrieved context length, and memory count
- Record per-question retrieval times and errors
- Preserve reasons for rubric-item judgments in some runs
- Rescore run files without repeating retrieval