preseason
An open-source benchmark measuring which development tools LLMs recommend for real web app questions
SITE CAPTUREBased on information published on the service's website. Selection and review policy
WHAT IT DOES
What does it do?
Preseason is an open-source benchmark that measures the development tools LLMs recommend in real web app building questions. It collects responses using fixed prompts and a panel of models, analyzes the recommended tools and services, and publishes rankings and head-to-head comparisons. Its goal is to make the recommendations and methodology reviewable and reproducible.
KEY FEATURES
Key features
- Measure recommendations using fixed web app prompts and a model panel
- Analyze recommended tools and services
- Publish rankings and head-to-head comparisons
- Publish methodology notes
- Keep unknown tool names in a queue for administrator review
EDITOR'S NOTE
Why did it stand out?
By collecting development tool recommendations with fixed prompts and a set of models, it makes AI selection bias possible to examine. It also takes a careful approach to unrecognized tool names by sending them to a review queue instead of guessing.