fjmatrix
A benchmark for evaluating flight search intent parsing with real queries in three languages
SITE CAPTUREBased on information published on the service's website. Selection and review policy
WHAT IT DOES
What does it do?
It evaluates search intent parsing using real flight search queries written in three languages. The queries have been deduplicated, de-identified, and manually labeled, and all models are evaluated under the same conditions. Eleven independent evaluators judge whether each model captured the traveler's intent.
KEY FEATURES
Key features
- Uses real flight search queries in three languages
- Deduplicates and de-identifies queries
- Provides manually labeled evaluation data
- Applies the same evaluation conditions to every model
- Uses judgments from eleven independent evaluators
- Shows pass rates by dimension and failure cases by tag