WHAT IT DOES

What does it do?

StarSkirmish Bench gives each language model 1 hour to write a Protoss bot for StarCraft: Brood War in C++. It measures the finished bot’s win rate against competitive human-written bots and demo bots. v0.2 compares results from 5 runs per model, variation between runs, and run costs.

KEY FEATURES

Key features

• Gives each model 1 hour to write a bot
• Measures win rates by pitting C++ Protoss bots against opposing bots
• Compares average win rates across 5 runs per model
• Compares performance variation between runs and run costs