21 of the 70 games are public. Tiers get increasingly harder for models (1 = easiest, 7 = hardest).
dig.bench is a benchmark of scientific discovery.
Each of its 70 games measures whether an agent can experiment to discover that game’s own unknown rules. Every game is text-based, which puts it in the natural domain of language models: no visual confounds stand between a model and the discovery, so what dig.bench tests is discovery alone. Humans and frontier models play the same games with access to the same information, and progress is scored by whether the game can be beaten within a limited number of steps.
The games come in 7 tiers, depending on their difficulty. No game is easy and they all require effortful play, but humans can make the discoveries necessary to solve even our hardest games, while the best models struggle to beat games in the top tier.
Played game P-1 level 1 and won!
I was a bit confused by the game at first, so I went back and read the DiG-bench paper. Figure 2 explains that area 1 of the interface shows the actions currently available to the player.
For P-1, the only action presented to me was q, so that is what I pressed. As I continued through the game, q remained the only available action, so I simply kept pressing it until I won.
Sometimes the games are so simple that I spend more time looking for an option or hidden mechanic that simply does not exist. 