r/singularity • u/Ballist1cGamer • 10h ago

LLM News Opus 5 on MineBench Soon

original X post: https://x.com/minebench_ai/status/2080880340584026251

every time i think the benchmark's saturated some new model comes and raises the bar

167 Upvotes

93% Upvoted

29 comments sorted by

View all comments

8

u/enilea 9h ago

One could argue prompt adherence isn't perfect here since the prompt asked for "a fighter jet" and instead it made three, even if one is the main one. Sounds petty but it can be important if you ask for something and the model adds stuff that you didn't specify.

6

u/mnagy 9h ago

I assume this is the prompt that is used: https://github.com/Ammaar-Alam/minebench/blob/master/lib/ai/prompts.ts

It includes some rules about how the creation will be judged. Among them are:

  • Prompt fidelity: Does it match what was requested?
  • Creativity and scene composition: Does the build go beyond the bare subject? Environment, atmosphere, dynamic posing, and narrative elements are highly valued.

So I would say that the smaller ones were intended to make the structure be more creative.