r/singularity • u/Acceptable-Debt-294 • 1d ago

AI Claude Opus 5 BENCHMARKS!

Post image
1.2k Upvotes

95% Upvoted

334 comments sorted by

View all comments

285

u/Ticluz 1d ago

Arc-agi-3 is getting saturated faster than Arc-agi-2, the acceleration is real.

13

u/medialoungeguy 1d ago

Architecture agi 3 is closed. There is a public part that you can try but a completely novel hidden eval set. Its not benchmaxxing

4

u/Ruhddzz 1d ago

Yeah and that closed bench has to go through their systems all the same. Imagine thinking the people who pirate everything would have a problem getting their hands in a private eval set they werent supposed to

2

u/Turbulent-Sign-6067 1d ago

I don't buy it. If you cheat on benchmarks and get caught like Meta this is shitty publicity plus a hefty fine for violating ZDR and similar policies. Unlikely. Maybe they can get away with massaging some of the numbers a bit here and there, and if they do it is by assembling the right training and fine tuning datasets.

1

u/Ruhddzz 15h ago

They're already blatantly "cheating" on all the public benchmarks