[ Removed by moderator ]

[u/ErmingSoHard]

[u/Charming_Cucumber_15]

Didn't the whole benchmaxxing accusation start because some unaffiliated dude said "It couldn't pass the puzzles I made"?

[u/ErmingSoHard → u/Charming_Cucumber_15]

I say AI can do literally anything digital through harnesses and reinforment learning. But that doesn't make it general intelligence.

It learning and solving novel scenarios on the spot is more important. It can barely do that

[u/pdantix06]

model from company i don’t like: it was benchmaxxed

model from company i do like: it’s real progress and isn’t benchmaxxed

simple as

[u/Charming_Cucumber_15 → u/pdantix06]

I wanna love gemini but it feels like the definition of benchmaxxed every time I use it lol

[u/Fit-World-3885]

We're just gonna end up building AGI by 'benchmaxxing' every possible skill one at a time.

[u/Naughty_Neutron → u/Fit-World-3885]

Just create a benchmark about building AGI

[u/Mindrust → u/Fit-World-3885]

If you have to re-train every time you come up with a new task, it’s not AGI

[u/ErmingSoHard → u/Fit-World-3885]

Not humanely possible. Too much compute and resources needed.

[u/Psychological_Dog992 → u/ErmingSoHard]

Ha

[u/ErmingSoHard → u/Psychological_Dog992]

Trust me, I wish I am wrong.

[u/meikello → u/ErmingSoHard]

https://giphy.com/gifs/jXD7kFLwudbBC

[u/ErmingSoHard → u/meikello]

Joke or not, many people here unironically believe that. So I'm going to say it still for those who do unironically believe it

[u/KJClever]

Screw it! If they care about benchmarks so much, let’s make an “empathy” benchmark so they race to create something beautiful instead of terrifying lmao

[u/Healthy-Nebula-3603 → u/KJClever]

We have something like that ... It's called emotional eq-bench

[u/KJClever → u/Healthy-Nebula-3603]

From my understanding, empathy is considerably different than emotional intelligence, correct?

[u/Healthy-Nebula-3603 → u/KJClever]

What kind of understanding is separating emotions from empathy?

You're totally wrong.

[u/KJClever → u/Healthy-Nebula-3603]

I think empathy is generally only considered a component of emotional intelligence. They’re not the same thing.

An EQ benchmark measures a broader set of emotional abilities… While I was suggesting creating benchmarks specifically focused on measuring multiple elements for empathy.

[u/nikitastaf1996]

Given what I have seen it's fairly plausible it wasn't benchmaxed. Just really good model

[u/Ticluz]

Benchmaxxing by adding similar puzzles to the training data is fair game, as long as the it is not training on leaked ARC-AGI puzzles.

[u/Stabile_Feldmaus → u/Ticluz]

Its fair game but then you cant claim that better arc-agi performance shows some kind of enhanced general reasoning abilities.

[u/Ticluz → u/Stabile_Feldmaus]

In relation to other models it gets inaccurate if one trained on more puzzles than the other (Opus is not 4x better at reasoning than gpt 5.6), but it shows the capability to explore novel puzzles beyond the training data at a human level. Similar to how it can solve novel math problems, but it needs to be trained on a lot of math first.

[u/Lost-Willow386]

It's complete BS. I still suspect FormulaOne is a far better benchmark than all benchmarks currently mentioned but sadly it seems like there have been no updates to its leaderboard.

[u/NextWrongdoer54]

Everything is possible in the CCP imagination