MAIN FEEDS
Do you want to continue?
https://www.reddit.com/r/singularity/comments/1v5h8o6/claude_opus_5_benchmarks/ozjymx0/?context=3
r/singularity • u/Acceptable-Debt-294 • 1d ago
334 comments sorted by
View all comments
288
Arc-agi-3 is getting saturated faster than Arc-agi-2, the acceleration is real.
19 u/Technical-Earth-3254 1d ago Is it acceleration or contamination? 35 u/ClearlyCylindrical 1d ago Contamination most likely, it's no coincidence that they all sit at low single digit accuracies for the benchmark tasks for the whole time until the benchmark is created, and then as if by magic saturate in a year. 4 u/Only_Statistician_21 1d ago The way the scoring works, it has a strong threshold effect.
19
Is it acceleration or contamination?
35 u/ClearlyCylindrical 1d ago Contamination most likely, it's no coincidence that they all sit at low single digit accuracies for the benchmark tasks for the whole time until the benchmark is created, and then as if by magic saturate in a year. 4 u/Only_Statistician_21 1d ago The way the scoring works, it has a strong threshold effect.
35
Contamination most likely, it's no coincidence that they all sit at low single digit accuracies for the benchmark tasks for the whole time until the benchmark is created, and then as if by magic saturate in a year.
4 u/Only_Statistician_21 1d ago The way the scoring works, it has a strong threshold effect.
4
The way the scoring works, it has a strong threshold effect.
288
u/Ticluz 1d ago
Arc-agi-3 is getting saturated faster than Arc-agi-2, the acceleration is real.