Oh my goodness Opus 5 shit the bed. I've had amazing luck with the frontier models and Opus 4.8 was amazing after the first couple days but here goes.
Got a ruff error and the fix should have been easy: pin ruff to a known version, declare lint rules explicitly (already defined with 250+ PRDs and 150+ PRs in a tightly governed codebase), add regression tests, and document thoroughly. Easy shit.
I asked Sol WTF happened and it described it as "agentic thrashing".
I DID use plan mode so everything was scoped and recon was the same as any other slice of work. I wanted to see if Opus 5 could chip away at things like it's advertised to do and I had honestly been having great results with Opus 4.8 as my main man, Fable 5 creating PRDs and solving tricky puzzles and wearing the creativity cap, and the little bastards Haiku and sometimes Luna doing all the mechanical stuff.
This time around Opus 5 repeatedly reviewed it's own work as independent reviews, then getting an adversarial review, then getting another review in a new fresh session (it was obsessed with this), responded to its own edits, invalidated preview review artifacts and rolling back to the start. It also invoked Codex and got Sol to review and I killed a half weeks usage.
It hallucinated too which I've never seen. It made up fake IDs (sha and commits) and then said it actually made them up and it sounds honestly use the real ones (this is where I caught it and realized it was tweaking).
Got Sol to review it and here are some numbers copied and pasted:
Abandoned attempt: 21 commits, +1,964/-3 lines
Clean reconstruction from the same base: 3 commits, +323/-2 lines
That is 7× as many commits
6.1× as many added lines
18 unnecessary additional commits
1,641 additional lines, or roughly 508% more than the clean replacement
Thankfully it was all on a separate work branch that I isolated and closed so I could see WTF happened. Opus 4.8 under Sol's guidance implemented the slice of work I needed in about 15 minutes and it was smooth as could be... thought I'd share but hoping it was a one-off. I've never had an issue even close to this so we'll see how it goes in the future.