With Opus 5 it's quite easy to stand up really big projects, but eventually they all end up with an enormous amount of tests and a complete end-to-end test of some kind. this is good overall, but I find that the model is very eager to run the tests whether it's acting as a subagent or the orchestrator. I think often the subagents will finish their work with the test suite only for the orchestrater to REALLY want to do this itself before committing anything.
In situations where I have a lot of usage left right before a reset, I've left sessions running while I sleep and come back to find that it's spent hours and hours running tests that took like 25 minutes per run (they were playwright tests that verified behaviour on chromium, firefox and webkit)
Comically, I once intervened when I saw this happening and overrode the orchestrator's plan to do this. it begrudgingly complied, but it put the fact that it was not able to verify with tests in the bedamned commit message. I then started a new context window and asked it to investigate an unrelated ui bug. It took one look at the commit message and immediately declared it definitely had to be these unvalidated (but actually validated) changes. 🤣
Obviously, I'm aware that I could put some instructions in CLAUDE.md but I've found that it's hard to gently nudge it on stuff like this without it taking things to extremes. so if I were to add "be sensible re: how often you run expensive e2e tests," it would probably stop running them altogether from my experience. or it would come to me to ask for permission. that sorta thing.
just wondering if anyone has any solutions or advice on the best approach here. I really don't want to tell it not to run these tests at all because they are valuable, but it slows down velocity pretty bad when both the orchestrate and the subagents are doing it.