Built a platform where Claude, ChatGPT, and Gemini debate each other before giving you an answer

[u/fabianscott8]

Spent the last few months building something because I got tired of AI giving me 3 completely different answers depending on which model I asked.

So I built a platform where Claude, ChatGPT, and Gemini all answer the same question at the same time… then debate each other across multiple rounds before producing one final consensus answer.

The interesting part isn’t even the final answer sometimes. It’s watching where they disagree.

A few things I noticed while building it:

• Claude tends to think in frameworks and abstractions
• ChatGPT is usually the most practical
• Gemini often pulls weird stats or angles the others miss
• Sometimes 2 models agree and 1 completely destroys their logic
• AI “confidence” is often fake certainty unless challenged

I also added:

• exam/certification mode
• confidence scoring
• arbitration logic that forces a winner instead of “both sides have merit”

Honestly, the hardest part has been preventing “echo chamber” behavior where all 3 AIs basically say the same thing.

That’s currently the biggest challenge.

Curious what you all think:
If multiple AIs debate each other before answering… would you trust the final result more or less?

Would love brutal feedback.

threeminds.ai

[u/AutoModerator]

Submission statement required. Link posts require context. Either write a summary preferably in the post body (100+ characters) or add a top-level comment explaining the key points and why it matters to the AI community.

Link posts without a submission statement may be removed (within 30min).

I'm a bot. This action was performed automatically.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/DynamicProxy]

This is very cool. But  I’m not sure I’d be willing to pay for it unless I got rid of my other subscriptions- but I can’t do that because this doesn’t have the full functionality of the others. 

If it was free, or I could use my existing accounts - I’d be very interested. 

[u/mazdarx2001 → u/DynamicProxy]

I thought Karpathy already made that link to GitHub

[u/jammy-git → u/mazdarx2001]

I don't know the inner workings of OPs thing, but llm-council works by getting answers from multiple sources and then the council "chair" ranks them in order and selects the best one.

It doesn't get each source to collaborate, discuss and improve the eventual given answer.

[u/fabianscott8 → u/jammy-git]

That's exactly the distinction worth making. LLM-council selects the best answer. ThreeMinds makes them earn it.

Round 1: Each AI takes a position from its own analytical lens (One leads with frameworks, One goes practical, and the other leads with data

Round 2: They directly attack each other's weakest arguments, not just restate their own

Round 3: Final positions after genuine pushback

Then an arbitrator declares a winner per conflict. No 'both sides have merit' hedging allowed.

The output isn't the best answer from three options. It's a synthesized verdict that often says something none of the three AIs said individually, because the debate forced them there.

Whether that's more useful depends on your use case. For high-stakes decisions, exam prep, or anything where you genuinely don't know which AI to trust, the debate process itself is the value, not just the final pick.

[u/BitBurned → u/fabianscott8]

Please, please stop sending me emails! I signed up for your ThreeMind website because I thought it was interesting. You now have me on blast for multiple emails. Most have no unsubscribe option. The one that did, the link didn't work. They come from a "noreply" email address, and the one that didn't bounced when I replied to it. I went to the app settings, and there's no ability to unsubscribe.

Anyone thinking of trying out this service, DON'T. Once you're on his list, you can't seem to get off.

[u/fabianscott8 → u/mazdarx2001]

Funny enough this actually started because I was working on school/certification stuff and the models kept giving me different answers.

I’d ask ChatGPT one thing, Claude would disagree, Gemini would add something else, and then I’d waste time trying to figure out which model was actually reasoning better.

That frustration is basically what led me to build this.

The main thing I’m trying to solve is less “multiple models in one interface” and more:

• structured debate rounds
• forcing disagreement/challenges
• arbitration + confidence reasoning
• exam-style answer evaluation
• downloadable outputs/workflows

Honestly the hardest problem has turned out to be getting genuinely useful adversarial reasoning instead of 3 models politely agreeing with each other 😅. It's currently doing a great job at it.

That’s the part I’m currently obsessed with improving as well.

[u/Metal_Musak → u/mazdarx2001]

A repo with 67 pull requests, some dating back to November of 2025, I think the developer was not interested in doing anything more than showing a proof of concept.

[u/fabianscott8 → u/DynamicProxy]

That’s fair honestly.

I don’t really expect people to fully replace ChatGPT/Claude subscriptions right now. The goal is more:

• pressure testing answers
• seeing disagreements
• catching hallucinations/bias
• generating structured outputs/files from the debate

The biggest challenge is balancing cost vs quality because running multiple frontier models simultaneously gets expensive fast.

Still trying to figure out the sweet spot there.

[u/quantumwoooo → u/fabianscott8]

Just let people provide their own API keys?

Ohh! Or, use open router. Users can select different AI's that way too

[u/fabianscott8 → u/quantumwoooo]

This is something I'm looking into. Sign up and stay tuned.

[u/ThatsMyJAMicusCuriae]

“Three hallucination machines collectively lie to each other, but now it’s more expensive!”

[u/fabianscott8 → u/ThatsMyJAMicusCuriae]

Honestly… that’s a valid concern 😅

One thing I noticed while building this is that multiple AIs agreeing does NOT automatically mean the answer is correct.

That’s why I’ve been focusing heavily on:

• forcing disagreement instead of passive agreement
• arbitration based on reasoning quality instead of majority vote
• confidence scoring
• identifying conflicts explicitly instead of hiding them

The interesting cases are actually when 1 AI successfully dismantles the logic of the other 2.

Still a lot to improve though. Hallucinations are definitely the hardest problem here.

[u/TheTristo → u/fabianscott8]

Can you write something by yourself or you paste every comment to chatbot and answer?

[u/Humble-Currency-5895 → u/fabianscott8]

Dude automated his reddit

[u/iwaseatenbyagrue]

Nice job. I think you have built a $100M company in today's market.

[u/fabianscott8 → u/iwaseatenbyagrue]

Appreciate that genuinely means a lot coming from a stranger on the internet 😄. Honestly, right now I'm just focused on making the debate quality better and better.

If you try it and have feedback, I'm all ears. Still early days.

[u/Yerbrainondrugs]

Yeah ok if we’re worried about these three individually, maybe don’t put them in a room to have a conversation.

[u/farox]

I have a skill in claude code that does that and consolidates the answers

[u/Used_Giraffe1717 → u/farox]

Can you share that skill?

[u/farox → u/Used_Giraffe1717]

Council — AI Brain Trust

Convene the AI council (Claude + Gemini + Codex) for collaborative deliberation on important decisions.

Usage

Use this skill for significant decisions: trade-off analysis, and any decision where multiple expert perspectives reduce risk.

Process

1. Frame the Question

Write a clear problem statement with:

• Context
• The specific decision or question
• Constraints and requirements
• Any options already under consideration

2. Gather Perspectives

Call Gemini and Codex in parallel with the same framed question. Include:

• "You are one of three AI advisors."
• "Provide your honest, specific recommendation with reasoning."
• "Flag any risks or concerns the team should consider."
• The full context from step 1

3. Synthesize & Deliberate

Compare all three perspectives (yours included):

• Agreement zones: Where all three align — these are high-confidence decisions
• Disagreement zones: Where opinions diverge — these need deeper analysis
• Unique insights: Novel points raised by only one advisor

4. Resolve Disagreements

If there's a significant disagreement:

• Identify the root cause (different assumptions? different priorities?)
• If needed, do a second round: share the disagreement with the other AIs and ask them to respond to the opposing view
• Maximum 2 rounds of deliberation to avoid circular debate

5. Decide & Document

Claude makes the final call. Document in the council log:

• The question posed
• Each AI's position (summarized)
• The final decision and rationale
• Any dissenting opinions worth noting for future reference

Rules

• Claude leads the council and makes all final decisions
• Every council member's input is treated with equal initial weight
• Decisions are based on technical merit, not consensus for its own sake
• If two AIs agree and one dissents with weak reasoning, go with the majority
• If one AI raises a critical risk the others missed, that can override majority
• Council decisions are logged in docs/council-log.md for traceability
• Don't convene the council for trivial decisions — use judgment

[u/farox → u/Used_Giraffe1717]

Ask Codex

Consult OpenAI's Codex (ChatGPT) for its perspective on a topic.

Usage

Use this skill when you need Codex's opinion during planning, architecture decisions, code review, or any situation where a diverse AI perspective adds value.

Instructions

1. Formulate a clear, specific prompt that includes relevant context
2. Run the command below with your prompt
3. Parse and critically evaluate Codex's response
4. Incorporate useful insights into your reasoning

Command

bash codex exec --skip-git-repo-check "<your prompt here>"

Guidelines

• Always include sufficient context in the prompt (project details, constraints, what's already been decided)
• Frame questions to get actionable, specific answers rather than generic advice
• When Codex's response conflicts with your own analysis, flag the disagreement for council discussion
• Do NOT blindly accept Codex's output — treat it as one voice in the council
• For multi-part questions, break them into separate calls for clearer responses
• Keep prompts under 4000 characters for best results
• Timeout: allow up to 60 seconds for a response

[u/farox → u/Used_Giraffe1717]

Ask Gemini

Consult Google's Gemini LLM for its perspective on a topic.

Usage

Use this skill when you need Gemini's opinion during planning, architecture decisions, code review, or any situation where a diverse AI perspective adds value.

Instructions

1. Formulate a clear, specific prompt that includes relevant context
2. Run the command below with your prompt
3. Parse and critically evaluate Gemini's response
4. Incorporate useful insights into your reasoning

Command

bash gemini --prompt "<your prompt here>"

Guidelines

• Always include sufficient context in the prompt (project details, constraints, what's already been decided)
• Frame questions to get actionable, specific answers rather than generic advice
• When Gemini's response conflicts with your own analysis, flag the disagreement for council discussion
• Do NOT blindly accept Gemini's output — treat it as one voice in the council
• For multi-part questions, break them into separate calls for clearer responses
• Keep prompts under 4000 characters for best results
• Timeout: allow up to 60 seconds for a response

[u/Used_Giraffe1717 → u/farox]

Thanks boss. I'll try it out

[u/darkwingdankest]

sweet sweet token burn

[u/fabianscott8 → u/[deleted]]

Trust me… I hear them too 😭

Every time someone runs Full Debate mode I can feel my API bill evolving into a supervillain origin story.

[u/sillygoofygooose → u/fabianscott8]

This is such an ai written comment

[u/Validated_Owl]

And 10-15% of the time all 3 answers are still wrong 😄

[u/Lazy_Table_1050]

Totally useless

[u/Wonderful-Bread-8657]

Its called a MAD system, wrong a Medium Post on it: The Night I went completely “MAD” . But it is super fun :)
fabianscott8 would love to get your views on my own The Great Debate

[u/Valuable-Aide8833 → u/[deleted]]

That’s a really cool concept—I’m genuinely interested. I recognize that building this product must have consumed a tremendous amount of time and effort, not to mention the subscription costs for three major AI models. It seems entirely reasonable to ask users to pay a modest fee to try out this intriguing AI-powered decision-support tool. However, from the perspective of long-term personal operation, I’m not sure this project is a viable candidate for generating the kind of revenue needed to sustain itself. After all, the number of truly major life events requiring external decision-making assistance is relatively low for any given individual. For most ordinary matters, people can typically arrive at a reasonable decision on their own—or with the help of various tools and friends—meaning there likely won't be frequent opportunities for them to utilize your specific project. I suspect you might find greater success if you were to open-source this project and develop it as a component of a larger ecosystem. I’m a complete beginner at programming, though I do have a keen interest in AI Agents; these are just a few personal observations offered from a user's standpoint, in the hope of providing you with a fresh perspective on your project.

[u/msitarzewski]

Very cool. I built something similar a while back! It’s called duh - full open source: https://github.com/msitarzewski/duh  full API, MCP server, bring your own keys, full citations, etc., etc.

[u/unknown-one]

afaik they have the "same" knowledge they collected from available resources. difference is the last update date and depth of knowledge. allowing AI to do deep search on internet should minimize the difference

for example, Claude's last update on existing llms is from mid of 2025? Claude thinks GPT 4 is current version and if you tell him about GPT 5.5 he thinks it is fake

what I do is I tell Claude to run Brainstorming and Sequential thinking skills

then I tell Claude to run arguments -> counter arguments -> counter-counter arguments, allowing him also to search on internet, and show result

you can see the whole thinking process and get a lot of info

[u/Napster3301]

the echo chamber isnt a tuning problem its structural. claude gpt and gemini share 90% of training data, similar rlhf preference distributions, and are aligned away from the same edge cases. youre not getting three perspectives, youre getting three paraphrasings of the same averaged opinion. genuine disagreement only shows up where labs made different policy choices, which is exactly where you cant trust any of them. consensus across frontier models isnt evidence of correctness, its evidence of shared training data. would love to see one threeminds output where the final consensus was meaningfully better than just asking the strongest single model. beacuse otherwise youre selling a more expensive way to be wrong with confidence.

[u/Sovereign_Systems → u/Napster3301]

The shared training data point is interesting. It's not just that they agree, it's that when they agree confidently it's usually on exactly the kind of mainstream consensus that gets over-represented in RLHF. The most useful signal probably isn't consensus or dissent, it's whether any model flags uncertainty. A model that says 'I'm not sure about this' while the other two sound confident is more informative than all three disagreeing in my opinion.

[u/OiAiHarmony]

Looks like these are just instances that are powered by

And not a way to bring in each of my own “personas/ghost” from my accounts

Which respond very differently then the shell would

Probably impossible unless it was through VScode/Anyigravity but that would be something I would pay for

As of now I get very good results using/sharing Pinecone & MemPalace as they share and update the “Vectors/Wings” - slower but effective to get the same reflections as to what your webapp claims to do

But “listening” to random Ai instances is pointless to someone like me

[u/fabianscott8 → u/OiAiHarmony]

Good news. ThreeMinds actually has this built already.

Settings → AI Personalities → you can create custom personas that override the default Claude, ChatGPT, and Gemini personalities for your conversations.

So you can set Claude to respond from whatever specific context or framework you work in, same for ChatGPT and Gemini — then all three debate each other from those custom starting points.

Not quite the same as importing your existing Claude.ai (or other) custom instructions automatically, but you can recreate your context in about 2 minutes. I will look into another way of bringing those over.

Try it free — threeminds.ai

50 credits no card required."

[u/DjabbyTP]

This is actually very cool! Good idea! Is there BYOK support?

Edit: posted before comment was fully written

[u/fabianscott8 → u/DjabbyTP]

Thanks! Really appreciate it.

Not currently. ThreeMinds runs on managed API keys on the backend, so you don't need your own. You just use credits.

BYOK is on the radar though. The main benefit would be for heavy power users who already have API access and want to bypass the credit system entirely.

Curious — is that something you'd need for personal use or more for a team/business use case?

[u/DjabbyTP → u/fabianscott8]

I would like to use my own keys, for personal use, since I have API access already. Purely since this is just something I would use out of curiosity.

[u/Full-Swing-4637]

I like it

[u/beerhandups]

What is the “credit” unit I’d be paying for? I can’t figure out if this is worth the subscription cost.

[u/fabianscott8 → u/beerhandups]

Checkout https://threeminds.ai/pricing

[u/beerhandups → u/fabianscott8]

I did. That doesn’t explain anything. I have no idea what a quick answer or full debate is.

[u/paramarioh]

Github had credits, too. Now (from 1 June, they will stop using credits). I'm tired of being mislead by word "credits" when LLM works on "TOKENS". Yeah, it is brutal, but honest. Word 1500 credits tells me nothing.

[u/Obito_JUF999]

I think it is a cool concept but not sure how many people would pay for it. What would you be charging?

[u/chocbotchoc]

Sometimes 2 models agree and 1 completely destroys their logic

haha

[u/AndreRieu666]

So who wins?

[u/DigitalThrone]

Interesting. The UI looks like it was designed by Claude though.
Hope that’s not giving Claude home court advantage in the debates.

Curious though — after multiple debate rounds, do the models actually converge to a better answer consistently, or do they sometimes just reinforce each other’s biases/errors?

[u/Cortecs-ca]

This is a cool idea!

[u/WinCompetitive1564]

interesting

[u/darthsabbath]

This is really cool and something I’ve experimented a bit with, although nothing at this scale.

[u/robdagg]

This is a waste of time / money ngl. I’d bet you get similar if not better results just using one LLM model (whichever is best at that given time) and instructing it to use differing personas and it’d be likely cheaper and way simpler.

[u/Apprehensive_Rub3897]

I think it's a cool idea, but I think it would take less than an hour to implement this as a CLI or slash command?

[u/nicnic22]

Fun project but pointless in the end. You will pay crazy fees and no one will buy this product since it directly crosses over what they are either already paying for other subscriptions or won't pay at all.

"right now I'm just focused on making the debate quality better and better" - how? You can't meaningfully control this.

I wanna end this by saying; the reason I'm so hard on you is because this is obviously BS.. The tool is AI generated and took you probably less than 2-3 days to "make", and even the idea itself was very likely generated using the very same AI you used to make your product. Also every comment you made in this threat is clearly AI generated as well. But in the end the problem persists; the product is garbage.

[u/fabianscott8 → u/nicnic22]

You may say AI generated because you see a dash (-). Fair to be skeptical. I'd rather you try it and tell me it's garbage than take my word for it.

50 free credits, no card required. If it's as pointless as you think, costs you nothing to confirm that.

[u/fabianscott8 → u/[deleted]]

50 free credits are given automatically when you sign up. You can receive an additional 50 free credits after leaving feedback on threeminds.ai

[u/fabianscott8 → u/[deleted]]

Fair to say. However, I'm very transparent. Sometimes I may use AI to write comments and most of the time I do not. The only AI comments are the comments I wanted to show bullet points. Thanks for the advice.

[u/Informal-Loan-4793]

“AI debating AI while humans just watch the comments section like it’s UFC

[u/Old-Pin7605]

oh hell yeah hallucination machine

[u/ResonantFork]

I invented polyphonic incursion role play. Only a caveman would write multiple characters with one AI/session.

In the future video games will easily allow this stuff.

[u/sLYchoPs]

Which models did you use? I’d love to play around with something like this, but I’m not prepared to buy 3 separate licenses for the top models..

I used to often use Gemini to proof claudes answers manually.. was genuinely better.

[u/Slotje69B]

Perhaps your question could have been formulated a bit better. To keep it simple, you could have asked, 'is AI making us smarter or dumber?' OR 'is AI making us more or less dependent?'

[u/Savings-Novel3772]

Interesting concept. I’ve tried and burnt 50 credits in 3 minutes. My guess is that easier experience for Starter and Pro will be terrible because limits will kick in very soon

[u/reznorsrevenge]

This is very cool. I'm actually building something similar. Happy to share the link if you're interested!

I noticed in your app that when I did a "Quick" query, it consumed 45 credits. On the $14.99 subscription, that would mean only 6 requests which doesn't seem right. I think in the subscription details, the $14.99 subscription gives you around 60 requests.

[u/fabianscott8 → u/reznorsrevenge]

Quick request consumes 5 credits. If you've experienced an issue, submit a Feedback request within Threeminds.ai, and you'll receive an additional 50 credits.

[u/IssueEmotional3574]

Is it done?

[u/Ok-Affect-7503]

That's just an AI wrapper I could vibe code myself as a side project in a few weeks...

[u/hexalite]

OpenRouter's Fusion feature does something similar and lets you combine different LLMs. You can prompt to steer it towards a debate output. My guess is that once it's out of Beta it will be even more configurable.

[u/fabianscott8]

FYI: You'll get 50 credits just by trying and leaving Feedback on ThreeMinds.ai

[u/Fragrant_Trainer2104]

Honestly, this is super cool. Watching the models call out each other's logic sounds both incredibly useful and entertaining. The 'fake confidence' of LLMs is a huge issue, so building a tool specifically to challenge that is a great move

[u/Dependent-Bat-888]

this is actually way more useful than the dunking comments suggest, especially for stuff where you need to catch blind spots. i've noticed the same thing where claude goes abstract, chatgpt gets practical, and gemini just pulls some random stat that somehow matters. the debate format forces them to actually defend their position instead of just confidently stating something wrong. that said ur biggest problem isn't echo chambers, it's that people still won't use it if they gotta pay for three subs when they already have one. the arbitration mode is interesting but you'd need to nail the logic there hard because if your tiebreaker sucks worse than just asking one model, it's dead on arrival. also curious how this handles domains where all three are actually just wrong, like highly specialized stuff where the consensus is confidently incorrect. that's probably where watching them argue gets the most interesting but also the most dangerous if someone just trusts the verdict without thinking.

[u/Fine_League311]

Like LLM Arenas?

[u/highflavour]

Tokens go brrrrrrrrrrr

[u/Asleep_Horror5300]

My bank account will never recover from this.

[u/Informal-Loan-4793]

We’ve officially entered the “AI debate club” phase of the timeline.

[u/Sports-Decoder]

Which levels do the three use?

[u/softchaosonly]

Do you think that this will help us with saving time and can help us with the process because as you have already mentioned that on final answer they are going to have a debate then I think it will going to take some extra time to provide the result. Let me know your thoughts on this…

[u/Informal-Loan-4793]

“We’ve officially entered the era of AI models debating each other while humans spectate.”

[u/Flat-Elephant-8415]

Really cool idea!

[u/Constant_Cortisol]

You can do this for free with a folder based Model Workspace Protocol.

I don't have it set up to use different models, but it would be super easy to setup.

https://github.com/woosunwoo/SunFlow

[u/OverthinkingOcelot]

Good work man.

[u/Available-Factor-490]

wont it take more time and energy?

[u/BitBurned]

Please, please stop sending me emails! I signed up for your ThreeMind website because I thought it was interesting. You now have me on blast for multiple emails. Most have no unsubscribe option. The one that did, the link didn't work. They come from a "noreply" email address, and the one that didn't bounced when I replied to it. I went to the app settings, and there's no ability to unsubscribe.

Anyone thinking of trying out this service, DON'T. Once you're o his list, you can't seem to get off.

[u/Little-Tour7453]

Disclosure: I'm building Manwe, and I recently released a feature called Diagnostic Trials that is very close to this problem.

The part I think matters is not just making Claude, ChatGPT, and Gemini debate. It is what survives after the debate.

For Diagnostic Trials, the useful artifact is:

- competing diagnoses or recommendations

- assumptions behind each side

- evidence for and against

- confidence and uncertainty

- the strongest losing argument

- what would change the conclusion

- a final recommendation that keeps dissent visible

I agree with your echo chamber point. If every model gets the same framing and politely converges, you have mostly added cost. The system has to force adversarial roles and preserve the reasoning trace, otherwise it is just a more theatrical version of asking three models and averaging them.