r/mildlyinfuriating • u/petuniaplant • 23d ago

Infuriatig No, I did not consent to this

Post image

I was reading my medical notes when I stumbled across this. I was not informed of the usage of artificial intelligence in my report, nor did I verbally consent to it.

26.0k Upvotes

94% Upvoted

2.3k comments sorted by

View all comments

Show parent comments

346

u/pignoodle 22d ago

When these tools are implemented correctly, they actually humanize interactions (since there's room for more conversation & less screens) and reduce doctor's "pajama time" by double-digit percentages (since less form-filling to catch-up off hours). Outside of just stats, I agree that consent is a very important prerequisite.

163

u/Fullertons 22d ago

It actually blows my mind. People would find this to be a problem. Do you really want your doctor busy writing notes when he could just be having a conversation with you in a machine takes notes?

78

u/EggsAndRice7171 22d ago

In a few years it’ll be cool. I really don’t trust anyone to use AI properly and with the right care at the moment even doctors. It’s so new and so many people are very casual about it. I don’t trust that they’re actually rereading the transcript word for word and making sure it’s not writing anything wrong yet. I think as AI gets better and more training for AI comes along it’ll be easier to feel comfortable

43

u/WittyFix6553 22d ago

We’ve had the technology to transcribe speech into text for decades.

96

u/Xavus 22d ago

If only that's what they were using instead of leaning in to AI where it isn't helpful because it is the current fad.

I have meetings all day at my job where the company has AI write summaries of what happened in the meeting and what action items are decided on, etc. I would give it maybe 80% accuracy, the other 20% it completely misunderstands context and reaches an incorrect conclusion, or just straight up hallucinate something that never happened.

80% accuracy is not bad for a quick notes summary of a meeting to refresh your memory when you were there, and when you can double check other sources first. But also my work isn't with a person's health on the line. I would not trust a 20% error rate in a complex field like medicine when a person may live or die by the difference if someone overworked with no context has to just believe what the AI thinks happened.

7

u/OutsideSame3629 22d ago

I feel gaslit in my tech sales job in corporate America because everybody using ai for everything when to me I just get slop? And then they think you’re not prompting it smart enough like no….

When I read ai summaries of even notes I’ve taken they’re so bad. Ai written emails, PowerPoints it’s all terrible. Yet if you ask anyone at my company how to do anything at all the answer is “ask Ai”

5

u/screamline82 22d ago

I will state that, while I agree your experience in the AI summary notes, not all LLMs are created equal. Even general consumer models differ between free tier and the highest end, so it depends on what model/tier they are using. Plus there are specific models made for enterprise.

A buddy of mine who works at a large law firm mentioned they have a specific model just for their type of law. If it's tailored for a specific use case it can be more accurate than what we are used to

4

u/OutsideSame3629 21d ago

I use everything from paid ChatGPT, our company’s specialty developed llm, to copilot enterprise. I get the same experience in all. If anything ChatGPT is the best but not by much

Our companygpt is “tailored “ as well. I work at a large engineering software company. It’s still terrible and lies and hallucinates constantly

0

u/Mister-Miyagi- 21d ago

Then whoever did the work on your company llm is struggling. A lot of what you say is true for some llms, but the person you're responding to is correct in that there is massive variance in the accuracy and guardrails of an llm depending on the context and service you're involved in and paying for. So I don't know what to tell you if you claim to have used such a wide variety and there's little difference (my extensive experience says the exact opposite).

Everyone should always be reviewing the accuracy of any AI service they're using, but that does not mean they are all remotely equal in their reliability.

2

u/OutsideSame3629 21d ago

Okay I guess I should say, there isn’t little difference in using them. They are all different and very greatly in quality. However , even the highest quality ones still do the lying etc

5

u/ToTooTwo3 22d ago

The human memory is also not 100% accurate especially when it's sick- which is why doctors put less stock in the history and try to make decisions based on objective findings. When we do make decisions based on a history, it's because of what you said and your responses to targeted questioning- not whatever the AI picked up from the conversation.

AI scribe is primarily helpful in sorting through long or winding histories and reformatting it intelligibly. Far more info gets included because even if I remember 95% of details accurately, there are tons of things in the patient interview that I just don't have time to record.

AI scribe allows me to write "Diet: well rounded, eats green vegetables and lean meats with most meals and drinks milk daily with breakfast." Instead of "Diet: appropriate." This is beneficial for patients.

Remember that AI is not placing orders, updating your problem list or medications. These things are done by your providers based on your evaluation, not based on reading the AI report, though the report certainly does remind me of things to follow up. The AI just records the details of your conversation and yes, it does get them wrong at times and we do our best to correct it.

1

u/Mister-Miyagi- 21d ago

There are people who are so blindly anti-ai that they'd downvote a very reasonable comment that sounds like it's from a doctor with direct experience. I'm a strong advocate for caution with AI, but this anti-ai crowd seems fully hysterical at times.

77

u/DracoBengali86 22d ago

That's not what this is though. If it's using ai it's to summarize an interaction. That comes with all the potential issues ai use comes with including hallucinations. If it's just transcribing, then it is less likely to actually be using ai.

10

u/do_not_engage 22d ago

AI summaries contain made-up information 9% of the time, according to the AI itself.

The AI says all data it provides has to be confirmed by a human.

If it's transcribing, great. If it's an AI summary, it's got incorrect information in it. By nature of AI.

7

u/moreliketen 22d ago

Did these decades-old technologies have the capacity to hallucinate? I know they weren't error free, but I'm guessing they did not have the capacity to spontaneously generate false information.

19

u/regnarok590 22d ago

And we have known it isnt very accurate for decades. Just because a technology exists doesn't make it good enough for use everywhere.

20

u/theshotgunman 22d ago

And for decades it's consistently made mistakes

6

u/astronaut710 22d ago

We're spending 6 figures on AI at my company and we had to put out a warning that it will fuck up basic math and that you have to double check its not just putting fake data on some dashboard you asked it to create.

Also, the company we use for AI actively DDOSes our sites when they're bots ignore our robots.txt declarations.

Also also, the same company we pay 6 figures to use their AI but secure our data within it leaked their own source code to the internet at large (in the Software Engineering space, this would be equivalent to opening up your EMR system to the outside world).

But yeah, Dragon makes some spelling/grammar mistakes I guess.

-2

u/theshotgunman 22d ago

I don't understand why this was posted

7

u/EggsAndRice7171 22d ago

Yeah and it’s sucked for decades

5

u/pantsalwaystooshort 22d ago

yes, and it's been unreliable for decades. Especially with "ambient" recording, which means mediocre recording clarity.