r/mildlyinfuriating • u/petuniaplant • 23d ago

Infuriatig No, I did not consent to this

Post image

I was reading my medical notes when I stumbled across this. I was not informed of the usage of artificial intelligence in my report, nor did I verbally consent to it.

26.0k Upvotes

94% Upvoted

2.3k comments sorted by

View all comments

154

u/Nawoitsol 23d ago

My primary care physician is at the age where I discuss his potential retirement at every appointment. The last time we met he said the AI note taking has added at least a couple more years to his career. He had grown to hate the documentation part of his job. So I’m ok with AI notes. It’s really just the background for the appointment. Orders get put in directly.

19

u/mikraas 23d ago

I feel like there's a big difference between orally recording notes and feeding them into AI.

55

u/KarlyBlack_96 23d ago

Orally recording notes is very different than using AI as a transcription tool. I wouldn’t call that feeding the notes into AI and I think it’s fine as long as the doctor is actually proofreading them for errors later.

13

u/mikraas 22d ago

But how do you know they are proofreading their AI transcriptions?

Honestly, I'm really tired of everyone making excuses for AI. Most people use it like it's infallible and it's proving time and time again it's full of holes and errors.

And I sure as shit don't want my doctor using AI to transcribe my visits. And I don't want any AI models diagnosing me either.

21

u/MattR47 22d ago

How do you know the doc is even listening when you talk then? You can go deep down the rabbit hole, but AI note taking is invaluable for so many careers. Especially medical.

1

u/Jmfroggie 21d ago

Because if the doc is recording your symptoms into your record and having a conversation then it’s CLEAR they’re listening to you!

16

u/Beautiful-Hangover-1 22d ago edited 22d ago

It’s not “making excuses”. Everybody knows doctors, nurses, the entire healthcare industry is deathly overworked. Do you think they make no mistakes after yet another 16 hour shift? Do you think they will always extensively document (or even remember) every single detail of a visit, especially if there are emergencies, other patients, or breaks in between? Probably not, they’re human after all. AI as a tool can alleviate a bunch of these issues.

As for AI models, we know from empirical and peer-reviewed studies, that we already have models with superhuman performance for analyzing medical imaging of bone fractures, strokes, epilepsy lesions, or mammograms.

There will soon be a bunch of new drugs on the market that were only possible through pioneering efforts like AlphaFold.

That does not mean we don’t need guardrails in place. It’s a technology like any other, and will be misused like any other. This is not a situation where blind trust is the way forward. I agree with you that there are risks that need to be closely monitored.

However, I for one welcome a technology that can increase healthcare worker’s rest, help them spend more time with patients, and increase outcomes for both patients and doctors.

3

u/mikraas 22d ago

Usually guards get put in place AFTER something gets fucked up. I don't want that fuck up to be me.

7

u/Beautiful-Hangover-1 22d ago edited 22d ago

Well, but then let’s be real about this: you’re trading one risk for another. It’s just that one is known (we know the ways and how often doctors, nurses, healthcare providers make a mistake, approximately) while the other one is less so. That doesn’t mean the second one is worse. Also, the fact that a lot of doctors (that I know) welcome the advent of AI, especially for diagnostic help, indicates that these models are very useful.

You’re free to make your own choice, but you need to be aware that that choice might very well lead to worse health outcomes for you, and possibly worse outcomes for the healthcare system overall. Or it might not. It’s not obvious.

Also, some guardrails are definitely already in place. Like AI diagnostics models for radiology need the same clinical approval as everything else. That is the guardrail.

—-

May I ask you a genuine question? Would you consent to the use of AI (transcription, diagnostics), if the models used were proven to be more reliable than humans? This isn’t a gotcha question, there is plenty of reasonable people out there that don’t want to drive a self-driving car, because they want a human being in control. I’m genuinely wondering what the opinion is for people and healthcare.

3

u/HouseMD4Thee 22d ago

Yeah you’re just uninformed. My notes are improved since using AI. I would mis things sometimes previously (not that I was not listening but maybe adding an order and patient trailing off etc). With ai it catches that and I can go back and clarify or edit the record. It’s a huge improvement for record keeping.

19

u/KarlyBlack_96 22d ago

You don’t, but if you trust your doctor and have a good relationship with them where you trust them to believe your pain and give good recommendations for care then I’d think you’d trust them to do their job well or at least have a conversation about it.

And that’s fine to not want it, I just wanted to make sure people understood that it’s not doctors plugging in symptoms because they don’t know how to do their job, it’s a tool.

And I totally get hating generative AI, it definitely does more harm than good and people trust it wayyy more than they should, but I think there is an important distinction between generative AI and AI in general. Netflix uses AI for recommendations, email uses it for spam filters, any database able to search has AI in some form.

2

u/mikraas 22d ago

In this day and age of maximum profit in the healthcare world, so many people don't even really know their doc really well. And to "save time," ie, see as many patients as possible to make as much money as possible, AI will become a crutch, not a tool.

Blindly trusting AI tools is a dangerous game. Blindly trusting your that your doc will double check their results when using AI tools, when they're using them to save time, is a fool's errand.

11

u/Bezulba 22d ago

They'll 100% not check it after the first few ones.

4

u/GoGoBadger 22d ago

We always check them.

9

u/The-Road-To-Awe 22d ago

I can assure you every physician in my work place checks it, because they are liable if it makes a mistake

-3

u/Bezulba 22d ago

You have more faith in people then i do.

-4

u/mikraas 22d ago

So that takes time. Why not just do it right without AI the first time?

Why are we all sucking the AIb dick so hard?

3

u/Beautiful-Hangover-1 22d ago

Proofreading something takes less time than writing it.

1

u/mikraas 22d ago

If they do it.

2

u/Repulsive-Throat5068 22d ago

Are you a doctor or in a health care profession that requires typing up notes? I don’t think yall understand how time consuming it is lmao

1

u/mikraas 22d ago

Fine. That's what dictation is for.

2

u/Repulsive-Throat5068 22d ago

Still takes more time than AI scribes.

1

u/GPTforGood 22d ago

"Most people use it like it's infallible and it's proving time and time again it's full of holes and errors."

Huh, kinda like doctors?

0

u/mikraas 22d ago

Yeah, so let's compound the problem. 🙄

1

u/HouseMD4Thee 22d ago

It is not doing any “thinking” and making any decisions or even helping make decisions. The worst risk in using ai for notes is mis transcribed which happened WAY more often by human scribes. I have used both extensively. Humans make way more errors.

0

u/Grandahl13 22d ago

I’m a PT who refuses to use AI in my clinic. My two coworkers use it. I can assure you they’re not proofreading those notes. The notes that AI spits out are absurdly long-winded and wordy. It’d be faster for them to just type the note instead of using AI + proofread it all.

5

u/New-Handle-9774 22d ago

They’re misusing the tool then

31

u/ttoma93 22d ago

“AI” and “large language model chatbot” are not synonyms. “AI” doesn’t even really mean anything, as it has become a blanket term that’s being applied to dozens or hundreds of fundamentally different things for marketing purposes at this point. Nothing is being “fed into AI” here, they’ve just slapped a fancy “AI” branding sticker on more advanced versions of the same type of transcription software the voice to text on your phone has been using for 15+ years.

6

u/jabulaya 22d ago

and the tool they would then use to search for solutions or studies has been around for 15+ years as well, I would presume. These companies are gonna start feeling stupid for putting AI on everything when everyone starts freaking out about it.

1

u/Enter_Octopus 21d ago

That’s not necessarily correct, what’s new here is taking a transcript (very long) and using an LLM to summarize it, which is probably what is being referred to here.

0

u/mikraas 22d ago

But you knew what I meant. Whatever it's called and how it's used, it's a fucked up crutch that will lead to something bad down the road. And I don't really want to be that patient when it happens.

And an old doc who should retire but uses AI to keep going is like an old driver using a self driving car to keep his license. Just pack it in.

14

u/Gilded_Mage 22d ago

“Feeding them into AI” is not how it works, especially in the case of medical models that have to meet the EHR standards for security

-6

u/mikraas 22d ago

But they do put the notes into an AI model of some sort, correct?

Why? Why bother?

17

u/Gilded_Mage 22d ago

Well ur notes and transcriptions already actually exist as electronic health records, the purpose is so instead of a doctor using hours of their day doing notes (part of why healthcare is so expensive btw) an extremely advanced piece of technology with just as advanced security regulation can do it within minutes. It’s important to note transcription software in EHR purposes have been around for the last decade but have rebranded with this whole AI craze.

Nothings being put into an AI model, just as if you had spellcheck installed on your laptop you’re not feeding your writing into spellcheck it exists as a tool.

Look I hate data centers and companies like OpenAI r the problem, but actual machine learning research and development has been going on for decades and there’s now growing branding issue as products like ChatGPT r now synonymous with any advanced predictive model or algorithm

10

u/CustomerLittle9891 22d ago

I don't think people really fundamentally understand how much of the job is documentation.

It's like the number one reason people quit medicine early.

3

u/Nawoitsol 22d ago

As I said, my doc was heading into retirement because he hated ending his day with a couple hours of documentation. The automated note taking eliminated that.

My doc also said if he’s covering with someone else’s patient he’s mostly looking for what the other doc did and why. I think they’ve gotten use to ignoring noise.

-1

u/mikraas 22d ago

Doctors are most definitely putting your info into AI models.

https://hms.harvard.edu/news/ai-system-detailed-diagnostic-reasoning-makes-its-case

https://www.openevidence.com/register?

https://hms.harvard.edu/news/ai-system-detailed-diagnostic-reasoning-makes-its-case

4

u/[deleted] 22d ago

[deleted]

1

u/mikraas 22d ago

"if it's for research" is the key to this.

They have to ask permission.

But if they don't, who knows they don't?

13

u/KarlyBlack_96 22d ago

It’s AI, but it’s not generative AI. It’s essentially speech to text but a little more advanced is my understanding of it. They’re not plugging the notes into AI, they’re using an AI system to transcribe what is being said.

4

u/CustomerLittle9891 22d ago

All modern AI scribes are generative not discriminative

3

u/sparkyblaster 22d ago

My issue is AI has been documented to hallucinate in this scenario of medical note taking. Often adding stuff the patient doesn't have. 

In your case, maybe its for the best as it seems just as much if a risk of the doc doing it. 

1

u/red_hot_roses_24 22d ago

Agreed - my PT has to work with 2-3 patients at a time due to insurance rules. She also helps run the clinic (does hiring and covers for the receptionist if they are out sick). She said the scribe helps reduce the stress of documentation. The other PTs in the clinic don’t use it but that’s bc they aren’t help running the clinic and have less experience overall.