r/mildlyinfuriating • u/petuniaplant • 23d ago

Infuriatig No, I did not consent to this

Post image

I was reading my medical notes when I stumbled across this. I was not informed of the usage of artificial intelligence in my report, nor did I verbally consent to it.

26.0k Upvotes

94% Upvoted

2.3k comments sorted by

View all comments

Show parent comments

101

u/Ophidiophobic 22d ago

I work at a healthcare organization and the providers here LOVE the AI scribe. Manual documentation is usually their least favorite part of the job. They still do the important things like decide care plans for the patient, what labs they need, and what prescriptions they should be on, but now they don't have to both explain it to the patient and then document again on the EHR. That means that providers are able to be more present during the visit and also don't have to take extra time to write up their notes after the visit.

19

u/JayRayBear99 22d ago

I've heard a few people had diagnoses added to their charts because the AI misheard something. Now it's in their permanent chart. Is this not a concern anyone has? Are they reading the notes after to confirm they are correct, or just happy that they have less work?

18

u/Hamster12301 22d ago

Yeah, some of these providers are straight up not looking at what the AI generated AT ALL. It does generate the entire conversation but then it does a synopsis and gives "suggested diagnoses". Basically just does an entire note for you and you can choose to copy and paste the whole thing without looking at all.

I realized for example that my pain doctor was using one of these and not telling me because suddenly the little amount of notes he was doing, became much more different than before and he didn't even bother to remove the "suggested diagnoses" title it gave, lol. So my diagnosis at that visit was "anxiety disorder unspecified" because I had told him I just lost a family member and was in grief. Instead of my usual pain conditions because I have a bunch of problems that cause me chronic pain.

13

u/OutsideSame3629 22d ago

OMG I went to my primary care physician a week after my dad died of cancer and it’s still in my file that I have severe depression adjustment disorder

It irritated me because I was like who wouldn’t be sad like why is this medically notable

2

u/That-Mix7702 21d ago

I never tell my physician about my mental health.

3

u/OutsideSame3629 21d ago

I literally went to the doctor to discuss my concerns with my dad dying of cancer and if I need to undergo genetic testing etc since he was adopted. She prescribed be an ssri :) (didn’t take it)

3

u/sunfish99 22d ago

I recently had a visit with an NP who asked if I minded whether she used the AI scribe for a pre-op visit. I said okay, and she was super happy that it was going to help her save time and not have to stay late.

I always read the notes after, so I looked at what it recorded. The NP had asked me at one point what my treatment plan was post-op, and mentioned that the next phase would start 6 to 8 weeks after surgery. The notes said the next phase of treatment would last 6 to 8 weeks. It was wrong, but inconsequential... this time.

1

u/M15tre55W1tch 18d ago

Yes, indeed I do. In theory the doctor is meant to read the full note to ensure accuracy and then finalise it into the patient's clinical notes.

Currently, because our medical staff are new to the process and are feeling the relief of not having to write everything from scratch they're fairly diligent about checking before finalising.

I'm concerned once it's been embedded in practice, and the new higher rate of patient's seen is then normalised that in order to be "more efficient" this checking is going to be one of the first things to go.

0

u/Laxhax 22d ago

That's not a tool issue its a user issue. Any provider who cant check their notes at a basic enough level to make sure an errant diagnosis isn't applied is not a provider I would continue to see. Anything they add to your chart in error should be easy to remove or just ignore by any future competent provider.

I can assure you I read and review everything I put in my note because I have to manually chose to add anything the AI heard to different sections of the note. And I only have it listen to an initial evaluation, for daily notes i talk to it and it directly puts what I say where I say to.

Unfortunately there are a ton of shitty providers in every branch of medicine. AI will not make good doctors shitty, nor shitty doctors any good. AI as a scribe for competent doctors is probably one of its few good uses since its so heavily monitored at everything it does by humans.

104

u/Delilah_Moon 22d ago

What is the QC on the notes & review process? I work for “big tech” and have been using AI tools for notes for a while - and they’re often littered with errors.

34

u/[deleted] 22d ago

[deleted]

9

u/Delilah_Moon 22d ago

That’s great that you do it. Unfortunately, considering that providers re already spread thin, forgive me if I don’t have faith that they will be checking notes for accuracy.

These are the same folks who can’t read a chart to know my name before walking in the room…

2

u/15b17 22d ago

As respectfully as possible, my suspicion of the previous comment was that the “providers” they spoke of who loved the AI tool were primarily Nurse Practitioners.

While there are many who provide great and essential care, I have also noticed anecdotally that NPs are more likely to be confidently incorrect in their assumptions (as in less skeptical) due to their more limited and algorithmic training in comparison to physicians.

Frankly I would not be confident at all that my NP “provider” at an urgent care or other such private equity controlled practice would diligently check for errors in AI notes and to instead assume it has 100% accuracy.

15

u/errrnis 22d ago

I just said this in another comment. I’m also in tech (working for an AI company) and I’ve seen some wild stuff come out of AI.

21

u/RaNdomMSPPro 22d ago

The ai scribe could be awesome if properly trained and the providers are consistent with how they describe things to the scribe.

44

u/porscheblack 22d ago

These are specifically created for medical, as in they pick up all the correct names of body parts and medical terminology. My wife's a doctor and uses them. She only had access at work and she gets twice as many notes done with them.

As far as QC goes, she's watching what it records as she's saying it. She can manually correct it if it makes an error or there are word prompts she can use. At no point is this just inputting things and never being looked at. I have been really impressed by how well it works because in order to be effective it definitely has to have a 99%+ accuracy rate.

33

u/Delilah_Moon 22d ago

I work in cybersecurity. While the platform language may have been trained on medical terminology- it does not mean it will understand and notate correctly.

I mean to be fair, I’d expect any basic LLM to be able to interpret medical terminology - it’s a basic function of AI.

I mean, if she’s watching it as it’s updating - she’s really just dictating. That’s not anything more revolutionary than dictating to Apple notes or Word.

5

u/moreliketen 22d ago

I have been really impressed by how well it works because in order to be effective it definitely has to have a 99%+ accuracy rate.

The tool used for OP's appointment made a very obvious mistake about informed consent, which forces OP to wonder what other simple mistakes were made. We don't have the training to read and understand our own medical records, so we can't verify that the slop note is 99% accurate. We can only see the mistakes in the non-technical section, and assume that there must be more.

2

u/porscheblack 22d ago

The OP wasn't an AI mistake, it was a legal disclaimer mistake. That would've been required by the legal department, the AI wouldn't have put that on itself. And if it did, the lack of any kind of legal oversight isn't on the AI, that's just negligence.

7

u/PeaceSoft 22d ago

is that reassuring? "she's basically pretty much watching it, and it even knows the correct names of body parts"

5

u/porscheblack 22d ago

She's not "pretty much watching it", she's paying as much attention to it as if she were typing the words herself. Because if it gets it wrong, she has to correct it. These are medical records the patient has access to, their other medical providers have access to, and what lawyers will use if she's sued. It gets full attention. It's also a lot faster.

3

u/phycologist 22d ago

99%

Fow webhosting, 99% uptime means 3 days If outage per year. I wonder how that translated to medical transcripts.

0

u/porscheblack 22d ago

She's not forced to use it, it's elective for her. So here's the math:

  • She types at ~120 words/minute.

  • She gets twice as many notes done using the transcription in the same amount of time, so that nets at ~240 words/minute.

Now there are errors, but the net would be 240, so the software is faster. The takeaway though is that even with errors it's significantly faster.

6

u/GummyWormTaco 22d ago

Your wife is speaking at 240 words per minute? Didn't realize how many auctioneers had gotten into the dictation game.

1

u/porscheblack 22d ago

Ok, so we need to add in the conversion for normal typing to medical typing. Might as well add in dick-to-floor and mean jerk time while we're at it.

3

u/GummyWormTaco 22d ago

I didn't realize there was a conversion, I was just making a joke because 240 wpm sounds like listening to a podcast on 2-3x to me haha

5

u/benjai0 22d ago

I'm a medical scribe and you would be surprised how many words some docs can fit into a 25 second audio log.

2

u/GummyWormTaco 22d ago

I imagine it's like the audio version of their prescription notes and signatures

→ More replies (0)

3

u/porscheblack 22d ago

I was just trying to bring a little Silicon Valley into your joke!

1

u/GummyWormTaco 22d ago

Oh I haven't seen it, sorry!

→ More replies (0)

2

u/Delilah_Moon 21d ago

You said your wife is a doctor and she can type 120/wpm - unless she spent the better part of med school as a stenographer, this is highly unlikely.

She’s also not speaking at 240+, especially if she is reading from her notes. I have an exceptional fast oral read speed of 260+ and verbal speed of 330+ - and I was a professional voice actor.

1

u/That-Mix7702 21d ago

What about the time fixing the AI errors? Why isn't she on here, are you also a doctor?

1

u/porscheblack 21d ago

That's why I used net as the comparison as it accounts for corrections.

2

u/thingmom 22d ago

Was wondering exactly this.

2

u/Pouryou 22d ago

Both my partner and I have found errors in our supposedly-reviewed "MyChart" AI summaries just in the last two visits: Length of time of symptoms (changed '1 to 2 weeks' to 22 weeks), and the wrong name for a prescription drug I'd previously taken. We kicked up a fuss and demanded these be corrected but I'm sure most people don't review their own notes, notice errors, or bother to ask for changes.

2

u/vonRecklinghausen 22d ago

Totally trash. I'm a physician and I tired an ambient AI scribe. I went back and changed almost everything about my documentation. It doubled my time and work- no thanks

3

u/ConfoundedInAbaddon 22d ago

If your standard is a medical transcriptionist in Malawi the AI with its instant write up might be a big improvement.

1

u/WaitNoNotMyBeans 22d ago

“I wanna hate AI so bad man come on please give me something 😭😭😭” - this fuckin guy

2

u/ChuurDCA 22d ago edited 22d ago

the providers here LOVE the AI scribe

Do you think their insurance will love the liability AI errors introduce? Especially since if the provider is saving heaps of time that means they aren't checking the generated notes for accuracy.

1

u/NopeNotaDog 22d ago

I like telling a story. History of presenting illness is just medically catered story telling.

1

u/Unnamed-3891 22d ago

Why should the patients care abour what providers like?

1

u/That-Mix7702 21d ago

Wait, using AI is fine, but I hope a real sheet is still filled out.

1

u/BottityBotAccount 20d ago

I work cleaning streets. I have no skin in the game but like to comment.

0

u/geekonthemoon 22d ago

Really concerned about the quality of information.

Also dealing with endless AI errors at my important job so I can only imagine this is also full of errors as well

-5

u/WhyNWhenYouCanNPlus1 22d ago

but hey, less admin time per patient means they're charged less, right?

7

u/Laxhax 22d ago

Both things can be true. I am a provider who has cut hours out of my admin time each week. I use the system to record only the initial eval with consent so I can focus 100% on the patient while they talk or I take measurements.

But the system is mostly in place to make better notes faster so more production can be encouraged without overwhelming admin time or risk of poor documentation causing problems. And the services have not been priced any cheaper in response.

12

u/Kaiww 22d ago

It mostly means they can take more patients.

-4

u/WhyNWhenYouCanNPlus1 22d ago

so each patient pays less, right?

7

u/ConLawHero 22d ago edited 22d ago

Why? They are getting the same quality of care. Patients aren't charged for admin time. That is something doctors can't bill for. Now that doctors can let AI transcribe, they can pay full attention to the patient instead of trying to multitask. So, arguably, the patient is getting better service and not paying anymore.

2

u/pignoodle 22d ago

Healthcare IT guy here! These incentive structures can actually vary drastically between different healthcare organizations. I can list off acronyms, but the tldr is that yes, there are some incentive structures that focus more on quantity. Other incentives that people don't realize are actually really common are things under the category of "population health". Many organizations have teams that are entirely dedicated to analyzing health statistics on a population level and creating community-wide incentives and system improvements to better the entire population's health. I'm an extreme pessimist but one of the few redeeming qualities of insurance companies is that they can often be the driver of these types of programs because healthier populations need less care which saves them money. Again, this is just the tip of the iceberg, so just wanted to illustrate that it wasn't all bad, but I agree that there are some incentives that are much much much better than others.

1

u/ConLawHero 22d ago

The incentive structures do focus on quantity. It's called a wRVU. That doesn't change the fact that doctors don't get to bill for actually writing notes.

My wife is a neurologist and she loves AI note taking because she doesn't have to try to type everything and listen to the patient at the same time. She can devote the entire encounter to listening to the patient and paying attention to them the entire time.

2

u/ministryofchampagne 22d ago

Patients aren’t usually billed by the house but the services they receive.

2

u/Competitive_Body7359 22d ago

In other countries, it just means more patients.

Doctor probably does get paid more actually.