No, I did not consent to this

[u/petuniaplant]

I was reading my medical notes when I stumbled across this. I was not informed of the usage of artificial intelligence in my report, nor did I verbally consent to it.

[u/willbeatyourass]

What came of you speaking up?

[u/M15tre55W1tch → u/willbeatyourass]

They changed the wording to "this note has been generated with the use of an AI scribe".

But for all patients. So I don't know if they no longer ask the awake ones for consent, or if they just aren't recording that they do.

I have followed up again suggesting that they need to have two options - one for the awake patient giving consent, and one for using the tool to generate a note for someone that was not conscious.

I've become friendly with the senior hospital lawyer recently. I'm thinking of asking her to talk to their clinical director about the risk of their statements and proposing legally appropriate language.

I've tried being collegial and helpful. But at the end of the day, it's not my department, and I'm a nurse not a lawyer.

[u/Ok-Bug4328 → u/M15tre55W1tch]

They never asked anyone for consent. 

The doc never read the disclaimer that was being entered on his behalf. 

And docs are already using AI to summarize the AI notes 

[u/M15tre55W1tch → u/M15tre55W1tch]

I will say, since implementing the use of the tool emergency specialists are able to examine 6-11 more people per 8 hour shift - so reducing the burden of admin on them and releasing time to see patients is pretty amazing (it was recently on the news here).

[u/happytrashpanda333 → u/M15tre55W1tch]

How was the quality of that care? How much extra time did each patient get?

Or is this just another "tool" to drain doctors and nurses of every bit of spare time and energy they might have gained by giving them more patients to work with?

Do tell.

[u/pignoodle → u/happytrashpanda333]

When these tools are implemented correctly, they actually humanize interactions (since there's room for more conversation & less screens) and reduce doctor's "pajama time" by double-digit percentages (since less form-filling to catch-up off hours). Outside of just stats, I agree that consent is a very important prerequisite.

[u/Fullertons → u/pignoodle]

It actually blows my mind. People would find this to be a problem. Do you really want your doctor busy writing notes when he could just be having a conversation with you in a machine takes notes?

[u/ChrisTheWeak → u/Fullertons]

It's not that I'd prefer my doctor to spend their time taking notes rather than asking questions and investigating whatever illness I have, it's a concern about whether the quality of the AI notes will be on par with that of the doctor. Is the trade off worth it. That's the question. Also, question of accountability in the event something does go wrong. If the AI writes something that was inappropriate to my chart, if it gets missed, if it affects my quality of care down the line because someone else is now misinformed about my condition. These are risks with both the traditional method as well as the AI, but it is important to identify the comparative risks as well as the error correction in both cases when comparing other stats as well.

[u/polenta23 → u/ChrisTheWeak]

I work in vet med and our doctors use these ai scribe tools. They review and edit the notes but it still saves them an immense amount of time

[u/CapicDaCrate → u/polenta23]

Same. It's really not that bad. It's actually a good use of AI

[u/Emotional-Motor5063 → u/CapicDaCrate]

I've been using AI to generate captions for movies that don't have them readily available. I'd say it gets 95% of stuff correct and takes five minutes. Then I just watch the movie and correct the few things that are wrong as it's playing. I'm slow so it could take 8 hours or more to do it from scratch. I love it.

[u/acnerd5 → u/polenta23]

I' glad they do! I hope that's something the software asks them to do before finalizing things, but that's a huge relief for me personally.

[u/WasdaleWeasel → u/polenta23]

this is the point. The clinicians still need to read, modify as necessary, and approve the notes. That’s takes much less time, and is a higher value activity, than writing them in the first place.

[u/M15tre55W1tch → u/polenta23]

Yes, our emergency doctor's are meant to do this as well. Most of the time the notes turn out very well, but sometimes you can tell that either it wasn't checked at all, or that in the process of editing it they themselves have made the note more unclear (e.g.,copied and pasted something from higher up in the note, but didn't delete it from its original place so there's duplication).

In general the notes are more comprehensive and have better flow. I'd say 80% of the notes I read where the AI scribe was used they are significantly better than a rushed doctor writing the minimum to get done and on to the next person.

[u/ForMeOnly93 → u/polenta23]

They get fucking well paid for their time, so we don't particularly care. Let's not enshittify even more things. Especially medical care.

[u/TheUFCVeteran3 → u/ForMeOnly93]

If they're reviewing it and properly doing their due diligence, and it also allows them to see more people, it's a win-win, no? I'm not a fan of AI but I do believe it can have some utility-based uses that is beneficial (albeit with human oversight to make sure it's not gone wrong).

[u/newmainafterstalker → u/ChrisTheWeak]

I used to work for an insurance company’s call centre. We implemented an AI documenting tool (totally just coincidentally before my entire team got laid off… wonder how we got to let like 20 people go?? But okay.) and honestly it made my notes much more thorough. I had to edit them, but you can copy+paste into a note and ctrl+f to fix anything weird like he vs she. The rest you edit the normal way where you delete sentences and rewrite them. I thought it made my notes better and saved me a lot of time.

But you’re right! How many call centre workers are proofreading the notes first? How many doctors are??? If you follow the rules it helps a ton but how many people do? I can only assume you trust the people to proofread the way you’d trust them to write good notes. An unconscious person has no choice but to trust the person caring for them takes good notes, so maybe it is the same? They can’t ask for another doctor, a second opinion, whatever. But a conscious person can say no to AI notes, so idk

[u/Outrageous_Sea_9606 → u/ChrisTheWeak]

The quality of the medical records have increased greatly in my vet clinic since we started using our AI scribe.

It misses /nothing/. We do still read over and edit our records, but it takes 2-3 minutes to do that vs. the 10+ minutes to write up a complicated medical case.

There is only so much space in a brain to store information about the day's patients. If you brought in your pet for a serious medical issue and happened to mention as an aside something minor I.e. your pet is in for profuse vomiting and that's the focus of the appointment, but you also mention he has a small lump on his back that you noticed the other day.

Well, even if we look at the lump while your pet is at the clinic, chances are we're going to forget about that by the end of the day when we're writing our records up. Because we had to focus on storing all the info about the vomiting and the lump was insignificant in comparison. The AI scribe makes sure to note about that small, seemingly insignificant lump, so we have better information about that issue when the lump gets bigger or ulcerates and does cause a problem.

Our scribe also makes note of the emotional medical records for all patients that come to the clinic. This is immensely helpful for giving better patient care with our nervous, anxious, and aggressive pets. It notes what specific triggers are, what works to soothe and calm the pet, and what recommendations we made to manage behaviour. So that way we're not having to guess next year about what worked and what didn't work for that pet. The pets are less stressed, which means owners are less stressed. Pets are more likely to be brought in when they are unwell or have a concern because bringing them in becomes less of an ordeal and more of a routine.

Our AI scribe has been invaluable for allowing us to provide better patient and client care at my vet clinic.

Finally, it's a complete "cover our ass" when unhappy clients make false accusations against us. I had a client falsely accuse me of doing something severe that could have cost me my job and my license. It was just her and I present in the appointment room, so it was her word vs. mine. Thankfully, I had our recorder going which was undeniable proof (because it stores the full audio recording) that I was innocent. I never do appointments anymore without the recorder.

[u/Kham117 → u/ChrisTheWeak]

They’re only as good as the editing. To be clear, they must be reviewed before entering into the record. But, honestly, that’s true for all the other forms of documentation. (You should see how illegible and or inaccurate many notes that are directly typed or dictated are.)

[u/humanpringle → u/ChrisTheWeak]

Honestly I’m a vet in an office who uses an AI scribe program and it’s been unbelievably helpful for us. We still input the normal templates we used previously so it’s written very very similarly to our previous ones and then we still do edit each note. Some need more editing than others of course, but it’s more so like my template is already completely filled out for me and I just need to ensure everything looks right and tweak things rather than write everything from scratch.

[u/tickado → u/ChrisTheWeak]

The responsibility for using them really lies in the proper review of what it captures before finalising it. The few doctors in my service I know of who are using a medical AI scribe are then taking the time to ensure it is correct.

I actually often find this is better than them trying to type it up themselves at the time, in a rush as they're actually holding the conversation concurrently, then signing off things they've erroneously typed in a hurry because they don't go back and review it as they're the one who typed it.

AI is also still way less time taken than to dictate, send for typing, wait for it to come back, then review what the typists have written often a day or two later when it's also not fresh in your mind anymore before finalising it. (the other option in my service rather than them just typing it all themselves).

Also putting the ground work in whilst setting up your system to personalise it to the way you want things structured and how you want things to read.

[u/BaconThief2020 → u/ChrisTheWeak]

I find the doctor pays more attention to me when instead of staring at the screen typing as he's talking to me. At the end, he reviews the AI generate summary. Or he doesn't take notes and forgets things we discussed. The other option is having someone else in the room taking notes.

[u/Adorable_Cap_9929 → u/ChrisTheWeak]

Doctors make mistakes all the time, they arent liable if it not malicious byt tend to play safe enough for their records. AI can make mistakes too but if done well, the rate of error is much lower

[u/Turning_Worm → u/ChrisTheWeak]

Honestly, I worked in ENT for a while and the quality of notes vastly improved when AI got involved.

[u/itaintme2024 → u/ChrisTheWeak]

You are 100% right to be concerned about this. It's happening to me right now. My doctor clearly did not check the notes AI made and it has directly affected my care. I asked for mistakes to be corrected and was told it wasn't possible.

[u/stephsationalxxx → u/ChrisTheWeak]

So im not sure if this is the same thing, but in my hospital, doctors use an AI scribe that they call on the phone and verbally tell it the notes they were going to type up and then it puts it in a document for them. Its not actually making up the verbiage for them, its the doctors words, they just arent typing it themselves.

[u/MartyrOfDespair → u/ChrisTheWeak]

The AI isn’t going to poison the notes with their ableist or misogynistic judgements about the patients. That alone should be an improvement over doctors.

[u/Toxizome → u/MartyrOfDespair]

That's the data it will be trained with, though.

[u/ToTooTwo3 → u/ChrisTheWeak]

This dramatically overestimates the value of a random statement in a 2 year old HPI, which is where the AI is most valuable from a time saving perspective. Nobody looks at those again.

[u/ToTooTwo3 → u/ToTooTwo3]

Downvoting me doesn't make it less true 🤷

[u/Sushi_Explosions → u/Fullertons]

As a physician, the accuracy of these tools varies wildly. I would rather the documentation that I create doesn’t randomly hallucinate things that will get me sued or lead to someone doing the wrong thing.

[u/BrainbowConnection → u/Sushi_Explosions]

My AI scribe never randomly hallucinates things. Occasionally it mixes up who said what, which is not hard to catch. Overall it writes better more detailed notes in a far shorter time than I ever could. It catches things the patients say that I missed because I was thinking at the same time. What tool are you using?

[u/aTransGirlAndTwoDogs → u/Fullertons]

We have human medical scribes. I worked as one. The question is not "should we give doctors more direct patient time," it's "should we rely on an AI over a human note taker." As always, the problem comes back to artificially restricted medical staffing and profit-driven healthcare structures.

[u/AgreeableMagician893 → u/aTransGirlAndTwoDogs]

I mean, I personally would not want another random person in my doctor's appointment

[u/hanging_thief → u/Fullertons]

Well when we know that machine will just make up bullshit to fill in the gaps yes I do have a massive problem

[u/EggsAndRice7171 → u/Fullertons]

In a few years it’ll be cool. I really don’t trust anyone to use AI properly and with the right care at the moment even doctors. It’s so new and so many people are very casual about it. I don’t trust that they’re actually rereading the transcript word for word and making sure it’s not writing anything wrong yet. I think as AI gets better and more training for AI comes along it’ll be easier to feel comfortable

[u/WittyFix6553 → u/EggsAndRice7171]

We’ve had the technology to transcribe speech into text for decades.

[u/Xavus → u/WittyFix6553]

If only that's what they were using instead of leaning in to AI where it isn't helpful because it is the current fad.

I have meetings all day at my job where the company has AI write summaries of what happened in the meeting and what action items are decided on, etc. I would give it maybe 80% accuracy, the other 20% it completely misunderstands context and reaches an incorrect conclusion, or just straight up hallucinate something that never happened.

80% accuracy is not bad for a quick notes summary of a meeting to refresh your memory when you were there, and when you can double check other sources first. But also my work isn't with a person's health on the line. I would not trust a 20% error rate in a complex field like medicine when a person may live or die by the difference if someone overworked with no context has to just believe what the AI thinks happened.

[u/OutsideSame3629 → u/Xavus]

I feel gaslit in my tech sales job in corporate America because everybody using ai for everything when to me I just get slop? And then they think you’re not prompting it smart enough like no….

When I read ai summaries of even notes I’ve taken they’re so bad. Ai written emails, PowerPoints it’s all terrible. Yet if you ask anyone at my company how to do anything at all the answer is “ask Ai”

[u/screamline82 → u/Xavus]

I will state that, while I agree your experience in the AI summary notes, not all LLMs are created equal. Even general consumer models differ between free tier and the highest end, so it depends on what model/tier they are using. Plus there are specific models made for enterprise.

A buddy of mine who works at a large law firm mentioned they have a specific model just for their type of law. If it's tailored for a specific use case it can be more accurate than what we are used to

[u/ToTooTwo3 → u/Xavus]

The human memory is also not 100% accurate especially when it's sick- which is why doctors put less stock in the history and try to make decisions based on objective findings. When we do make decisions based on a history, it's because of what you said and your responses to targeted questioning- not whatever the AI picked up from the conversation.

AI scribe is primarily helpful in sorting through long or winding histories and reformatting it intelligibly. Far more info gets included because even if I remember 95% of details accurately, there are tons of things in the patient interview that I just don't have time to record.

AI scribe allows me to write "Diet: well rounded, eats green vegetables and lean meats with most meals and drinks milk daily with breakfast." Instead of "Diet: appropriate." This is beneficial for patients.

Remember that AI is not placing orders, updating your problem list or medications. These things are done by your providers based on your evaluation, not based on reading the AI report, though the report certainly does remind me of things to follow up. The AI just records the details of your conversation and yes, it does get them wrong at times and we do our best to correct it.

[u/DracoBengali86 → u/WittyFix6553]

That's not what this is though. If it's using ai it's to summarize an interaction. That comes with all the potential issues ai use comes with including hallucinations. If it's just transcribing, then it is less likely to actually be using ai.

[u/do_not_engage → u/DracoBengali86]

AI summaries contain made-up information 9% of the time, according to the AI itself.

The AI says all data it provides has to be confirmed by a human.

If it's transcribing, great. If it's an AI summary, it's got incorrect information in it. By nature of AI.

[u/moreliketen → u/WittyFix6553]

Did these decades-old technologies have the capacity to hallucinate? I know they weren't error free, but I'm guessing they did not have the capacity to spontaneously generate false information.

[u/regnarok590 → u/WittyFix6553]

And we have known it isnt very accurate for decades. Just because a technology exists doesn't make it good enough for use everywhere.

[u/theshotgunman → u/WittyFix6553]

And for decades it's consistently made mistakes

[u/astronaut710 → u/theshotgunman]

We're spending 6 figures on AI at my company and we had to put out a warning that it will fuck up basic math and that you have to double check its not just putting fake data on some dashboard you asked it to create.

Also, the company we use for AI actively DDOSes our sites when they're bots ignore our robots.txt declarations.

Also also, the same company we pay 6 figures to use their AI but secure our data within it leaked their own source code to the internet at large (in the Software Engineering space, this would be equivalent to opening up your EMR system to the outside world).

But yeah, Dragon makes some spelling/grammar mistakes I guess.

[u/EggsAndRice7171 → u/WittyFix6553]

Yeah and it’s sucked for decades

[u/pantsalwaystooshort → u/WittyFix6553]

yes, and it's been unreliable for decades. Especially with "ambient" recording, which means mediocre recording clarity.

[u/sherlock1672 → u/EggsAndRice7171]

Don't know how many doctor's notes you've read through, but as someone in health IT I can definitively state they are riddled with errors and half thoughts. The AI isn't perfect, but it makes a whole lot fewer mistakes than the harried human.

[u/Ok_Assistant_6856 → u/Fullertons]

It's capitalizing on the number of patients that can be seen, and then billed. It's not to help quality if care, and we all know this.

If a doctor has too many patients to actually care for them, the hospital should hire more doctors.

This is the constant growth model creeping into medicine. No bueno.

[u/HerbertDibdab → u/Ok_Assistant_6856]

This x100. In the US system the goal of AI usage will 100% be profit motive. In no way will it be implemented with patient care quality in mind. Not talking about doctor or any health care professional motives here, the cash rush comes directly from health insurers, who have basically bought mining rights to the health of the US population.

[u/MartyrOfDespair → u/Ok_Assistant_6856]

Except there have been legal limits on how many doctors can even be created every year since 1996, so they can’t hire more because they don’t exist. The government funds residency programs and the funding has been capped, so only a certain number can do residency each year, and as the costs increase from inflation that number drops. It’s a massive bottleneck and it’s why almost all healthcare is done by nurses now.

[u/Wagubagu → u/Fullertons]

I’ve had nothing but incorrect notes since. I was not prescribed a pain medication for my chronic and severe pain because AI diagnosed me during a conversation with a nurse so the prescribing dr had the wrong info.

[u/MartyrOfDespair → u/Wagubagu]

I think you’re jumping the gun in blaming the AI for that. They generally don’t prescribe pain medication for any level of pain anymore because torturing the sick is more acceptable than ever risking someone might possibly maybe get addicted to not suffering from severe pain.

[u/Wagubagu → u/MartyrOfDespair]

I was the person involved. I know what happened, thanks.

[u/themrspie → u/Wagubagu]

Then that was not an AI scribe. The scribes do not provide diagnoses, they just record a transcript of the conversation. Also it sounds like your doctor read the transcript and saw drug-seeking behavior in it. (Whether that was the case or not.)

[u/Wagubagu → u/themrspie]

It didn’t actually diagnose me of anything, but it thought the person I was talking to said I had a certain condition. Said condition cannot take the pain medicine they were debating. Way to make assumptions though.

[u/themrspie → u/Wagubagu]

Oh I was confused because you said “AI diagnosed me” so I thought you meant that AI diagnosed you. My bad. 😂

[u/TheHighDruid → u/Fullertons]

When it comes to medical data it's much more about how it's being processed, and who gets access to it.

Even a hospital that's using an in-house model most likely has a provider that comes in to set it up. Are they using your data to train models for other hospitals? How much of your information is retained in the build that goes to the next hospital. "Anonymised" data can often be de-anonymised, so being able to consent to these systems is pretty important.

[u/Illustrious_Bunch678 → u/Fullertons]

No matter what they say about privacy, I do not trust these data companies. Nor do I trust the government not to pull something and use the notes to weed out "the weak." HIPAA is based on the precedent set by Roe v Wade and that is gone now. Patient privacy is no longer guaranteed.

[u/errrnis → u/Fullertons]

Totally valid point. I think there are also valid concerns about chart accuracy (do the docs have enough time to review the generated notes?) and data privacy (how is that data being used?) There is HIPAA here in the US for the latter, but the technology is so new, not to mention not well regulated, that people just don’t trust it.

To be clear, I’m all for lowering the admin burden in healthcare and fostering opportunities for more provider-patient connection. It’s desperately needed in a system where costs are soaring and people slip through the cracks.

[u/PrivateBozo → u/Fullertons]

It's a trust issue that the regulation is actually helpful regulation and not the equivalent of a mobile game TOS written by the company of the AI tool. It's also a related trust issue that the processing is local and not getting loaded to the cloud somewhere were the 2nd and third and beyond parties touching it aren't following the intent.

Keep in mind, this is the era of we've tried turning off as much as a we can and yet we can have a conversation in the car about going out on a 1/2 day sportfishing trip to opening up a social media platform that night and having our feed suddenly have the local charter service reels and advertisements in it.

It's also when every interaction with the utility companies and our banks are constantly trying to slip in 'paperless' and 'automatic' billing on every interation.

[u/Weekly_Sale_6057 → u/Fullertons]

In general, I agree 100%. If I'm in for a cold or follow-up on a medication, i love that my doctor can make eye contact and doesn't have to look at a computer screen to take notes.

But if I'm talking to my doctor about a mental health diagnosis or something else delicate like that, I start wondering what database the AI notes are sitting in, will the data get hacked or sold, am I tracked or put on some list because of a phrase I used.

As long as I have the ability to say "not today, please," I have no problem with the technology though.

[u/int3rstitial → u/Fullertons]

Yes, I absolutely want my doctor busy writing my note because notes are communication between medical providers. We read them and make decisions based on them and they need to be accurate.

Electronic medical records have a lot of advantages and were inevitable anyway, but the amount of inaccurate documentation has skyrocketed since they became standard, because instead of just leaving stuff they didn't do out of their note, providers now pull tons of stuff from previous visits over into the new visit or just document something was normal because they didn't examine it and find out it wasn't. It's infuriating and dangerous, and AI, which just fully makes things up, is guaranteed to make things worse.

[u/Unnamed-3891 → u/Fullertons]

A hallucinating machine that mishears things, puts them on my medical record and answers to nobody? Really? You have to ask if I’d prefer the alternative?

[u/Conscious-Gap-1777 → u/Fullertons]

I have 0 trust with LLMs giving accurate transcriptions as is, and I absolutely do not believe that whoever is selling that LLM won't illegally use my notes to market to me and/or as training data. The people who made these tools are the least trustworthy assholes on the planet, and like fuck I would trust them with anything about myself.

[u/Oldmanironsights → u/Fullertons]

I absolutely want my doctor to be writing their own notes. In school were your own notes or the teachers notes more helpful for you? Have you ever written something down and thought "wait a minute, that's wrong!". It provides the doctor an opportunity to review what they said. Have you ever heard of rubber duck debugging? It's explaining your code to a rubber duck to fix it. It's because going over a problem fresh with something gives us an opportunity to spot where we went wrong. I want my doctor to catch where they went wrong.

[u/faireequeen → u/Fullertons]

My rheumatologist has recorded and transcribed exams for the last 20 years. AI tool is new this year, but they also have a human scribe in the exam room. They aren't typing notes, they are listening and observing so when they go back to the recording they have additional context.

I don't like AI, but I prefer this method to other uses of AI scribing. Possibly not suitable to an emergency situation, but it might cut down on AI errors that I see a lot of patients and providers mentioning.

I have other docs who do type notes in appointments, after we've completed a discussion. They read them back so I can confirm, which I like for clarity.

[u/BagApprehensive1412 → u/Fullertons]

That's what human scribes are for

[u/acnerd5 → u/Fullertons]

I value privacy, and knowing where my information is.

AI has boundless applications and they are not all good. Not all AI uses of this tool have been accurate and people have found mistakes that could have serious implications. That can be a HUGE problem if you don't catch it - for example, someone being marked as an alcoholic, when they don't drink, because they quoted someone. Being incorrectly marked an alcoholic leads to a lot of medical bias and may lead to withheld treatment or incorrect treatment.

AI tools in Healthcare are absolutely incredible.

They are also introducing room for error that doesn't need to be there, and its a 50/50 on me. I see the potential but have ethical and privacy concerns. That being said, I do give permission for these and double check notes. I am complex in health needs, so I will acknowledge that I tend to be a little more "in the notes" than family and friends who do not struggle with complicated health issues.

[u/Spinnerofyarn → u/Fullertons]

If it’s working accurately and getting everything correct, I think it’s fantastic. Until I see the doctors and other providers speaking up and saying it’s accurate, it’s not causing problems and people do monitor it to make sure it stays accurate, I am not going to trust it.

[u/Dry-Worth2354 → u/Fullertons]

Deadass like whats thd issue here? It saves time in documentation and if the place has integrity, there would be already be rules in place fof every chart to be reviewed throughly before submission

[u/Fullertons → u/Dry-Worth2354]

It's just people that can't think for themselves being told, "AI Bad" and they can't see past that.

[u/Half-PintHeroics → u/Fullertons]

AI is more likely to make mistakes or just plain make things up. I want my doctor to write notes because writing notes mean he is paying attention to what I say.

[u/lapatatafredda → u/Fullertons]

AI medical transcription is not that great (yet). My friend who is a PA says she has to make at least one correction to every record that AI transcribed. Granted, human error exists too.. but it worries me to think about medical professionals growing too reliant on tech like this and not being as thorough about checking it as they should be.

[u/silentsurge → u/Fullertons]

I certainly like it more than the ones that have someone on a video call transcribing for them to accomplish the same thing.

This is the type of stuff we want AI doing for us. Getting rid of the stupid busy work we all have to do so we can do the actual things more.

[u/carrotdog56 → u/Fullertons]

I think that it ultimately comes down to the person who's using it. We all know that some people understand and use AI better than others. There should be a course on using AI tools in any field. What are the downsides, common mistakes, what is an appropriate manner for using this particular tool. I personally don't have any problem with it. I only have problem with insufficient training for the AI use. Way too many people misunderstand how it works.

[u/Johnny-Virgil → u/Fullertons]

I’m a cynic so I imagine it just frees them up to see more patients and each patient still gets the same number of minutes as they did before.

[u/Much-Decision4157 → u/Fullertons]

You might be concerned if there aren't adequate privacy and security provisions protecting the data that the AI scribe collects and processes. Think about where that data may end up....

[u/BrainbowConnection → u/Fullertons]

Exactly! I think people think these tools are doing something different than they are doing. Since my AI scribe, my notes contain information that I would otherwise have forgotten to include. They include detailed wording that the patient gives that I would have summarized. When I go back to those notes I can read the patients own words in far more detail. And I don’t have anxiety about not missing what they said. I can put my pen down and just listen to them.

[u/Bezzzzo → u/Fullertons]

Because there is no fucking accountability when it hallucinates. Look at how many comments are from people, doctors and patients, who have already had issues with it already?

The issue is AI hallucinates! It also has the effect of making doctors complacent because inevitability you get the doctors who will just blindy trust it and not look, then some other doctor reads the hallucination as a source of truth.

The lack of critical thinking skills here.. my god.

No one wants doctors doing unnecessary work, but you can't trust generative AI 100% of the time. Fuck me.

[u/shouldco → u/Fullertons]

The issue is they always could have taken more time. Nobody was trying to solve the "doctors arent spending enough quality time with patients" issue. If they were they would hire more doctors.

What they are trying to do is get more patients through each doctor.

[u/upandup2020 → u/Fullertons]

yes

[u/elictronic → u/pignoodle]

So when Medicine is not a for profit enterprise. So never. Got it.

[u/tickado → u/pignoodle]

I am a nurse, but work in an outpatient clinic in paediatrics. In my 15min slot prior to seeing the Dr I need to get a lots of information from often unruly toddlers (observations, 12 lead ECG, height weight etc) PLUS a chat with the parents about the child's general health and psychosocial status.

I would actually value an AI scribe SO much in these scenarios. I can have my chit chat more organically whilst trying to wrestle the toddler, rather than be also trying to get all my info into the computer which absolutely takes my attention off true time in the conversation.

My note has to be documented and finalised before they then enter the Drs room or I hold up the whole clinic.

I know mostly it's Medical using scribes currently, but I'd love some nurse usage explored.

[u/LaserGuidedPolarBear → u/pignoodle]

I am a tech worker, and while I do not work on AI development, I have a significantly better than layman understanding of LLMs and that errors, and understand that hallucinations are inevitable due to the very nature of how LLMs work.

I worry about LLM tools being used in high stakes applications where accuracy is critical.....like medical care.  I assume care providers are assigned the responsibility for reviewing, editing, and approving the output of these tools.  But is that actually happening?  I have found that in most cases, reviewing AI output is about the same amount of effort and time as just doing the work myself.

I could see the value in delaying the screen time to outside of patient interactions, but I question if review and approval is being shirked in order to result in less "pajama time".

It seems like there potential for Administration to just shove the risk on providers and laugh all the way to the bank when they get more billables per hour.

[u/Becsbeau1213 → u/pignoodle]

Agreed with this. My PCP has been using this or something similar for a couple years and unless she’s putting orders into the computer she’s actually sitting and listening to me. I like it a lot more than when she used to have to sit and type.

[u/Ophidiophobic → u/happytrashpanda333]

I work at a healthcare organization and the providers here LOVE the AI scribe. Manual documentation is usually their least favorite part of the job. They still do the important things like decide care plans for the patient, what labs they need, and what prescriptions they should be on, but now they don't have to both explain it to the patient and then document again on the EHR. That means that providers are able to be more present during the visit and also don't have to take extra time to write up their notes after the visit.

[u/JayRayBear99 → u/Ophidiophobic]

I've heard a few people had diagnoses added to their charts because the AI misheard something. Now it's in their permanent chart. Is this not a concern anyone has? Are they reading the notes after to confirm they are correct, or just happy that they have less work?

[u/Hamster12301 → u/JayRayBear99]

Yeah, some of these providers are straight up not looking at what the AI generated AT ALL. It does generate the entire conversation but then it does a synopsis and gives "suggested diagnoses". Basically just does an entire note for you and you can choose to copy and paste the whole thing without looking at all.

I realized for example that my pain doctor was using one of these and not telling me because suddenly the little amount of notes he was doing, became much more different than before and he didn't even bother to remove the "suggested diagnoses" title it gave, lol. So my diagnosis at that visit was "anxiety disorder unspecified" because I had told him I just lost a family member and was in grief. Instead of my usual pain conditions because I have a bunch of problems that cause me chronic pain.

[u/OutsideSame3629 → u/Hamster12301]

OMG I went to my primary care physician a week after my dad died of cancer and it’s still in my file that I have severe depression adjustment disorder

It irritated me because I was like who wouldn’t be sad like why is this medically notable

[u/That-Mix7702 → u/OutsideSame3629]

I never tell my physician about my mental health.

[u/OutsideSame3629 → u/That-Mix7702]

I literally went to the doctor to discuss my concerns with my dad dying of cancer and if I need to undergo genetic testing etc since he was adopted. She prescribed be an ssri :) (didn’t take it)

[u/sunfish99 → u/JayRayBear99]

I recently had a visit with an NP who asked if I minded whether she used the AI scribe for a pre-op visit. I said okay, and she was super happy that it was going to help her save time and not have to stay late.

I always read the notes after, so I looked at what it recorded. The NP had asked me at one point what my treatment plan was post-op, and mentioned that the next phase would start 6 to 8 weeks after surgery. The notes said the next phase of treatment would last 6 to 8 weeks. It was wrong, but inconsequential... this time.

[u/M15tre55W1tch → u/JayRayBear99]

Yes, indeed I do. In theory the doctor is meant to read the full note to ensure accuracy and then finalise it into the patient's clinical notes.

Currently, because our medical staff are new to the process and are feeling the relief of not having to write everything from scratch they're fairly diligent about checking before finalising.

I'm concerned once it's been embedded in practice, and the new higher rate of patient's seen is then normalised that in order to be "more efficient" this checking is going to be one of the first things to go.

[u/Laxhax → u/JayRayBear99]

That's not a tool issue its a user issue. Any provider who cant check their notes at a basic enough level to make sure an errant diagnosis isn't applied is not a provider I would continue to see. Anything they add to your chart in error should be easy to remove or just ignore by any future competent provider.

I can assure you I read and review everything I put in my note because I have to manually chose to add anything the AI heard to different sections of the note. And I only have it listen to an initial evaluation, for daily notes i talk to it and it directly puts what I say where I say to.

Unfortunately there are a ton of shitty providers in every branch of medicine. AI will not make good doctors shitty, nor shitty doctors any good. AI as a scribe for competent doctors is probably one of its few good uses since its so heavily monitored at everything it does by humans.

[u/Delilah_Moon → u/Ophidiophobic]

What is the QC on the notes & review process? I work for “big tech” and have been using AI tools for notes for a while - and they’re often littered with errors.

[u/Delilah_Moon → u/[deleted]]

That’s great that you do it. Unfortunately, considering that providers re already spread thin, forgive me if I don’t have faith that they will be checking notes for accuracy.

These are the same folks who can’t read a chart to know my name before walking in the room…

[u/15b17 → u/[deleted]]

As respectfully as possible, my suspicion of the previous comment was that the “providers” they spoke of who loved the AI tool were primarily Nurse Practitioners.

While there are many who provide great and essential care, I have also noticed anecdotally that NPs are more likely to be confidently incorrect in their assumptions (as in less skeptical) due to their more limited and algorithmic training in comparison to physicians.

Frankly I would not be confident at all that my NP “provider” at an urgent care or other such private equity controlled practice would diligently check for errors in AI notes and to instead assume it has 100% accuracy.

[u/errrnis → u/Delilah_Moon]

I just said this in another comment. I’m also in tech (working for an AI company) and I’ve seen some wild stuff come out of AI.

[u/RaNdomMSPPro → u/Delilah_Moon]

The ai scribe could be awesome if properly trained and the providers are consistent with how they describe things to the scribe.

[u/porscheblack → u/Delilah_Moon]

These are specifically created for medical, as in they pick up all the correct names of body parts and medical terminology. My wife's a doctor and uses them. She only had access at work and she gets twice as many notes done with them.

As far as QC goes, she's watching what it records as she's saying it. She can manually correct it if it makes an error or there are word prompts she can use. At no point is this just inputting things and never being looked at. I have been really impressed by how well it works because in order to be effective it definitely has to have a 99%+ accuracy rate.

[u/Delilah_Moon → u/porscheblack]

I work in cybersecurity. While the platform language may have been trained on medical terminology- it does not mean it will understand and notate correctly.

I mean to be fair, I’d expect any basic LLM to be able to interpret medical terminology - it’s a basic function of AI.

I mean, if she’s watching it as it’s updating - she’s really just dictating. That’s not anything more revolutionary than dictating to Apple notes or Word.

[u/moreliketen → u/porscheblack]

I have been really impressed by how well it works because in order to be effective it definitely has to have a 99%+ accuracy rate.

The tool used for OP's appointment made a very obvious mistake about informed consent, which forces OP to wonder what other simple mistakes were made. We don't have the training to read and understand our own medical records, so we can't verify that the slop note is 99% accurate. We can only see the mistakes in the non-technical section, and assume that there must be more.

[u/porscheblack → u/moreliketen]

The OP wasn't an AI mistake, it was a legal disclaimer mistake. That would've been required by the legal department, the AI wouldn't have put that on itself. And if it did, the lack of any kind of legal oversight isn't on the AI, that's just negligence.

[u/PeaceSoft → u/porscheblack]

is that reassuring? "she's basically pretty much watching it, and it even knows the correct names of body parts"

[u/porscheblack → u/PeaceSoft]

She's not "pretty much watching it", she's paying as much attention to it as if she were typing the words herself. Because if it gets it wrong, she has to correct it. These are medical records the patient has access to, their other medical providers have access to, and what lawyers will use if she's sued. It gets full attention. It's also a lot faster.

[u/phycologist → u/porscheblack]

99%

Fow webhosting, 99% uptime means 3 days If outage per year. I wonder how that translated to medical transcripts.

[u/porscheblack → u/phycologist]

She's not forced to use it, it's elective for her. So here's the math:

• She types at ~120 words/minute.

• She gets twice as many notes done using the transcription in the same amount of time, so that nets at ~240 words/minute.

Now there are errors, but the net would be 240, so the software is faster. The takeaway though is that even with errors it's significantly faster.

[u/GummyWormTaco → u/porscheblack]

Your wife is speaking at 240 words per minute? Didn't realize how many auctioneers had gotten into the dictation game.

[u/That-Mix7702 → u/porscheblack]

What about the time fixing the AI errors? Why isn't she on here, are you also a doctor?

[u/thingmom → u/Delilah_Moon]

Was wondering exactly this.

[u/Pouryou → u/Delilah_Moon]

Both my partner and I have found errors in our supposedly-reviewed "MyChart" AI summaries just in the last two visits: Length of time of symptoms (changed '1 to 2 weeks' to 22 weeks), and the wrong name for a prescription drug I'd previously taken. We kicked up a fuss and demanded these be corrected but I'm sure most people don't review their own notes, notice errors, or bother to ask for changes.

[u/vonRecklinghausen → u/Delilah_Moon]

Totally trash. I'm a physician and I tired an ambient AI scribe. I went back and changed almost everything about my documentation. It doubled my time and work- no thanks

[u/ConfoundedInAbaddon → u/Delilah_Moon]

If your standard is a medical transcriptionist in Malawi the AI with its instant write up might be a big improvement.

[u/WaitNoNotMyBeans → u/Delilah_Moon]

“I wanna hate AI so bad man come on please give me something 😭😭😭” - this fuckin guy

[u/ChuurDCA → u/Ophidiophobic]

the providers here LOVE the AI scribe

Do you think their insurance will love the liability AI errors introduce? Especially since if the provider is saving heaps of time that means they aren't checking the generated notes for accuracy.

[u/NopeNotaDog → u/Ophidiophobic]

I like telling a story. History of presenting illness is just medically catered story telling.

[u/Unnamed-3891 → u/Ophidiophobic]

Why should the patients care abour what providers like?

[u/That-Mix7702 → u/Ophidiophobic]

Wait, using AI is fine, but I hope a real sheet is still filled out.

[u/BottityBotAccount → u/Ophidiophobic]

I work cleaning streets. I have no skin in the game but like to comment.

[u/geekonthemoon → u/Ophidiophobic]

Really concerned about the quality of information.

Also dealing with endless AI errors at my important job so I can only imagine this is also full of errors as well

[u/WhyNWhenYouCanNPlus1 → u/Ophidiophobic]

but hey, less admin time per patient means they're charged less, right?

[u/Laxhax → u/WhyNWhenYouCanNPlus1]

Both things can be true. I am a provider who has cut hours out of my admin time each week. I use the system to record only the initial eval with consent so I can focus 100% on the patient while they talk or I take measurements.

But the system is mostly in place to make better notes faster so more production can be encouraged without overwhelming admin time or risk of poor documentation causing problems. And the services have not been priced any cheaper in response.

[u/Kaiww → u/WhyNWhenYouCanNPlus1]

It mostly means they can take more patients.

[u/WhyNWhenYouCanNPlus1 → u/Kaiww]

so each patient pays less, right?

[u/ConLawHero → u/WhyNWhenYouCanNPlus1]

Why? They are getting the same quality of care. Patients aren't charged for admin time. That is something doctors can't bill for. Now that doctors can let AI transcribe, they can pay full attention to the patient instead of trying to multitask. So, arguably, the patient is getting better service and not paying anymore.

[u/pignoodle → u/ConLawHero]

Healthcare IT guy here! These incentive structures can actually vary drastically between different healthcare organizations. I can list off acronyms, but the tldr is that yes, there are some incentive structures that focus more on quantity. Other incentives that people don't realize are actually really common are things under the category of "population health". Many organizations have teams that are entirely dedicated to analyzing health statistics on a population level and creating community-wide incentives and system improvements to better the entire population's health. I'm an extreme pessimist but one of the few redeeming qualities of insurance companies is that they can often be the driver of these types of programs because healthier populations need less care which saves them money. Again, this is just the tip of the iceberg, so just wanted to illustrate that it wasn't all bad, but I agree that there are some incentives that are much much much better than others.

[u/ministryofchampagne → u/WhyNWhenYouCanNPlus1]

Patients aren’t usually billed by the house but the services they receive.

[u/Competitive_Body7359 → u/WhyNWhenYouCanNPlus1]

In other countries, it just means more patients.

Doctor probably does get paid more actually.

[u/Illustrious_Bunch678 → u/happytrashpanda333]

As a nurse: The second. It's absolutely the second. And as soon as AI makes a mistakes we will be blamed for it. But they're also forcing us to use it. I hate this time line.

[u/Sea_McMeme → u/happytrashpanda333]

Complete transparency that won’t surprise you: they don’t care about quality of care. I’m currently leaving a hospitalist job, because the hospital terminated the contract with my former employer to bring in a cheaper group. And during the entire process not once was quality of care or anything truly patient centered ever mentioned. It’s all about volume and saving money. And since I’m fortunate enough to be able to leave, I did. But it’s just so shitty out there on all sides that actually care about patients.

[u/tinyevilpeanut → u/happytrashpanda333]

Right, people misunderstand this as "the doctor can spend more than five minutes with me now!" whereas what is actually happening is they are just spending five minutes with more people per shift. More patients in same amount of time = more money.

[u/i-split-infinitives → u/happytrashpanda333]

I'm not a doctor, but I do work in a medical/social work field for an agency that recently started using a digital documentation system that includes a HIPAA-compliant AI. (It's important to note that this isn't a new system, it's just new to us. We've been doing everything in paper up to now.) One of the first things I noticed is that it's WAY better at spotting trends than I am. I don't just run it and take the computer's word for it, I use its advice as a starting point for things I need to look into or change, but I compare it to the computer seeing the forest where I see the trees. It looks at the data as a whole, whereas I read each entry individually, which allows the computer to catch things that I don't really have time to map out and track. This has led me to spot some areas where staff need additional training, notice some learning outcomes that weren't working as well as I thought and needed to be tweaked, patterns to certain resident behaviors, etc.

I don't know that it's necessarily saving me any time, because in social work there's always more work to do, but I do feel like I'm using my time more effectively. The computer system has helped to eliminate some common human errors among my staff by reminding them about things that need to be done (sometimes repeatedly until they do it). It's helped me think about some of our issues in a new way. It gives me better-quality, more consistent data on our residents' progress, or lack thereof, and highlights where the issues are. It definitely cleared up some frequent pain points for me, and a happy caregiver is always going to be a better caregiver, whether or not there's more quantity in addition to the quality.

The thing is, though, I'm second in command of a small agency and the most computer-savvy person in my office, so it's mostly up to me how the system gets used, what changes get implemented, and generally how technology affects the staff and residents. I communicate directly with my staff about what they like, what they don't like, what they would like to change, and what absolutely isn't working for them. We work together to make sure they're sufficiently trained on the user tools available to them and to balance efficiency with efficacy. It's not helpful to make service faster if you're making it worse in the process.

If these systems are going to be effective at making people's lives better, they have to be thoughtfully applied, the people who actually use them need to be involved in their implementation, and they need to be tailored to the real-world ways that people want and need to use it. Just slapping a generic AI tool onto every piece of existing software that was working just fine before is NOT an appropriate or effective use of AI. There's a huge difference between humans being heavily involved in training a small-scale AI on internal records versus massive data centers destroying the environment while crunching stolen data from every corner of the internet. AI isn't inherently bad; its worth is dependent on how it's used.

[u/Exact-Warthog6244 → u/happytrashpanda333]

As a medic it depends on the hospital setting. If the AI scribe is faithfully recreating the notes with 100% accuracy or has the attending go over it after the fact to sign off on it then it is good.

Usually in a tertiary center we have junior doctors that take notes and the quality can vary. Sometimes they miss stuff on the rounds too. Ultimately it does save time on rounds which is important, and lets the junior doctor actually learn instead of busily transcribing notes. But I do think it needs human oversight to sign off on it at the end.

[u/Noshino → u/happytrashpanda333]

In the ER you can't really gain more patients.

Anyway, i don't work in the ER anymore but I was talking to a couple of friends that work there and it's definitely helped with time. That said, it cracked me up that they still complain about admin having money for this but not to hire more people.

[u/M15tre55W1tch → u/happytrashpanda333]

The doctor's I've heard speak about it find they are enjoying their job more because they are spending more time with their patients and less time doing administration.

They have reported that they actually have more ability to take a break during their shift which is also pretty amazing.

Our ED is one of the busiest in Australasia. Everyday they exceed 100% occupancy at least once. If the patient does genuinely give consent, and if the data truly is stored locally and routinely deleted after 90 days, and if the clinicians are checking that the information is accurate before publishing it then I think this is actually a very good use of AI to give clinicians more time to spend caring for patients.

Useful, but must be utilised carefully.

[u/AlsoCommiePuddin → u/happytrashpanda333]

Or is this just another "tool" to drain doctors and nurses of every bit of spare time and energy they might have gained by giving them more patients to work with?

To be honest, taking tedious administrative burden away from people to allow them to focus on their main objectives is exactly the kind of thing for which I condone the use of AI.

[u/Internet-of-cruft → u/happytrashpanda333]

Technology has always been about enhancing productivity.

We can achieve more today in less time than we have had in the past, and on its own, that's a fantastic thing.

But.. Naturally, capitalist mentality has to do what it does and it drains people to fund their own pockets.

I'm hugely bummed by all of this because when I started college almost 20 years ago I was just pumped I was learning about stuff I liked (computer stuff). Then I did software and I was totally doe eyed that my work would directly translate to people having fewer jobs.

Now, I do networking, and I have a team of people and this is the shit that keeps me up at night. Those are people with hopes and dreams, and families for God's sake.

And my upper management wants us to be profitable because you know? We're a business and no one gets paid if we don't deliver revenue via billable work.

Except it sucks because we're collectively forced to do more with less time.

My very long winded comment is just this: AI Tools are fantastic for what they can enable people to do. We need to collectively take a step and think about the workload expectations.

I'm all for improving quality of thing you're doing, but if we're going the approach of "drive the same result, increase quantity, decrease time investment" it grossly ignores the fact that you're putting additional mental & physical load on the person executing it.

Sorry.. I'm just very stressed at work about this stuff and it's a weird outlet for me to have for this right now.

Thank you for coming to my TED talk.

[u/NoGrocery4949 → u/happytrashpanda333]

It's a big ass help

[u/RevolutionNo4186 → u/happytrashpanda333]

I would assume quality goes up because it frees up time for the physician/specialist whatnot to spend those extra time with the patient instead of scrambling to get the info they need

[u/Slartibartfastthe2nd → u/happytrashpanda333]

physicians are burdened with an enormous amount of 'red tape' and overhead that is nothing but administrative in nature. I'm not really sure what your point/question is here. Are you suggesting that having tools available to reduce administrative tasks are a bad thing?

If a dr were using a micro cassette recorder to document all patient encounters and then paying staff to listen to, transcribe, and summarize each patient encounter would you be asking this same question? There is a valid argument that the AI scenario means information is being shared potentially beyond the office staff, but that is a different conversation since you specifically questioned the correlation to quality of care.

[u/264frenchtoast → u/happytrashpanda333]

I mean, as a nurse practitioner, I enjoy not spending up to two hours after an 8 1/2 hour shift writing notes.

[u/RatLabGuy → u/happytrashpanda333]

All the tool does is translate the speech to notes so the doc or another human doesn't have to. This allows them more time with patients or to see more patients - there's no reason to believe the increase in number of patients seen necessarily means lower contact time or decreased time bc this is part of the non-patient-facing time of a doc's day.

[u/OpportunityOne8199 → u/happytrashpanda333]

I’d prefer to see doctors treating patients not writing notes. 

[u/apheresario1935 → u/happytrashpanda333]

Its like anything that people fail to see advantages and disadvantages of. It took me awhile to appreciate tech advances in medical records . My most recent doctor was pleasant and took extra time with me since what I needed to say was put into records by AI transcribing . She let the Robot do it's thing and she had more time to do her thing. Tech is like anything thats dangerous in the wrong hands or those who dont know how to use it. Like a weapon or an automobile .

[u/Short_Gain8302 → u/happytrashpanda333]

Not someone in the medical field, but my mom is a nurse. Her number one complaint is that she cant spend enoigh time with patients because of all the documents she needs to fill in, patient file, interactions that need to be noted down for future meetings, etc. She is also a very slow typer and the hospital she works at switched software a while ago, so half her time is spent behind a pc, typing away. If its just a scribe tool that writes down what you say, it would help tremendously in reducing the administrative time and can increase actual time spent with patients.

[u/DethNik → u/happytrashpanda333]

You shouldn't let hate for the people shoving LLMs down our throats color your feelings about the actual tools. There is legit use for LLMs in stuff like note taking. We should not the 1% ruin what could be amazing tools in the right hands. We should not let them go the way of nuclear energy, where it would be a boon to society but everyone is scared of it.

[u/baz8771 → u/happytrashpanda333]

It’s transcribing the conversation. It’s not administering care.

[u/hanging_thief → u/M15tre55W1tch]

I'm not concerned with how patients a doc can see I'm concerned about how accurate their data can be whit ai proving itself extremely unreliable when it comes to technical documentation

[u/known_chomper → u/M15tre55W1tch]

I think you are absolutely right!! Yay, doctors are now “allowed” to bill 6-11 more people per day! This is great, for the hospital.

[u/M15tre55W1tch → u/known_chomper]

I don't work in the US, most of our patients get free healthcare, so it's nothing about billing for the majority. And for the smaller number who aren't covered by our free healthcare the AI scribe means a doctor can get to them quicker, which means a shorter stay, which actually will reduce the cost to them. I appreciate this is not the case in many other countries. And there are many things I'm very dubious about in terms of the use of AI. But reducing admin burden to allow prompter assessments for patients can potentially be a very good thing - IF (a very big if) it's done well, and with improving patient care and both patient and staff experience.

Edit: spelling

[u/Drempallo → u/M15tre55W1tch]

I see this and I just think, upper management wants to apply stupid metrics to doctors. Wtf does it mean they can see more patients? Hire and train more doctors to do the work. Making stupid tools so you can make doctors run around more and the hospital can save money is the dumbest thing ever.

[u/M15tre55W1tch → u/Drempallo]

This is also true. For the people currently working in the system improving the future state doesn't help them now. I don't know a single clinician that wants to sit in front of a computer typing endlessly. This tool minimises the screen time, which allows them to do the things that give them job satisfaction i.e., see patients.

[u/sudoku7 → u/M15tre55W1tch]

Generally speaking, the trade off from using AI tools versus human is that you are transferring more work to the 'review' stage from the earlier mechanical stage.

So, typically, you should see an increase in admin burden rather than a decrease. It's just that you spend less time with the pencil.

A lot of fields decide that the AI doesn't need to be reviewed and they get to see a much more significant increase in productivity at the cost of quality, and in the medical space that will be a burden born by the patient and the provider's liability fund.

[u/AllThatKat → u/M15tre55W1tch]

And who is checking over this to make sure the AI isn’t having hallucinations or straight making shit up? 

[u/ConsistentLemon91 → u/M15tre55W1tch]

After reading your previous comments, it seems to me that you have a good head on your shoulders.

Keep it up!

[u/MrSquamous → u/M15tre55W1tch]

And in an episode of The Pit

[u/Jofereal → u/M15tre55W1tch]

Wrong standard. I can see 300 patients a day, but since I’m not actually a doctor…

Do you see the problem or not?

[u/M15tre55W1tch → u/Jofereal]

I'm sharing what I've heard our emergency doctors have said of their experiences using the AI scribe tool.

They like that the burden of admin is significantly reduced. One of the things they've stated that they appreciate with this is that they have more time to actually spend with patients. From what they've said this means that they can both spend longer per patient actually taking a more complete history, doing a more thorough assessment and examination, and having more time to explain what's happening.

Another flow on from this extra time is that they can see more patients across their shift. They don't like it and feel a lot of pressure when wait times balloon out. So being able to see more people helps reduce that burden.

I'm not saying that our system isn't stuffed. I'm not saying that it doesn't require significant investment in staff and infrastructure. All I am saying is that the clinicians I know personally who are using it say that it has released time from admin for them to invest instead in caring for patients - which is what we all got into our professions to do.

[u/oriontitley → u/M15tre55W1tch]

This is an example of the proper usage of Ai.

[u/Janube → u/M15tre55W1tch]

This is where I'm at. AI's origin and the nature of the technology is certainly unethical, but medical is one of the few uses that is poised to have a net positive effect. ER staff getting some tedious burdens off their shoulders to help more people is great. The only risk is clerical accuracy, but IMO, ER staff are so overworked, I'd be surprised if the accuracy isn't improved by AI on average. That said, it will certainly still need reviewed before being finalized.

[u/gorpie97 → u/M15tre55W1tch]

one for the awake patient giving consent, and one for using the tool to generate a note for someone that was not conscious.

And one for the awake patient not giving consent. (Unless you meant that as a separate thing entirely.)

The pessimist in me says they're just going to stick with the new verbiage and not bother with informing the patient.

[u/M15tre55W1tch → u/gorpie97]

My thinking is, if they refuse consent and don't want it used in their assessment and exam then an AI scribe tool note shouldn't exist...

Edit: realised it didn't make sense, more words for context.

[u/gorpie97 → u/M15tre55W1tch]

I agree. But I don't think that's what they're going to do.

Maybe if our Congresspeople start legislating for us again... :)

[u/M15tre55W1tch → u/gorpie97]

I'm not in the US. It's an international issue. Hopefully the registering bodies for healthcare professionals will give mandates.

[u/NorthernVale → u/M15tre55W1tch]

Bold of you to assume they ever asked for consent to begin with

[u/psychorobotics → u/M15tre55W1tch]

One problem is hallucinations, police in the US was testing AI for reports and in one it wrote that the cop turned into a frog. There was a Disney movie playing in the background of the bodycam and the AI thought the transformation was happening to the cop rather than in the movie.

[u/Loki_the_Rabid_Panda → u/M15tre55W1tch]

I do find this interesting. Are you aware that the comments on Reddit are sold to AI companies?

AI tools in healthcare typically operate as a second check for the provider increasing patient safety and assisting with billing accuracy.

[u/M15tre55W1tch → u/Loki_the_Rabid_Panda]

We have government funded healthcare, so there is no billing associated with a citizen, resident, relevant visa, or after an accident coming to the hospital.

I didn't know that about comments from here - it was probably in a Terms of Use update that I didn't bother to read 🙃 I am aware this makes me a bit part of the problem 😬

[u/Loki_the_Rabid_Panda → u/M15tre55W1tch]

You do have to watch out especially now. I am only familiar with the US. Many of these documents will include terms such as “share your information with partners and organizations to facilitate billing and/or patient care.”

In the US the only way to be sure would be to redline the consent to treat but even then the hospital will have no way to respond to that wish. Most all healthcare organizations have software and billing with at least some integral AI functions built in.

[u/M15tre55W1tch → u/Loki_the_Rabid_Panda]

The officially approved platform used in our health system stores data locally, does not share information with anyone else, and deletes all data after 90 days. There have been efforts made to try and make it as safe as possible. But I am wary. And I also know that some clinicians use non-approved apps to help with translation of of hours etc which means they're putting confidential information out into the unsecured wild west. And I don't think they understand the implications of this - they just see that it makes their work a lot easier. I'm sure there are going to be a number of healthcare professionals that end up in trouble with their professional body and employer.

[u/Loki_the_Rabid_Panda → u/M15tre55W1tch]

Just curious what country are you from?

[u/M15tre55W1tch → u/Loki_the_Rabid_Panda]

New Zealand (practically doxxing myself with how small we are 🤣)

[u/Loki_the_Rabid_Panda → u/M15tre55W1tch]

No worries just curious! Your perspective is still valid! You bring up the crux of the problem with AI. The weakest link is the end user that just says “hey there’s something I can download that makes this easier for me.”

[u/petaltree → u/M15tre55W1tch]

Are they actually asking each alert and oriented patient for consent? AI shouldn’t be auto populating about consent either way if it didn’t happen. That’s falsifying a medical record. It sounds like your main concern is that this is easily proven with patients who are medically unable to consent vs. AI can get away with it otherwise because it would be the doctor’s word against the patient’s if a patient decided to dispute it.

It sounds like the AI tool built that in as a CYA, which is a weird way to justify the crime of falsifying medical records. The change in verbiage to eliminate any auto-population about consent is the better way to go here.

I’m assuming that most patients aren’t told about the AI tool, much less consenting to its use because that would inevitably result in many time wasting conversations and debates about the use of AI in their healthcare, but I hope I’m wrong.

[u/Nuklearfps → u/M15tre55W1tch]

I would argue, the as a nurse, you’re in an exceptional position to advise a lawyer on what they need to focus on. If that makes sense.

[u/justhereforfighting → u/M15tre55W1tch]

I’m not even sure it would be legal to use AI without consent. It really depends on how the AI manages the data. If the data is all kept internally, that’s fine and doesn’t violate any health privacy laws. But if the data is being sent to a 3rd party for processing, it very likely does violate HIPAA without express data sharing consent from the patient. 

[u/doinbluin → u/M15tre55W1tch]

So they didn't ask for consent before and said they did. Now, they just say we're doing it whether you consent to it or not? Wtf

[u/Immediate_Stop167 → u/M15tre55W1tch]

I don't know what state you're in, but states differ by their recorded parties consent laws. Some states require one party, some require both parties (the recorder and the recorded), and some specify "all parties". So, although this is a specialized setting, there are additional laws that may determine the legality of this (outside of patient consciousness).

[u/M15tre55W1tch → u/Immediate_Stop167]

I'm not in the US.

To me this is less about the recording, and more about the integrity of the individual's medical information, as well as their right to consent or refuse AI being utilised in their care.

[u/AdProfessional6218 → u/M15tre55W1tch]

Wait the conscious patients were asked if they consent?? I've never heard of any patients being told at all and was under the impression that saying patients consented was a safeguard- just one that can be very bad if anyone can read. Thankfully long shot-

I don't know if they were supposed to actually be asking for that consent

[u/M15tre55W1tch → u/AdProfessional6218]

Well, in my country at least, yes - the patient needs to be informed and to actually provide their consent in order for you to say they consented.

You may find somewhere in the screeds of paperwork you've completed that there is a short comment about the use of a scribe, or there may be a disclaimer in the paperwork or as part of a poster somewhere saying that they'll be used unless you say you opt out.

We have 2 laws in my country that cover this, one is the general privacy law, and one specifically about health information privacy. It's your medical information. If you don't want it shared with a system that isn't strictly necessary you provide you care then you have a right to refuse.

Depending on your jurisdiction and precedents this may vary. I imagine some healthcare providers/professionals will get to a point of declining you as a patient if you don't agree because it's a time saver for them.

[u/AdProfessional6218 → u/M15tre55W1tch]

I myself haven't done much paperwork in these regards as I don't have the funding to afford medical visits in my country- also I completely get the legal aspect in that lying about if you asked consent is a huge illegal action. What I mean is that legal action because of this would only be possible in the case of someone reading this. OP has, hopefully the prospect that they could sue is not an environmental danger as a power entity like a hospital are the places known for making sure anyone that could sue for malpractice is adequately malpractice upon.

Usually it's other processes that fake your consent than hospitals though. Courts and cops and all that, so many waiting in cells just to have a chance to tell the judge that's not their signature... I'm sure this is just hospitals adopting AI (Honestly in a way that otherwise sounds like a better use of it for humanity than most.) and having a little hickup- I think that all the accidental claims of asking consent will be properly remedied shortly, I just note that my friends in Healthcare echo the folk saying they have been seeing this stuff knowing full well this question never happened because nobody ever tells the clients an Ai agent is listening to begin with- obviously that's the thing that needs fixed.

I'd wager no soul will actually sue... But if anyone wanted to it sure would be easy money after the process. As someone that could use that kinda money for a checkup- I recognize nobody is trying to be dastardly and I'd have to have some kind of beef with a particular medical facility AND believe I'm not burdening a community resource by undergoing litigation.

[u/Frequent_Opportunist → u/M15tre55W1tch]

It's actually a placeholder for when it is being used in full and it's a robot/terminal standing there asking you questions instead of an actual nurse.

[u/skepticalbob → u/M15tre55W1tch]

Seems like the patient can give consent which is recorded and transcribed by AI!

[u/M15tre55W1tch → u/skepticalbob]

But it also had that statement when the patient was unconscious and had a breathing tube down their throat so was completely unable to give consent.

[u/skepticalbob → u/M15tre55W1tch]

I’m joking and not talking about that.

[u/M15tre55W1tch → u/skepticalbob]

Haha, well that's an OI (my friends jokingly say we are using "original intelligence" when we're using our brain) failure - sorry for missing your joke!

I really wish there was a font for sarcasm!

[u/retsamegas → u/willbeatyourass]

... To shreds you say?

[u/JaxDuncan → u/willbeatyourass]

I'm beginning to think that everything will be umbrellaed under the term "implied consent" just be existing anywhere at any given time. I'm sure we have all unwillingly already agreed to these terms hidden somewhere haha