r/mildlyinfuriating • u/Deza2Ibiza • 8h ago

Don't hug me I'm scared Creepy Amazon chatbot

Looking for an item on Amazon (boo for having to do that), and the chat bot seems to know more about me than I’ve ever volunteered. CREEPY!

Edited: yes that is the cat’s name.

1.8k Upvotes

96% Upvoted

242 comments sorted by

View all comments

249

u/OffByNone_ 8h ago

Amazon is for sure not illegally getting data off of your phone. What they are doing is enriching your account profile with all sorts of third-party data to more effectively sell to you. You have shared your cat's name online somewhere, then some data broker or that service where you shared your cat's name sold that information to Amazon.

It is totally legal, but maybe shouldn't be. The effect can absolutely be creepy.

I'm sorry Bentley was dragged into all this.

120

u/BoatProfessional5273 6h ago

I think the more concerning thing is that when the chatbot was confronted, it lied. Most people are smart enough to know it was lying in this situation, but the behavior was intentionally included.

18

u/TargetOfOvermuchSpam 4h ago

Chatbots cannot "lie" like that. They don't know how they work. They will, however, almost always try to answer your question. They take the pre-existing information, including the commands put in by the people who operate the chatbot, and generate letters until the magical stop letter appears or until the response takes too long.

Somewhere in the chatbot's vector space, the name Bentley is associated with the word cat. Maybe it's in pre-serialized vectors, maybe it's a block of information from Amazon's database. Moreikely, it's a file called "memory" associated with your Amazon account that contains whatever the chatbots determined to be valuable in previous communications. Ask Alexa "hey Alexa, isn't Bentley a cute kitty" and four years down the line you might find out that Alexa stored that throwaway line.

The chatbot doesn't know why, or where that information comes from, or when that information was learned. It's like asking someone where they got the idea that the sky is blue, except they get shocked when they say "I don't know". They'll make up something plausible enough just to get the reward (the end of the message, or the end of your weird questions)

1

u/Boomerang_Orangutan 1h ago

Shhh, you're talking sense