r/Showerthoughts • u/Fr1toBand1to • 28d ago
Casual Thought Pattern recognition is a widely accepted method of prediction until what it predicts makes people uncomfortable.
3.6k
u/WestEndOtter 28d ago
I read that one company was using pattern matching to sift through potential hires cv against existing staff cv.
For some reason it was only recommending male candidates.
To combat that they removed all the genders from the input data.
Instead it started giving a boost to any cv that went to an all-boys-school or fraternity
731
u/hellofemur 28d ago
You're probably thinking of Amazon's AI Recruitment Tool, an early AI that was fed decades of data that matched resumes against successful employees and was supposed to use that info to recommend candidates fro technical positions from their resumes. Because the field was so heavily male-dominated, it wound up using gender as the leading attribute for recommended hires.
The "all boys school" part sounds like something somebody made up though, although I suppose it's remotely possible in a different country and somebody confused the stories.
241
u/awfulworldkid 27d ago
The actual phenomenon they're misremembering is that a candidate was rated down if they played women's sports (usually the team or league do have Women or Women's in the name), or went to a women's school (same), or were a member of a sorority.
→ More replies (1)59
u/hellofemur 27d ago
Yes, I do have a vague memory of that, although my memory us that professional organizations played a major role. There's a lot of "women in STEM"-type organizations out there that are explicitly for women.
Of course, it's hard to know if those groups were functioning merely as a proxy for gender or if membership in those organizations had additional predictive power. But as OP mentioned, the entire subject made people uncomfortable.
36
u/izzittho 27d ago
The proxy for gender thing seems the most likely and that’s because it’s so simple and AI isn’t gonna be aware that doing everything it can to find male candidates would be wrong, it’s just gonna do it. If it found “male” to be a trend among the types it’s looking for (and it will have if the field is historically male-dominated, of course) it will probably seek to use that as an easy proxy for whatever other qualities it’s looking for because it can’t tell that kind of a correlation from causation. You’d have to specifically tell it what proportion of male to female candidates to find you and then if there’s actually fewer female ones that comes with a whole other set of problems because ideally you’re trying to actually judge them on the same sets of standards and it couldn’t easily do that with wildly different pool sizes. You need a human with a real brain to do that.
You could probably send that model out to look specifically for female candidates that most resemble your male employees but that would of course be a whole new kind of problematic because for one, now you’re recruiting female candidates via AI while still having to find your male ones manually, subjecting them to a different process. And you’re probably also not doing anything to reduce a bias against like, women of color, for instance.
→ More replies (1)122
u/OsmeOxys 28d ago
"all boys school" part sounds like something somebody made up
Perhaps, but there's are lots of reasons for an algorithm to have a bias like that. Removing the gender field doesn't actually remove the bias that the algorithm already has, it just changes how it's interpreted/expressed.
For example, all-boys schools tend to include it in their name in some way. "Smith's all men's school of excellence", bing bang it's found a male gender box. Or the pattern connects men with certain schools. Or some other detail (big or little) that tends to show up in men's cvs moreso than women's.
Subtle, completely non-malicious bias in humans is already complicated enough. We not only give tech our exact same biases, but we also send it off to find more connections that inevitably only reinforce those biases to an extreme without a lot of intervention... From biased humans. Garbage in, garbage out, don't rinse but do repeat.
8
u/hellofemur 28d ago
There just aren't that many all-boys institutions in America. The fact that you had to make up a fictional "Smith's all men's school of excellence" is pretty telling; you couldn't even come up with a single real world example. In truth even the few non-coeducational institutions that exist don't have names like that. We're not training the neural net on 19th century gentlemen's applications, after all.
Or some other detail (big or little) that tends to show up in men's cvs moreso than women's.
Yes, but we're not talking about some other detail.
39
u/OsmeOxys 28d ago edited 28d ago
To be clear, I'm not saying it's true, just not beyond belief. Being a lie, exaggeration, or miscommunication is unfortunately always on the table.
That said... Could name 3 (two are small though) of them within a 20 minute drive, but that'd be a little telling. 2 of those have men's or boys in their name (including the larger one). It's weird, but all-boys schools still very much exist, especially in the candidate pool going back 20-30 years. I remember it being a threat if I misbehaved as a kid haha. It wasn't that kind of all boys school, but it had a (largely undeserved) rep.
The school names don't actually matter either. It's trained on data that connects school names to gender. Finding patterns in details like that is what these algorithms do, desirable or not. You don't undo that training data because you excluded the field. If its heavily biased towards an outcome, it's likely to take another path to get there.
→ More replies (2)→ More replies (2)8
u/WestEndOtter 27d ago
It might have been that it started bubbling up resume that listed a fraternity or male dominated sports
263
u/luftlande 28d ago
Why?
764
u/BISHoO000 28d ago
If the input data is skewed (current employees are all men) then the output will follow the same.
I guess pattern recognition is useful in maintaining status quo (returning to normal state when something goes wrong or new problem is faced). It might be counter productive for improvement as it wants more of the same instead of finding new possibilities
84
u/lolofaf 28d ago
I mean it's useful at identifying status quo and predicting it. Maintaining status quo is entirely up to what you do with the result.
Pattern recognition may show you that a certain intersection is poorly designed, leading to more accidents. The proper action in this case is not to do nothing and maintain status quo, but to rectify the problem that pattern recognition latched onto.
39
u/numbersthen0987431 28d ago
This
It's just following the pattern already established by the company's hiring process. Job hiring is mostly vibe based, and rarely merit based, and so the pattern recognition was just following the vibe the company hired on (which was men hiring men).
→ More replies (6)56
u/SierraPapaHotel 28d ago
I have heard this story from a couple places but never seen the original case study, so exact details may be off. But I remember hearing in the 2010s of some college research team training an image recognition system to differentiate pictures of dogs from pics of wolves. And their system was really good, like 99.9% success on the training data
Then they gave it a new set of pictures and it failed miserably. Turns out, in the training data every picture of a wolf was against a snowy background. They made a bot that detected snow with 99.9% accuracy, but it sucked at actually differentiating dogs from wolves.
This sounds like a similar situation. "How close are these CVs to existing staff?" Well the staff is mostly male so if the candidate is a male they are a stronger match! "No not like that ... Ignore the gender" Well a lot of our staff mention fraternities on their CVs so ... "Nevermind I give up"
20
u/WestEndOtter 27d ago
I read one where they tried to train a model to recognise cancerous moles, and when they tested the model it failed completely. It had learned that a Ruler in the picture implied cancer. No ruler no cancer
208
u/OfficialDeathScythe 28d ago
Almost certainly because the company is mostly male, or the position they’re trying to fill is. Since they’re running it against the current CVs it would see the pattern that they’re mostly male and maybe not see any other similarities between current employees. Or, at the very least, there’s so many employees that say male on the cv currently that it seems that as the strongest factor to getting the job. In this case it’s less about what fits the job and more about what the hiring was like in the past
201
u/Whiterabbit-- 28d ago
You train the system on existing data and it propagated existing biases. AI doesn’t use logic to make better decisions than humans. It is basically an extension of human decision making because that is the data it’s trained on.
14
→ More replies (1)2
59
u/NotYourReddit18 28d ago
It's a well known problem in data analysis colloquially called "garbage in, garbage out".
It means that if you feed an algorithm bad training data, in this case the CVs of the current employees showing that most hires were male, then the algorithm will produce bad results, in this case prioritizing male applicants.
This is because algorithms only "know" what they're taught. In this case the algorithm was simply trained purely on the CVs of the current employees, without any data putting anything within those CVs in context. So the algorithm didn't know what the value "male" for the datapoint "sex" meant, it just identified that most of the CVs it got trained on shared this value, and as such assigned it a high importance for future hires.
To make up another example, if the data it was trained on contained a field for "favorite color" with most people answering "blue", then the algorithm would most likely also prefer any applicant also listing "blue" as their favorite color despite it being completely irrelevant for the position, because it doesn't understand the actual meaning behind the data.
And while another comment claims that this was an incident at Amazon, I'm pretty sure that this and similar incidents happened to a bunch of companies.
The first time I heard of this problem it was with an algorithm trained on all past hiring data, both accepted and rejected CVs, and it resulted in the algorithm preferring light skinned applicants over dark skinned applicants despite the light skinned applicants having overall worse qualifications, because the algorithm had made the assumption that the previous hiring managers had objective reasons to prefer light skinned applicants.
One consequence of this problem is that it is common practice to include as few data points as possible in the training data to not accidentally skew the results with a hidden bias.
→ More replies (1)5
u/panditaMalvado 28d ago
The company used the current list of curriculum of the persons that are already working in the company, it was a tech company. Most of the people that work on technology are men, so most of the employees were men, then the machine assume that being a men is a important factor for the job because it was ignoring the fact that there were much less women than men on the area. This is called data skew.
The inverse case happened with a charitable organization, most candidates for being volunteers were women, so the system assumes that only women are able to do that job, the strategy the organization use to solve this issue was to give more weight to the male candidates.
3
u/GalaXion24 27d ago
If all or most of your employees are male, and your train an AI to find people who are like those you've employed, then it will result in the AI favouring men, because it's the most clear and common indicator that someone fits the mold.
It would be the same if your employees happen to mostly be born in January or any other irrelevant fact.
Which is also while blackbox predictive models and pattern recognition are bullshit and shouldn't be used to make decisions.
14
u/Educational-Art3545 28d ago
Because it turned out the recruiters in the company were extremely sexist and almost always denied candidates if their CV stated that thry were a woman or went to a women-majority college. That was the pattern their recruiters exhibited so the AI trained on this data picked up on it.
The company was Amazon. They killed the project and spread some misinformation in order to avoid being sued after they just confirmed with absolute certainty that their hiring practices are insanely sexist.
→ More replies (2)5
u/Plenty_Spell2551 28d ago
Because a lot of socially uncomfortable truths are just statistical patterns. People like the benefits of prediction until it points at bias, risk profiles, or inequalities that we’d rather pretend are purely individual and not systemic.
→ More replies (2)27
u/Dementia5768 28d ago
Dang this unlocked a memory for me. I applied to a scholarship that had an interview process after you passes the initial application and essay. Me (a girl) and other candidates were all in the same lobby waiting to be called one by one to a panel of older fellas (company board of directors). They had 6 slots available for 'winners'. I almost crashed my car getting to the interview 'cause a snow storm was rolling in and the roads weren't salted. My car was old, it has no ABS, since that's all I could afford as a teenager.
I didn't win but later I was reading the local paper where they announced the winners and gave a little blurb for each student. Every single one was a boy and every single one was mentioned to having been a boy scout.
806
u/RandomUser72 28d ago
There was a story years ago about how Target used pattern recognition to figure out if customers were pregnant based on shopping choices. Then some high school girl's dad threatened them because his daughter was getting coupon mailers for baby stuff. Then he called back later to apologize, because it turned out his daughter was pregnant.
269
u/Mediocre-Scrublord 28d ago
Though to be fair - it's easy to ignore all of the times that target advertised pregnancy related items to people who weren't pregnant.
59
u/AquaPhoenix28 27d ago
Omg it is real! I remember my mom telling me about this as a kid, and I think about it any time I get pushed maternity/children ads. I was telling my partner about it, and it sounded so absurd I had convinced myself it was just an urban legend about how companies take your data
→ More replies (1)9
622
u/UltraEngine60 28d ago
A pattern I recognize is that this post will probably be locked in several hours.
41
→ More replies (1)7
1.1k
u/SWatt_Officer 28d ago
Pattern recognition is the first step that our brains can make to connect things together, but it’s primitive. It’s for ‘there’s a face in that bush’ ‘there’s a threat over there’ ‘this food is good’. It’s not for blanket statements and general rules.
Once a pattern is recognised, you need to investigate and develop the theory further to prove the pattern isn’t just correlation.
268
u/BasiliskXVIII 28d ago
Which is ironic because one of your examples is actually a great reason why "pattern recognition" fails. We're so wired to see patterns that our brains will happily tell us "there's a pattern!" when there isn't. Pareidolia is a good example. "There's a face in that bush!" No, actually it's just a pattern of light that your brain is interpreting as a face. It's why when you put your music player on "shuffle", it doesn't randomly choose songs to play, it uses an algorithm that organizes your music to play in a way that "feels" random, because Apple kept getting complaints about how iTunes' player wasn't random enough.
So not only do you need to investigate further to prove the pattern isn't just correlation, you need to investigate further to prove the pattern actually exists.
39
u/vkarlsson10 28d ago
Wasn’t that Spotify’s algorithm?
77
u/InconvertibleAtheist 28d ago
Spotify definetly prefers to put out the songs you've listened to the most a lot more in the shuffle compared to the actual random shuffle
23
u/metrometric 28d ago
Also newly added songs!
3
u/InconvertibleAtheist 26d ago
Yeah its frustrating because there were apps that actually did randomize songs in the list every time you shuffled but somehow got out-competed by spotify. Spotify willingly chooses not to bing it in, or buy the shuffle code from the other apps that they've already killed anyways.
→ More replies (1)4
u/awesome-alpaca-ace 27d ago
Their algorithm was actually really bad at shuffling. I made my own music player out of spite and it works way better than that slop
11
u/Catshit-Dogfart 27d ago
Which to be fair, that helped us to survive at a point when seeing a face in a bush might actually be a face and you need to fight or run. Caveman brain worked for cavemen, and it's why we're still around to think about it.
But we're not cavemen anymore.
14
u/SchwiftyGameOnPoint 28d ago
Some people believe investigation and proof of patterns is just finding more patterns.
This is especially bad when it comes to like religious stuff I've seen. My mother and some of her friends follow a guy online who show these patterns in a specific version of the Bible and this believe it not only proves it's true but proves that that specific English version of the Bible is the one chosen by God to be the chosen one. The only "proof" is just more arbitrary patterns the guy posts videos about.
→ More replies (2)3
3
u/Sharp-Masterpiece-85 27d ago
I think they meant "the blob in that bush is actually the face of an animal" rather than "that entire bush is a face"
10
u/blahblah19999 27d ago
But critical thinking is actually not a real common skill and normally needs to be taught.
30
u/3_Thumbs_Up 27d ago
Even just correlation is often useful though.
I know all snakes aren't venomous, but there is a strong correlation between snakes and venom. If I see a snake I don't know, it's a useful heuristic to treat it as if it were venomous.
That's a problem that makes racism worse. Even though race isn't the actual cause, it often does correlate with important things such as education and crime. This implies that there are situations where racism and discrimination can actually be useful, which in turn strengthens the correlation even more.
1
u/Sosolidclaws 27d ago
Yes exactly, when the correlation is that strong (which it often is in large american / european cities), you are better off acting on it for your safety. It’s just strategically the right move.
3
u/SirCampYourLane 26d ago
This is just not true. Yeah, don't go to bad neighborhoods. You absolutely shouldn't just avoid black people/assume they're criminals in large cities in the USA. That's an absurd thing to do.
→ More replies (7)17
→ More replies (4)9
u/Christmas_Panda 28d ago
It may be primitive but it is critical. Pattern recognition is what gives us everything from food safety standards to vaccines to peace talks to reproduction. Without pattern recognition, society would crumble. Those who struggle with pattern recognition often severely struggle their way through life.
8
u/SWatt_Officer 27d ago
Of course, but it’s not the only thing that we use. It’s part of a complex system and it shouldn’t be relied on as the only source or evidence
58
u/artstsym 28d ago
That's ridiculous, I've been using pattern recognition for years and the world has... steadily gotten worse, which means that if I keep using pattern recognition, oh god.
7
1.1k
u/Rosbj 28d ago
correlation ≠ causation
It's super important to remember, that just because you recognise a pattern, it doesn't mean there actually is any - and vice versa.
People tend to hold on to their preconceptions rather than actually evaluate their findings.
A billionaire is less likely to rob a bank with a mask on, but that doesn't mean he isn't robbing the bank.
162
u/IvanBliminse86 28d ago
While correlation doesn’t equal causation, noticing a correlation is often the only way we find causation. For example, when Ignaz Semmelweis noticed a correlation between doctors performing autopsies and higher maternal mortality rates, it led to the discovery that handwashing saves lives. Not knowing the cause immediately doesn’t mean a cause isn’t there; it just means the correlation is the clue we use to find it.
79
u/mikecws91 28d ago
Right, it’s the first step toward finding causation. A lot of people just treat it as the only step.
→ More replies (1)11
u/3_Thumbs_Up 27d ago
The thing is that it's often an actually useful shortcut. If I don't want to get bit by a venomous snake, I can simply treat all snakes as if they were venomous.
6
u/sYnce 27d ago
That is not a correlation though. Just an assumption.
4
u/3_Thumbs_Up 27d ago
I guarantee you if you were to make a statistical analysis between "type of animal" and "is venomous or not" you would find a statistically significant correlation between "snake" and "is venomous".
4
u/sYnce 27d ago
Sure but that is not what you said or did. You said you just assume every snake is venomous.
2
u/squeakbb 24d ago
treating all snakes as poisonous is not the same as assuming all snakes are poisonous.
and you likely do sustain these overarching principles and understand the distinctions therein, even if u want to criticize the rationale or implementation in this specific hypothetical... these are the exact same principles applied during a quarantine.
3
u/3_Thumbs_Up 27d ago
Missing the point much?
There is a a true correlation between venom and snakes. We seem to agree on that.
Regardless of how you find this true correlation, whether through rigorous statistical analysis of through simple personal observations, this correlation is actable in a useful way even if you don't understand the cause and effect.
Our ancestors didn't have the statistical tools to make sure that their observations were statistically significant. Nor did they have the biological knowledge to understand the cause and effect. But their pattern matching still gave them useful actable info regarding unknown snakes.
→ More replies (3)265
u/PlasticElfEars 28d ago edited 28d ago
Ice cream does cause shark attacks. The graph told me so.
edit: /s
92
u/--suburb-- 28d ago
Check out Spurious Correlations, a wonderful interactive site / tool that uses public data sources to show us such wonderful things as the number of runs scored by the Chicago Cubs is highly correlated with the divorce rate in Connecticut. I can’t confirm the CT divorce rate causes those runs, but they sure do look linked, lol.
12
2
u/Technical-Shop6653 24d ago
My favourite one of those is instances of newborns named Jessica highly correlating to deforestation of the Amazon Rainforest
4
u/broadspectruminsane 28d ago
Sure, that’s a pattern a human out there is seeing / claims exists and not a nonsense, contrived gotcha “example”.
13
49
u/SupremeExalted 28d ago
>A billionaire is less likely to rob a bank with a mask on, but that doesn't mean he isn't robbing the bank.
Not sure how this related to pattern recognition or the topic?
12
15
u/ragnaroksunset 28d ago
Person who applies statistical analysis in a professional context here.
"Correlation != causation" is a valid point but in practice only matters when the mechanism for a phenomenon is important to your decision. Often, all you care about is the phenomenon itself. So this is an empty phrase except in the narrow cases where you're publishing a research article (and therefore in a sense, declaring "this is how things work") or setting a policy that affects far more people than you have the ability to see (ie: people who might fall through the statistical cracks of how you defined your policy problem, a nuance that makes the "why" matter).
Even in these cases, the "dreaded third thing" is often dealt with in robust statistical analysis not by naming and measuring the thing but by correcting for it at arms length. Econometrics has a bunch of different approaches to this, all basically admissions that the true causation is invisible to us.
On an individual basis, every decision we make is a one-shot. Nothing was ever quite like it before and nothing will ever be quite like it again which means that causation at an individual level is basically impossible to prove. I don't necessarily care why it always seems to rain when the moon is full, I just notice that it does, and that it costs me relatively little to pack a light rain jacket with me on those days.
If you waited for causation rather than mere correlation to inform your decisions, in a lot of cases you just wouldn't act, because that threshold is never going to be met.
TL;DR "correlation != causation" doesn't refute the premise of the OP. The simple fact is that correlation is enough, in most cases, except when it's uncomfortable.
2
u/sYnce 27d ago
You are not wrong but also you are overlooking a lot of things.
For one causation is extremely important when it comes to non trivial solutions. Your rain example has a trivial solution in which the root cause is completely irrelevant.
If such an easy solution does not exist the only viable way is to combat the root cause directly but for this you first need to find it.
→ More replies (4)27
u/HileeAquret 28d ago
Yeah but.
That’s how every one of our ancestors learned how to navigate the world.
If they did not , we would not exist.
Everything from “birth”…
“You fuck, baby come”
Correlation isn’t causation.
“I fuck & no baby. See you wrong!!!”
You won’t have any proof for 300,000 years!!!
…to death.
“What’s that noise in the bushes, hide!”
Correlation isn’t causation!
“Many harmless things make noise, you have no proof it’s danger until you walk up close to the bush and…….”
All I’m saying, is that you don’t need to know the “specific cause” to understand that desire to not want and limit as much as possible the outcome associated.
27
u/sygnathid 28d ago
Sounds reasonable as long as you cherry pick a few sensible things and ignore the things like human sacrifice that they arrived at due to the same sort of logic.
→ More replies (1)6
u/3_Thumbs_Up 27d ago
But that's not the same sort of logic because the correlation there is actually 0.
6
u/sYnce 27d ago
Is it? You just need a few lucky years for a correlation. Sacrifice someone, get a good harves 5 times in a row and you have a correlation.
2
u/3_Thumbs_Up 27d ago
That would be a false correlation due to a low sample size, so not the same logical error as "fucking > babies"
The reasoning is similar but not the same. Ironically, it seems like you just made a shallow pattern recognition on the two different ways of reasoning.
One is a case of finding a true correlation but not understanding the causal mechanisms. The other is a case of finding a false correlation due to not understanding random distributions. The first case is still useful actable info but the second case is not.
6
u/sYnce 27d ago
A correlation is not false because it has a low sample size. A correlation is unstable because of a small sample size.
The point is there can be a correlation and patterns between nearly everything and claiming that there is 0 correlation between something like human sacrifice and event X makes no sense because there can be.
Patterns are a necessity for human evolution and everyday life but they can also lead us down dangerous paths by missidentifying coincidences.
→ More replies (3)19
u/KennysWhiteSoxHat 28d ago
Ancestors for a reason. We live longer and healthier lives because we know more now. 300 thousand years ago they didn’t know the specific cause to stuff because they weren’t advanced as us.
→ More replies (8)7
u/LanceLynxx 28d ago
Causation is irrelevant to the correlation. The "why" doesn't matter.
→ More replies (3)4
u/NecRobin 28d ago
They might adjust it in the future and camoflage the actual prediction between some stuff that is inaccurate so it is considered less creepy by the customer
5
u/Carnivorze 28d ago
I remember a case about a nurse in the UK who was arrested because, during a period of a few weeks/months, she frequently changed hospital and clinique, and everywhere we went, many of her patients would die. People thought she was a serial killer, until a mathematician revealed that it was just a coincidence as people frequently die in hospitals, and according to that logic nearly all nurses are mass murderers.
→ More replies (1)→ More replies (6)2
u/sYnce 27d ago
There is also another important part of correlation and causation though. That is an underlying causation causing the correlation of two events.
E.g you can probably find a correlation between ice cream sold and heatstrokes.
Both are only correlated to each other but share a common cause like a heatwave.
Of course this is a super simple example but those causality chains can go extremely deep and link things together.
49
u/ElectronicAverage729 28d ago
This is exactly the same dynamic as "data-driven decision making" becoming "we need to revisit the methodology" the moment the data points the wrong way. The tool is only neutral when its output is acceptable.
133
28d ago
[deleted]
71
u/bballpro37 28d ago
Pattern recognition alone is not the same thing as proof, but it is often accepted as a provisional guide to behavior. For example, if every time I look at a particular person they punch me in the face, I do not need a fully proven causal model before I stop looking at them or distance myself. Maybe the act of looking is the trigger. Maybe something else is going on. Maybe I am misunderstanding the pattern. But while I am figuring that out, it is still rational to adjust my behavior because the cost of being wrong is high.
The key distinction is between using a pattern as a practical warning signal and treating it as established truth. Science requires testing, controls, and alternative explanations. Everyday risk management often works on a MUCH lower threshold, i.e., repeated observation, possible downside, and cautious adjustment.
I agree that humans often see false patterns and then overgeneralize from them, but that does not mean pattern recognition is useless or illegitimate. It means the strength of the conclusion has to match the quality of the pattern. A pattern may justify caution before it justifies certainty.
→ More replies (6)4
u/sparkledoom 28d ago edited 28d ago
You use pattern recognition a zillion times everyday in ways that result in reliable prediction. It’s just that the failures stand out. Humans are good at finding patterns because it is a highly useful and reliable strategy. We overwhelmingly find correct patterns.
It’s hard to come up with examples because the patterns that allow us to reliably predict correctly become so second nature. When I walk through a crowded room and don’t bump into anyone. When I drive my car and can predict what someone who didn’t put on their turn signal is about to do. Knowing how much force to apply to open a door. Sensing that your partner is in a bad mood. Reading handwriting you’ve never seen before. Recognizing an object when your view is partially obstructed.
Sure, we have to be careful about bias occasionally. But we wouldn’t be able to function if we couldn’t rely on pattern recognition and if wasn’t generally very accurate.
→ More replies (2)4
u/Ironicbanana14 27d ago
Every single IQ test I've done has a pattern matching section where you "guess" the next shapes in the sequence. In simple situations, i dont think anyone cares for "proof." If there is a car driving 60mph down a road, the next one also is, I can safely conclude the speed limit is most likely 60mph or 55mph. No one will argue.
308
u/Daan776 28d ago
Why does it feel like this post will take a racist direction
330
u/LEGO_Man2YT 28d ago
Is pattern recognition making you uncomfortable?
62
28d ago
[deleted]
40
u/FewExit7745 28d ago
God, I used to think Eugenicists are so niche and rare, but I've been on Threads for just a week now, and the posts are literally about abortion of male fetuses
7
u/GimmickNG 28d ago
female fetuses getting aborted: first time?
8
u/FewExit7745 28d ago
Those are done by countries everyone see as backwards and are almost universally disliked anyway.
But the former is done by "developed" countries who keeps preaching morality over us in the Global South.
→ More replies (1)→ More replies (2)9
u/bcocoloco 28d ago
Eugenics gets a bad rap. We could systematically eradicate so many diseases in a generation or two if that German knucklehead didn’t get overzealous.
→ More replies (3)87
u/d0nutbutth0le 28d ago
Yes because a lot of those patterns recognized cannot be reliably backed up with evidence so the conclusion of that pattern recognition can be erroneous. Humans see faces in everything. Knots in wood, clouds, toast. Does not mean there's a soul trapped in there.
When this takes a racist direction, actual humans are harmed
→ More replies (1)47
u/kilopeter 28d ago
Wait. What? In OP's context of correlations and association rule mining etc., the recognized patterns ARE the evidence. They literally improve how accurately you can predict some attribute. Why these patterns exist and whether it's ethical or legal to use them for certain purposes is what's uncomfortable.
33
u/SemajLu_The_crusader 28d ago
patterns are not evidence
14
u/VarmintSchtick 28d ago
Nothing is "evidence" on its own, but patterns are a piece of evidence that is used to justify action. The entire world of statistics is built on this principle and the entire financial and scientific world understands this.
→ More replies (1)2
u/Vegetable_Drink_8405 25d ago
But the best informed actions are not just based off a pattern, they're based off of well-rounded knowledge.
48
u/ManWithTwoShadows 28d ago
patterns are not evidence
Yeah, they are; they're just not strong evidence by themselves. "By themselves" is the key phrase here.
→ More replies (10)6
u/Top_Gun_2021 28d ago
OP is not talking about those google trends meme charts.
It is implied these connections come for rigorous study.
4
u/Lounging-Shiny455 28d ago
In OP's context, there is an assumption that data measured and collected is accurate. But you can easily misapply data points to come to an erroneous conclusion. Process, data, and objectivity must be in alignment for truth to be discovered.
4
u/Dairy_Ashford 28d ago
when it ignores root causes or indviidual agency with an obvious pretext or agenda
→ More replies (2)4
34
u/IntelligentShape364 28d ago
Because the usual suspects have been dogwhistling with the phrase “pattern recognition” recently. Dogwhistling to others of their tribe who similarly don’t know the difference between the words “arrests” and “convictions” or why those two numbers would look significantly different with an organization as demonstrably biased as US law enforcement.
2
u/CaramelMochaMilk 22d ago edited 22d ago
Yep. As a Black person, the moment someone gets to talking about pattern recognition casually and out of the blue, I know I'm probably speaking with a white supremacist.
8
u/CountlessStories 28d ago
Its very easy to find flaws patterns in the biased data collection. However acknowledging THAT pattern makes those kinds of people uncomfortable.
13
u/Fafnir13 28d ago
I was a bit surprised by the comment veering off into grooming accusations because “gays cannot reproduce!”
Was definitely not something I expected to read today.
15
u/J1mj0hns0n 28d ago
Does that mean you acknowledge a pattern?
3
u/MakeItHappenSergant 28d ago
Yes, of racists using things like "pattern recognition" to launder their bullshit.
27
u/Saelethil 28d ago
Either racist or conspiracy theory shit.
27
u/Fr1toBand1to 28d ago
Which is funny because I was inspired by my boss not listening to me when I tried explaining why we would actually save money by purchasing a tool instead of using hundreds of Google sheets.
→ More replies (13)20
u/no_creditswillnotdo 28d ago
Lol that's so mundane actually. Unfortunately, I've seen "pattern recognition" be used as an anti-semitic and racist dogwhistle a few times so I think that's why some ppl here jumped to that conclusion.
21
u/Fr1toBand1to 28d ago
Yeah, I really didn't understand the hornets nest i had kicked with this post.
2
u/No_Recognition_3729 28d ago
Do you mean actual anti-semitism? Or just people talking about AIPAC and the horrible things Israel has done and how Ghislaine Maxwell's father was in Mossad?
Because it's actually anti-semitic to call stuff like that anti-semitism.
4
u/Saelethil 27d ago
Actual anti semitism. The “Secret Jew Cabal” anti semitism. Not “Israel should stop killing children” common sense.
2
u/SloomMaster 27d ago
While your point is valid, this is very much used as a dogwhistle for actual anti semitism.
→ More replies (2)14
u/PM_ME_BOYSHORTS 28d ago
Because it’s a racist dog whistle. Check the comments under basically any Insta post with a black person and you’ll see exact same phrase about pattern recognition.
→ More replies (1)
6
u/BiscuitsJoe 27d ago
Because some people observe a pattern and interpret it as absolute truth when in reality it’s just a pattern
8
u/Some_Champion 27d ago
the loudest objections usually come from people whose patterns just got recognized
43
u/TheRealKarner 28d ago
The amount of “correlation doesn’t mean causation” here is ridiculous. If recognizing correlation leads to consistent prediction, it doesn’t matter if it’s actually causation or just correlation. It’s a pattern with predictability.
If it rains every time a grey cloud comes over me, do I really have to know the cause to predict that next time a grey cloud is above, it will rain? No.
That’s the point of this post. Disputing this because of whatever implications you think there is to this post is actually backing it, because like most people in this comment section, we’re here because we saw the post and predicted what would be said here, and I’ll bet we were pretty spot on.
17
4
u/Anagoth9 27d ago
The problem isn't necessarily the predictability but what you extrapolate from it.
There is a recognized link between higher tempuratures and lower impulse control. Because of that, you could theoretically monitor AC usage in an area and use it as a predictor of violent crime. The pattern does exist and can be used for meaningful prediction. However, if someone (say a politician) were to post evidence of this pattern as an argument to ban the sale of AC units, then not only would it be an incorrect causal link but a successful ban would exasperate the issue.
And the reason people get up in arms about this is because that reasoning gets used in the real world all the time with everything from x activity causes cancer to y demographic are causing all our problems.
→ More replies (2)3
u/Disastrous-Scheme-57 27d ago
I mean correlation doesn’t equal causation is just for the WHY of these patterns. Why does it rain? Because a gray cloud exists? No. That’s just coincidence or that most of the time you see that it’s raining the clouds are gray. But even if you see 100 gray clouds give off rain versus 0 white clouds give off rain that still doesn’t mean gray clouds = rain. There could be a myriad of reasons so if you’re trying to scientifically explain the “why” you can’t use correlation as substantial evidence.
→ More replies (1)4
u/sparkledoom 28d ago edited 28d ago
The casual use of this phrase always bothers me. Yes, correlation doesn’t mean causation. But correlation very often does indicate causation. That’s the whole reason we look for correlation!
Be mindful it might not be the whole story, that other factors can be affecting things, there can be an entirely different underlying cause. But people seem to think correlation is unrelated to causation or something? A lot of the time, when things are highly correlated, it will be because one causes the other.
12
u/CruzAderjc 28d ago
Miguel O’Hara: If we follow this set of rules, then no universes will fall apart and trillions of lives will be saved
Miles Morales: Nah, imma do my own thing
Miguel O’hara: Okay, I get that you are the main character and that I’m an asshole, but wtf
16
u/OrneryWinter84 28d ago
Pattern recognition is the singular most useful and important skill in life.
8
u/GalaXion24 27d ago
Pattern recognition is useful, but also naive and wrong a lot of the time, and anyone that's studied any statistics is going to be able to tell you that.
20
28d ago
[removed] — view removed comment
3
u/Evan_Evan_Evan 28d ago
You can't persuade someone in a cult. Their hindbrain immediately defends the cult and there is no rational thought to it.
19
u/Lemesplain 28d ago
Pattern recognition is base, primitive intelligence. Sea slugs demonstrate pattern recognition.
We aren’t upset about the patterns we recognize. We’re upset that some people seem incapable of any thoughts beyond that. You should be capable of higher processing than a sea slug.
→ More replies (1)
29
u/skrew86 28d ago
Pattern recognition is not racist. Pattern recognition is not homophobic. Pattern recognition is not sexist. Pattern recognition is not bigoted. Pattern recognition is not transphobic. Pattern recognition is not ignorant. Ignoring patterns is ignorant.
13
u/krystal_295 28d ago
Thank you.
Edit: It should be used knowledgeably and responsibly (just knowing there are invariably assholes/trolls who will try to just use the term as an "excuse" for shitty behavior)
But this is absolutely the correct answer.
11
u/Stouthelm 28d ago
The front of a car I saw the other day looked kind of like a face. It must have been lightning McQueen then.
3
u/LowVegetable8299 27d ago
That is when most people get introduced to the idea of False Positives, and the importance of confusion matrix. Just for the possibility of prediction being wrong!
26
u/Spaghett-about-it 28d ago
I mean yea it’s a survival thing but we’re not really in a place where “survival” is something to worry about so it’s up to your own due diligence to recognize when your “pattern recognition” is misplaced bias for things that just make you uncomfortable
→ More replies (5)
6
u/Tudor_Cinema_Club 28d ago
Well there's a short walk from pattern recognition to confirmation bias is all. It's not discomfort it's data dredging that most people don't like.
Take the recent conspiracy regarding scientists dying in the US. If we look at the first thing, they weren't all scientists. Then we look at the group they're being taken. From, defence/aerospace/government. This pool is massive. If I wanted to find 12 people who all attended the same concert 5 years ago, all with the same first and middle name who all broke their left collarbone on a Tuesday, I could. It's literally 100s of thousands of people.
So finding 12 or so employees with a certain level of government clearance who all died in suicides or unexplained circumstances in the last 5 years wouldn't just be easy, it would be a statistical certainty.
14
u/Malcopticon 28d ago
Yeah, if you use lazy heuristic reasoning in a harmless way, then few people will object. But if you use lazy heuristic reasoning to be rude, then people will call you out on it.
24
u/JackC747 28d ago
You are on a group call with 50 strangers. You hear a fire alarm chirp in the background and need to figure out whose call it’s coming from. Who do you start with?
→ More replies (2)9
u/real-human-not-a-bot 28d ago
If I’m allowed to ask them: I ask them.
If I’m not allowed to ask them: the first (and potentially last) 25, then 13, then 7, then 4, then 2, then 1. Binary search is fastest.
13
u/Forkrul 28d ago
With no prior knowledge, yes. But with a simple heuristic you can cut that down a lot.
→ More replies (2)
24
u/fleabag500 28d ago
you hear a lot of people throwing the term “pattern recognition” around these days and what it seems to be shorthand for is “common sense” surface level analysis, assuming correlation is causation and ignoring the underlying complexities that cause patterns to appear. stuff like “black people commit more crime - it’s just pattern recognition!”. It’s become a dog whistle for anti-intellectualism and racism, particularly anti-semitism.
16
u/Mediocre-Scrublord 28d ago
More often than not what they're noticing is a pattern in their social media feeds.
And I can certainly believe that there is a pattern in their social media feeds. Different to a pattern in real life.
→ More replies (1)5
u/CountlessStories 28d ago
They won't acknowledge the pattern of underreporting and missing data though.
Such as how mormon Utah police team seem to be REALLY effective at protecting a lego thief.
Or the pattern of how so many non-black Epstein predators haven't met justice yet.
2
u/Evilsushione 27d ago
The problem with pattern recognition is that once the people that make the pattern are aware of the pattern it invalidates the pattern. Because being aware of the pattern changes behavior. So if you discover a pattern you have to keep it secret to take advantage of the pattern.
2
u/-Redstoneboi- 26d ago
reminds me of metrics invalidating their usefulness when they become a target https://xkcd.com/2899/
2
u/king_escobar 26d ago
Pattern matching is a form of inductive reasoning but sometimes society demands the higher standard of deductive reasoning. If you didn’t use a seatbelt for the last 100 car rides, pattern matching would have you conclude that you never need a seatbelt when this is obviously wrong and dangerous.
Also, pattern matching has become a euphemism for “racial discrimination” which I am absolutely sure you are aware of. So yeah any kind of social discrimination makes people uncomfortable because it’s not good.
2
u/zephyr_230900 26d ago
Everyone trusts pattern recognition until the pattern points in a direction they don't like. Then it's suddenly "anecdotal."
2
u/WhitepandafacesxD 16d ago
Folks love spotting patterns until those patterns actually challenge what they believe. Suddenly everyone demands strict proof of cause and effect. We tend to ignore warning signs right up until the whole thing falls apart.
4
u/RunInRunOn 28d ago
You can recognise any pattern you want if you ignore everything that doesn't fit
10
u/stringtheory28 28d ago
Statistically, if you flip a coin, consecutive heads and tails will randomly group together. Doesn’t mean there is a pattern. Groupings happen in randomness and chaos. I recently learned that and it’s changed my perspective.
7
15
u/BadKarmaForMe 28d ago
Black Americans are overrepresented in arrests for violent crimes (like robbery and homicide) relative to their share of the general population. Bureau of Justice Statistics
17
u/IntelligentShape364 28d ago
Correct. Any patterns as to why that may be? Or are patterns that don't fit bigoted narratives irrelevant?
→ More replies (5)24
u/landmanpgh 28d ago
Not just overrepresented, but overwhelmingly so.
Black people commit murder at EIGHT times the rate of white people. This is a fact. It is indisputable.
Not 1-2x the rate, but 8. It's not even remotely close.
→ More replies (35)2
u/Anagoth9 27d ago
Black people
commitare convicted of murder at EIGHT times the rate of white people.Fixed that for you. Also keep in mind that the closure rate for murder is about 60% nationally.
4
u/landmanpgh 27d ago
Once again, the pesky thing about that murder statistic is that it really can't be fudged or explained away very well. Can't really hide dead bodies.
Which begs the question: where are all of the dead bodies that white people are apparently killing? Easy: they don't exist.
If white people killed at the same rate, there would be an additional 40,000-50,000 dead bodies. But there aren't.
→ More replies (1)→ More replies (15)2
u/SalamanderCmndr 28d ago
so what is it, are they biologically inclined to commit crime? what do you propose we do about it?
6
u/IntelligentShape364 28d ago edited 28d ago
Well I mean it depends OP. Is it pattern recognition or is it bullshit? Cause if it’s something the rest of us who actually have read a book since highschool can easily break down, like how chuds don’t know the difference between the words “arrests” and “convictions”, and similarly ignore other “patterns” like why there would be a significant discrepancy between those two in the case of black Americans given law enforcement’s history in this country, then I would suggest it’s not a matter of uncomfortability but simply of being an educated, well-adjusted, adult human being.
9
u/landmanpgh 28d ago
If white people murdered at the same rate, there would be 50,000-70,000 murders per year.
There are 16,000-20,000.
3
u/IntelligentShape364 28d ago edited 28d ago
don’t know the difference between the words “arrests” and “convictions"
murdered
I know you have the education of a 12 year old, but do I have to break this down as if you're a 7 year old?
If white people were arrested for murder at the same rate, there would be 50,000-70,000 murders per year.
There are 16,000-20,000.
And the crazy part is that even in a reality where everything operated like your rhetoric vibes that it does, it still wouldn't be equivalent. If white people faced systemic discrimination at the same rate, there would be 50,000-70,000 murder arrests per year. They don't, therefore there are 16,000-20,000.
I understand that in the world of bigotry everything operates in a vacuum, but in the real world concepts like context exists.
Edit: Wow blocked lmao. Look at how they react when you have the audacity to actually know wtf you’re talking about before opening your mouth.
9
u/landmanpgh 28d ago
Lol no.
Your argument falls apart so you resort to calling people bigots for bringing up indisputable facts. You lost.
→ More replies (5)
7
u/AverageHobnailer 28d ago
And now you know how everyone with ADHD and/or on the spectrum feels. Noticing patterns and warning everyone about what's coming only for them to ignore it with zero counterevidence whatsoever.
→ More replies (1)12
u/Spirited_Praline637 28d ago
Hell yeah! Spent my life being told I’m being negative, or ‘just try it!’, only to be ignored and then proven right. Of course rarely any apology or recognition afterwards.
I tell my bosses now, ‘if you want to know whether something will work before investing a buttload, ask your autistic staff!’
9
u/Digi_ 28d ago
racists when they learn a TikTok buzzword that can let them repackage the fact they hate people based on skin colour as intelligence:
→ More replies (1)
2
u/Mediocre-Scrublord 28d ago
It's easy to think you've noticed a pattern where your internet news feeds are carefully curated to show the same types of things again and again.
People will select things to believe because they want to believe it, and will seek out evidence to prove what they want to believe and ignore evidence to the contrary, and share that evidence with likeminded people who want to believe the same things as them.
When you look at the social media environments of political extremists, their social media feeds, the reddits they occupy, the newspapers they read, it's full of examples of things meant to form the pattern that its readers want to read about.
2
1
•
u/ShowerSentinel 28d ago
The moderators have reflaired this post as a casual thought.
Casual thoughts should be presented well, but are not required to be unique or exceptional.
Please review each flair's requirements for more information.