Research has shown that large language models (LLMs) tend to overemphasize information at the beginning and end of a document or conversation, while neglecting the middle.
This “position bias” means that, if a lawyer is using an LLM-powered virtual assistant to retrieve a certain phrase in a 30-page affidavit, the LLM is more likely to find the right text if it is on the initial or final pages.
MIT researchers have discovered the mechanism behind this phenomenon. … They found that certain design choices which control how the model processes input data can cause position bias. — Read More
Tag Archives: Bias
Black Nazis? A woman pope? That’s just the start of Google’s AI problem.
Just last week, Google was forced to pump the brakes on its AI image generator, called Gemini, after critics complained that it was pushing bias … against white people.
The controversy started with — you guessed it — a viral post on X. According to that post from the user @EndWokeness, when asked for an image of a Founding Father of America, Gemini showed a Black man, a Native American man, an Asian man, and a relatively dark-skinned man. Asked for a portrait of a pope, it showed a Black man and a woman of color. Nazis, too, were reportedly portrayed as racially diverse.
After complaints from the likes of Elon Musk, who called Gemini’s output “racist” and Google “woke,” the company suspended the AI tool’s ability to generate pictures of people. — Read More
Gemini and Google’s Culture
Last Wednesday, when the questions about Gemini’s political viewpoint were still limited to its image creation capabilities, I accused the company of being timid:
Stepping back, I don’t, as a rule, want to wade into politics, and definitely not into culture war issues. At some point, though, you just have to state plainly that this is ridiculous. Google specifically, and tech companies broadly, have long been sensitive to accusations of bias; that has extended to image generation, and I can understand the sentiment in terms of depicting theoretical scenarios. At the same time, many of these images are about actual history; I’m reminded of George Orwell in 1984:
Every record has been destroyed or falsified, every book has been rewritten, every picture has been repainted, every statue and street and building has been renamed, every date has been altered. And that process is continuing day by day and minute by minute. History has stopped. Nothing exists except an endless present in which the Party is always right. I know, of course, that the past is falsified, but it would never be possible for me to prove it, even when I did the falsification myself. After the thing is done, no evidence ever remains. The only evidence is inside my own mind, and I don’t know with any certainty that any other human being shares my memories. — Read More
Does ChatGPT have a liberal bias?
A new paper making this claim has many flaws. But the question merits research.
Previous research has shown that many pre-ChatGPT language models express left-leaning opinions when asked about partisan topics. But OpenAI said in February that the workers who fine-tune ChatGPT train it to refuse to express opinions when asked controversial political questions. So it was interesting to see a new paper claim that ChatGPT expresses liberal opinions, agreeing with Democrats the vast majority of the time. — Read More
From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models
Language models (LMs) are pretrained on diverse data sources, including news, discussion forums, books, and online encyclopedias. A significant portion of this data includes opinions and perspectives which, on one hand, celebrate democracy and diversity of ideas, and on the other hand are inherently socially biased. Our work develops new methods to (1) measure political biases in LMs trained on such corpora, along social and economic axes, and (2) measure the fairness of downstream NLP models trained on top of politically biased LMs. We focus on hate speech and misinformation detection, aiming to empirically quantify the effects of political (social, economic) biases in pretraining data on the fairness of high-stakes social-oriented tasks. Our findings reveal that pretrained LMs do have political leanings that reinforce the polarization present in pretraining corpora, propagating social biases into hate speech predictions and misinformation detectors. We discuss the implications of our findings for NLP research and propose future directions to mitigate unfairness. — Read More
New Tool Allows Users to See Bias in AI Image Generators
A new tool is allowing people to see how certain word combinations produce biased results in artificial intelligence (AI) text-to-image generators.
Hosted on Hugging Face, the “Stable Diffusion Bias Explorer” was launched in late October.
According to Motherboard, the simple tool lets users combine descriptive terms and see firsthand how the AI model maps them to racial and gender stereotypes. Read More
FLAWED AI MAKES ROBOTS RACIST, SEXIST
A robot operating with a popular Internet-based artificial intelligence system consistently gravitates to men over women, white people over people of color, and jumps to conclusions about peoples’ jobs after a glance at their face.
The work, led by Johns Hopkins University, Georgia Institute of Technology, and University of Washington researchers, is believed to be the first to show that robots loaded with an accepted and widely-used model operate with significant gender and racial biases. The work is set to be presented and published this week at the 2022 Conference on Fairness, Accountability, and Transparency. Read More
Artificial intelligence predicts patients’ race from their medical images
Study shows AI can identify self-reported race from medical images that contain no indications of race detectable by human experts.
The miseducation of algorithms is a critical problem; when artificial intelligence mirrors unconscious thoughts, racism, and biases of the humans who generated these algorithms, it can lead to serious harm. Computer programs, for example, have wrongly flagged Black defendants as twice as likely to reoffend as someone who’s white. When an AI used cost as a proxy for health needs, it falsely named Black patients as healthier than equally sick white ones, as less money was spent on them. Even AI used to write a play relied on using harmful stereotypes for casting.
Removing sensitive features from the data seems like a viable tweak. But what happens when it’s not enough?
Examples of bias in natural language processing are boundless — but MIT scientists have investigated another important, largely underexplored modality: medical images. Using both private and public datasets, the team found that AI can accurately predict self-reported race of patients from medical images alone. Using imaging data of chest X-rays, limb X-rays, chest CT scans, and mammograms, the team trained a deep learning model to identify race as white, Black, or Asian — even though the images themselves contained no explicit mention of the patient’s race. This is a feat even the most seasoned physicians cannot do, and it’s not clear how the model was able to do this. Read More
Why it’s so damn hard to make AI fair and unbiased
There are competing notions of fairness — and sometimes they’re totally incompatible with each other.
…Computer scientists are used to thinking about “bias” in terms of its statistical meaning: A program for making predictions is biased if it’s consistently wrong in one direction or another. (For example, if a weather app always overestimates the probability of rain, its predictions are statistically biased.) That’s very clear, but it’s also very different from the way most people colloquially use the word “bias” — which is more like “prejudiced against a certain group or characteristic.”
The problem is that if there’s a predictable difference between two groups on average, then these two definitions will be at odds. If you design your search engine to make statistically unbiased predictions about the gender breakdown among CEOs, then it will necessarily be biased in the second sense of the word. And if you design it not to have its predictions correlate with gender, it will necessarily be biased in the statistical sense. Read More
The Gender Bias Inside GPT-3
The 2022 theme for International Women’s Day is #BreakTheBias. With that in mind, I decided to do a little experiment to see what GPT-3 can show us about the gender bias that’s built into our language.
Let’s start with a quick overview of GPT-3, in case you don’t spend as much time as I do having conversations with a machine: Basically, it’s an AI trained on pretty much every word written online to generate conversational language that sounds remarkably human.
Before I get into the part where I depress you, I want to be clear: GPT-3 is a tremendous achievement. An enormously big deal that will have far-ranging implications. That’s why it’s important that we don’t build it on a rotten foundation.
And the rotten foundation? That’s us. Read More
Rick's Cafe AI 