Society problems

Researchers detail systemic issues and risks to society in language models

Hear from CIOs, CTOs, and other executives and senior executives on data and AI strategies at the Future of Work Summit on January 12, 2022. Learn more

Google’s DeepMind researchers have discovered major flaws in the release of large language models like GPT-3 and warn that these could have serious consequences for society, such as allowing deception and reinforcing prejudice. Notably, the co-authors of an article on the study claim that the harms can be multiplied by large language models without malicious intent on the part of the creators. In other words, this damage can spread accidentally, due to incorrect specifications of what an agent should learn or the process of training the model.

“We believe that language agents carry a high risk of harm, as discrimination is easily perpetuated through language. In particular, they can influence society in a way that produces a lock-in of values, making it more difficult to challenge existing problematic norms, ”the newspaper read. “We currently don’t have many approaches to correcting these forms of erroneous specification and the resulting behavioral problems. “

The article assumes that language agents could also allow “incitement to violence” and other forms of societal harm, especially by politically motivated people. Agents could also be used to spread dangerous information, such as how to make weapons or avoid paying taxes. In a great example of work published last fall, GPT-3 tells a person to kill themselves.

The DeepMind article is the most recent study to raise concerns about the consequences of deploying large language models created with datasets pulled from the web. The best-known article on this topic is titled “On the Dangers of Stochastic Parrots: Can Linguistic Patterns Be Too Big?” And was posted last month at the Fairness, Accountability and Transparency conference by authors who include former Google Ethical AI team co-leads, Margaret Mitchell and Timnit Gebru. This article argues that language models that appear to grow in size perpetuate stereotypes and incur environmental costs more likely to be generated by marginalized groups.

While Google fired its two researchers who chose to keep their names on paper and asked other Google researchers to remove their names from an article that came to a similar conclusion, the DeepMind research cites the article on stochastic parrots among related works.

Earlier this year, an article by OpenAI and researchers at Stanford University detailed a meeting between experts from fields such as computer science, political science and philosophy. The group concluded that companies like Google and OpenAI, which control the world’s largest known language models, have just months to set standards for the ethical use of technology before it’s too late. .

The DeepMind article joins a series of books that highlight the shortcomings of NLP. In late March, nearly 30 companies and universities around the world discovered major issues during an audit of five popular multilingual datasets used for machine translation.

An article written about this audit found that in a significant fraction of the major parts of the dataset assessed, less than 50% of the sentences were of acceptable quality, according to more than 50 volunteers from the NLP community.

The companies and organizations listed as co-authors of this article include Google and Intel Labs and are from China, Europe, the United States, and several countries in Africa. Co-authors include the University of the Sorbonne (France), the University of Waterloo (Canada) and the University of Zambia. Major open source advocates have also participated, such as EleutherAI, which is striving to replicate GPT-3; Face hugging; and the Masakhane project to produce machine translation for African languages.

Consistent issues with mislabeled data emerged during the audit, and volunteers found that an analysis of 100 sentences in many languages ​​can reveal serious quality issues, even for people who are unfamiliar with the language. tongue.

“We evaluated samples from 205 languages ​​and found that 87 of them had less than 50% usable data,” the newspaper read. “As the scale of ML research increases, it becomes more and more difficult to validate automatically collected and organized datasets. “

The article also finds that building NLP models with datasets automatically pulled from the Internet holds promise, especially for solving the problems faced by low-resource languages, but there is very little research today on them. data collected automatically for low-resource languages. The authors suggest a number of solutions, such as the type of documentation recommended in Google’s stochastic parrots article or standard review forms, such as datasheets and map templates prescribed by Gebru or the framework of nutritional labeling of data sets.

In other news, researchers from Amazon, ChipBrain and MIT found that the test sets of the 10 most frequently cited data sets used by AI researchers have an average label error rate of 3.4%, which has an impact on the benchmark results.

This week, the organizers of NeurIPS, the world’s largest machine learning conference, announced their intention to create a new track dedicated to benchmarks and datasets. A blog post announcing the news begins with the simple statement that “there are no good models without good data.”

Last month, the 2021 AI Index, an annual report that attempts to define trends in the performance of universities, businesses, policies, and systems, found that AI is industrializing rapidly. But he called the lack of benchmarks and testing methods major obstacles to the advancement of the artificial intelligence community.


VentureBeat’s mission is to be a digital public place for technical decision-makers to learn about transformative technology and conduct transactions. Our site provides essential information on data technologies and strategies to guide you in managing your organizations. We invite you to become a member of our community, to access:

  • up-to-date information on the topics that interest you
  • our newsletters
  • Closed thought leader content and discounted access to our popular events, such as Transform 2021: Learn more
  • networking features, and more

Become a member