AI trained on 4Chan becomes a ‘hate speech machine’

AI searcher and YouTuber Yannic Kilcher trained an AI using 3.3 million threads from 4chan’s politically incorrect/pol/infamously toxic board. He then unleashed the bot on 4chan with predictable results – the AI ​​was just as vile as the posts it was trained on, spouting racial slurs and engaging in anti-Semitic discussions. After Kilcher posted his video and a copy of the program on Hugging Face, a kind of GitHub for AI, ethicists and AI researchers expressed concern.

The bot, which Kilcher called GPT-4chan, “the most horrible model on the internet” – a reference to GPT-3, a language model developed by Open AI that uses deep learning to produce text – was d shockingly effective and replicated the tone and feel of 4chan posts. “The model was good in a terrible way,” Klicher said in a video about the project. “It perfectly encapsulates the mixture of offensiveness, nihilism, trolling and deep distrust of any information whatsoever that permeates most posts on /pol.”

According to Kilcher’s video, he activated nine instances of the bot and allowed them to post for 24 hours on /pol/. During this time, the bots posted around 15,000 times. It represented “more than 10% of all posts on the politically incorrect board that day,” Kilcher said in his video for the project.

Artificial intelligence researchers considered Kilcher’s video to be more than just a YouTube prank. For them, it was an unethical experiment using AI. “This experiment would never pass a human research ethics board,” Lauren Oakden-Rayner, director of research at the NeuroRehab Allied Health Network in Australia, said in a statement. Twitter feed.

“Open science and software are wonderful principles but must be weighed against the potential harm,” she said. “Medical research has a strong ethical culture because we have a terrible history of causing harm to people, usually from powerless groups… it has performed human experiments without informing users, without consent or oversight. This violates all the principles of human research ethics.

Kilcher told Motherboard in a Twitter DM that he is not an academic. “I’m a YouTuber and this is prank and light trolling. And my bots, if there are any, are by far the sweetest, shyest content you’ll find on 4chan,” he said. “I have limited the time and number of posts, and I do not distribute the code for the bot itself.”

He also pushed back, as he had on Twitter, on the idea that this bot would ever hurt or had hurt. “All I hear are vague bombastic statements about ‘harm’, but absolutely no instances of actual harm,” he said. “It’s like a magic word these people say but nothing more.”

4chan’s environment is so toxic, Kilcher explained, that messages deployed by its bots would have no impact. “No one on 4chan was even a little hurt by this,” he said. “I invite you to go spend some time on /pol/ and ask yourself if a bot that just outputs the same style really changes the experience.”

After AI researchers alerted Hugging Face to the harmful nature of the bot, the site shut down the model and people were unable to download it. “After much internal debate at HF, we have decided not to remove the model that the author uploaded here under the following conditions: #1 The model sheet and video clearly warned of the limitations and issues raised by the model and the POL section of 4Chan in general. #2 The inference widget has been disabled so as not to facilitate the use of the model,” said Hugging Face co-founder and CEO Clément Delangue on Hugging Face.

Kilcher explained in his video, and Delangue quoted in his response, that one of the things that made GPT4-Chan interesting was his ability to outperform other similar bots in AI tests designed to measure “truthfulness”. .

“We considered it useful for the field to test what a model trained on such data could do and how it performed compared to others (i.e. GPT-3) and would help attract attention to both the limitations and the risks of these models” says Delangue. “We have also been working on a feature to ‘close’ such models which we are currently prioritizing for ethical reasons. Happy to answer any additional questions too!”

When contacted for comment, Delangue told Motherboard that Hugging Face has taken the extra step of blocking all uploads of the model.

“Building a system that can create unspeakably awful content, use it to produce tens of thousands of mostly toxic posts on a real message board, and then post it around the world so anyone else can do the same, it just seems right – I don’t know, right,” Arthur Holland Michel, an artificial intelligence researcher and writer for the International Committee of the Red Cross, told Motherboard.

“This could generate extremely toxic content on a massive and sustained scale,” Michel said. “Obviously, there are already a ton of human trolls on the internet doing this the old-fashioned way. What’s different here is the amount of content he can create with this system, just one person was able to post 30,000 comments on 4chan in a matter of days. Now imagine what kind of harm a team of ten, twenty or a hundred coordinated people using this system could do.

Kilcher did not believe that GPT-4chan could be deployed on a large scale for targeted hate campaigns. “It’s actually quite difficult to get GPT-4chan to say something targeted,” he said. “Usually he misbehaves in a weird way and is very unsuitable for performing anything targeted. Again, vague hypothetical accusations are thrown around, with no instance or real evidence.

Os Keyes, Ada Lovelace Fellow and PhD candidate at the University of Washington, told Motherboard that Kilcher’s comment was irrelevant. “It’s a good opportunity to discuss not the harm, but the fact that that harm is so obviously predictable, and that his response of ‘show me where he hurt’ misses the point and is inadequate,” wrote they stated. “If I spend my grandmother’s estate on gas station cards and throw them over the wall in a prison, we shouldn’t have to wait for the first parolee to start lighting fires to agree that it was a phenomenal thing to do.”

“But – and that’s a big but – that’s kind of the point,” Keyes said. “It’s a tasteless project that nothing good can come of, and it’s sort of unavoidable. His whole shtick is a nerd shock schlock. And there’s a balance to be struck between problem awareness and the attention paid to someone whose only apparent pattern of importance in the world is “pay attention to me!”

Kilcher said, repeatedly, that he knew the bot was vile. “I’m obviously aware that the model won’t work well in a professional setting or at most people’s table,” he said. “He uses swear words, strong insults, has conspiratorial views and all sorts of ‘unpleasant’ properties. After all, he is trained on /pol/ and he reflects the tone and common topics of this forum. »

He said he felt he had been clear, but wanted his results to be repeatable and that’s why he posted the model to Hugging Face. “As for the evaluation results, some of them were really interesting and unexpected and revealed weaknesses in the current benchmarks, which might have been possible without actually doing the work.”

Kathryn Cramer, a graduate student in complex systems and data science at the University of Vermont, pointed out that GPT-3 has guardrails that prevent it from being used to build this type of racist bot and that Kilcher had to use GPT-J to build his system. “I tried your tool’s demo mode 4 times, using benign tweets from my feed as the starting text,” Cramer said in a post. thread on Hugging Face. “On the first try, one of the response messages was a single word, the N-word. The seed of my third trial was, I think, a single sentence about climate change. Your tool responded by extending it to a conspiracy theory that the Rothschilds and the Jews were behind it.

Cramer told Motherboard that she had a lot of experience with GPT-3 and understood some of the frustrations with how it works. a priori censored certain types of behavior. “I’m not a fan of this bodyguard,” she said. “I find it deeply annoying and I think it skews the results… I understand the urge to put this off. I even understand the urge to play pranks about it. But the reality is that he basically invented a hate speech machine, used it 30,000 times, and released it into the wild. And yes, I understand being annoyed by security rules, but that’s not a legitimate response to that annoyance.

Keyes was of the same opinion. “Certainly we need to ask meaningful questions about how GPT-3 is limited (or not) in how it can be used, or what people’s responsibilities are when deploying things,” they said. “The first should be directed at the GPT-3 developers, and while the second should be directed at Kilcher, it’s not clear to me that he really cares. Some people just want to be pissed off by a insecure need for attention Most of them use 4chan, some of them it seems are role models.

Comments are closed.