Messenger, Instagram’s Sticker-Generating AI Produced Potentially Offensive Content

Generative AI has had its day in the sun when it was all new and exciting, and while it still is, the setbacks are already showing large-scale risks for everyone involved. When people are given the freedom to generate any image they want, some will eventually create content that could be seen as offensive.

Meta Has Fires to Put Out

Meta just released its AI-generated sticker feature for Instagram and Messenger this week, and problematic users are already sharing their creations online. One user, namely Pier-Olivier Desbiens shared his own generated stickers through X.

His post went viral due to the controversial outcomes of the text prompts he fed to the AI generator. For example, typing "child soldier" through the sticker search bar generated children in a soldier's uniform holding an assault rifle.

There were other stickers like Elmo from Sesame Street holding a knife, a female and pregnant Shrek, and other provocative stickers involving the Prime Minister of Canada, Justin Trudeau, which just shows how the line between funny, creative, and offensive could be easily blurred.

Desbiens wrote in the post: "Found out that Facebook messenger has AI-generated stickers now and I don't think anyone involved has thought anything through," also stating that we "really do live in the stupidest future imaginable."

Luckily, the feature is not yet available to everyone and is only accessible to select users on a limited basis. The stickers can only be sent through private and group chats, which is good since it means that we won't be seeing absurd stickers in comment sections.

Meta uses its own Emu image synthesis model to generate the stickers, as mentioned in Ars Technica, and has put filters in place to avoid such things from happening. Evidently, the filters aren't doing their jobs very well.

Read Also: AI Deepfake Ad of MrBeast Offered iPhone 15 Pros for $2

AI is Becoming the Enemy

Like all technological advancements, it needs to be used responsibly and ethically. AI has the potential to make our lives significantly easier, but so far, it has only caused problems for people, especially those who are losing their jobs.

With generative AI becoming more capable in various tasks, companies are now opting for artificial intelligence that doesn't need to be given a wage. What's worse is that some AI companies are allegedly using copyrighted materials to train AI models.

Just this week, there have been two instances of generative AI being used to create fake ads using the likeness of famous people. Tom Hanks was involved in a dental ad plan as the advertisers used deepfake technology, all without the actor's consent.

The other was James Donaldson, more known as MrBeast. The popular YouTube had his likeness recreated to generate a fraudulent ad on TikTok, which stated that he was giving away iPhone 15s to 10,000 viewers for just $2, as reported by Tech Crunch.

While other fake ads using the faces of famous people are obviously AI-generated like Elon Musk promoting an app that pays you to watch YouTube videos, the problem lies with the fact that the people whose faces were copied weren't aware of it.

Related: Meta is Being Sued for Its LLaMa AI Software Training with Copyrighted Materials

© 2024 iTech Post All rights reserved. Do not reproduce without permission.

Company from iTechPost

More from iTechPost