Early user tests of Meta’s new AI-generated sticker tool on Facebook Messenger have yielded some concerning and, in some cases, quite humorous results. For example, a user with the handle @Pioldes was able to generate a variety of inappropriate sticker images. These included depictions of child soldiers, Nintendo characters armed with guns, Mickey Mouse in unseemly situations, and even nude illustrations of the Canadian Prime Minister, Justin Trudeau.
This highlights the challenges and potential pitfalls associated with AI-generated content. While AI can be a powerful tool for generating creative content, it also needs careful oversight and moderation to prevent the creation and dissemination of inappropriate or harmful materials. It’s a reminder of the importance of responsible AI usage and content monitoring in online platforms.
found out that facebook messenger has ai generated stickers now and I don't think anyone involved has thought anything through pic.twitter.com/co987cRhyu
— podesbiens.bsky.social (@Pioldes) October 3, 2023
It appears that Meta’s AI-generated sticker tool can sometimes produce content that is both unexpected and potentially inappropriate. Examples include the tool adding female breasts to various characters and figures, such as Sonic the Hedgehog and Karl Marx. There’s even a sticker depicting a woman breastfeeding Pikachu.
These instances highlight the need for robust content moderation and oversight when implementing AI-generated content tools. While AI can be creative and generate a wide range of content, ensuring that it aligns with community guidelines and remains respectful is crucial to maintain a safe and appropriate online environment. It’s a reminder that AI-generated content tools should be carefully managed to prevent unintended or objectionable outcomes.
Meta recently introduced AI-generated chat stickers at their Connect event, along with a new AI image editor for Instagram. These chat stickers are powered by Meta’s Llama 2, a large language model that competes with ChatGPT. With these AI-generated stickers, users can quickly create numerous unique and high-quality stickers by simply providing text-based prompts.
The availability of these AI-generated stickers is currently being rolled out to select English-speaking users across various Meta platforms, including Facebook Stories, Instagram Stories and DMs, Messenger, and WhatsApp. The exact number of users who currently have access to this feature is not clear, as the rollout is likely gradual to ensure a smooth experience and gather user feedback. This new feature aims to enhance the creative expression of users by making it easier to generate personalized stickers for their conversations and stories on these social media platforms.
It appears that Meta has implemented some restrictions on certain words within the AI-generated sticker tool. If users include these restricted words in their prompts, they may receive a warning that their description could violate Meta’s community guidelines.
However, there have been reports from users indicating that it’s still possible to generate inappropriate content by using typos or descriptions of restricted words. In some instances, problematic images were generated without any additional descriptors, even with seemingly innocent prompts like “World Trade Center.”
LPT: typos of restricted words will eventually generate the prohibited content! pic.twitter.com/6PnrTJ3U7x
— Buttered Jorts (@ajn142) October 4, 2023
This highlights the ongoing challenge of ensuring responsible and safe use of AI tools on social media platforms. While restrictions and warnings can be put in place, users may find ways to work around them. It underscores the importance of continuous content moderation and refining AI algorithms to prevent the creation of inappropriate or harmful content, even in cases where users attempt to exploit the system.
The mischievous use of AI tools is indeed a common occurrence, especially during their initial launch. Meta’s decision to pursue a limited rollout of the AI-generated sticker feature is likely a proactive measure to address and rectify any misuse or abuse before making it available to a broader audience. This approach allows Meta to gather feedback, identify issues, and fine-tune the tool’s safeguards and content moderation systems.
It’s a responsible strategy to ensure that AI-powered features meet community guidelines and provide a safe and enjoyable experience for users. It’s also important for Meta to maintain an open line of communication with users and address any concerns or issues that arise during this rollout phase.
Your effort to reach out to Meta for comment is a good step in seeking transparency and understanding their perspective on this matter. It’s crucial for technology companies to continually refine and improve AI tools to mitigate misuse and maintain a positive user experience.
