OpenAI Cracks Down on Malicious AI Use, Bans Accounts Linked to Scams

February 25, 202568

OpenAI has recently stepped up efforts to combat the malicious use of its ChatGPT tool, banning multiple user accounts involved in fraudulent activities.

The company’s proactive stance aims to tackle scammers and other malicious actors using AI to generate harmful content, fake resumes, and scam networks.

AI Use for Scams and Fraudulent Activities

In a new report, OpenAI highlighted that scammers have been using ChatGPT for a range of malicious activities, such as debugging code and generating content for nefarious purposes.

With ChatGPT surpassing 400 million weekly active users, its accessibility has made it a prime tool for malicious use.

Among the cases OpenAI uncovered, one involved a banned account that generated fake news articles that denigrated the U.S. and were published under a Chinese company’s byline in Latin American media outlets.

Another involved accounts reportedly linked to North Korea, which used AI to generate fictitious resumes in an attempt to infiltrate Western companies.

Romance Scams and Social Media Manipulation

OpenAI also reported identifying accounts that were likely connected to Cambodia, using ChatGPT to translate and create comments for a “romance baiting” scam across social media platforms like X, Facebook, and Instagram.

These scams often prey on unsuspecting individuals, leading to financial exploitation and emotional harm.

OpenAI’s Commitment to Safety and Integrity

In response to these threats, OpenAI has implemented stricter measures to prevent its tools from being used in fraudulent schemes. The company emphasizes that its policies strictly prohibit the use of its output for scams or fraud.

By sharing insights with partners like Meta, OpenAI is working to enhance its ability to detect and prevent malicious activities across platforms, ensuring the safe use of AI technologies.

While OpenAI’s recent bans reflect its commitment to preventing AI misuse, the company continues to monitor and investigate potential threats to maintain the integrity of its tools.