OpenAI Blocks Iranian Influence Operation Using ChatGPT for U.S. Election Propaganda

OpenAI on Friday said it banned a set of accounts linked to what it said was an Iranian covert influence operation that leveraged ChatGPT to generate content that, among other things, focused on the upcoming U.S. presidential election.

“This week we identified and took down a cluster of ChatGPT accounts that were generating content for a covert Iranian influence operation identified as Storm-2035,” OpenAI said.

“The operation used ChatGPT to generate content focused on a number of topics — including commentary on candidates on both sides in the U.S. presidential election – which it then shared via social media accounts and websites.”

The artificial intelligence (AI) company said the content did not achieve any meaningful engagement, with a majority of the social media posts receiving negligible to no likes, shares, and comments. It further noted it had found little evidence that the long-form articles created using ChatGPT were shared on social media platforms.

The articles catered to U.S. politics and global events, and were published on five different websites that posed as progressive and conservative news outlets, indicating an attempt to target people on opposite sides of the political spectrum.

OpenAI said its ChatGPT tool was used to create comments in English and Spanish, which were then posted on a dozen accounts on X and one on Instagram. Some of these comments were generated by asking its AI models to rewrite comments posted by other social media users.

“The operation generated content about several topics: mainly, the conflict in Gaza, Israel’s presence at the Olympic Games, and the U.S. presidential election—and to a lesser extent politics in Venezuela, the rights of Latinx communities in the U.S. (both in Spanish and English), and Scottish independence,” OpenAI said.

“They interspersed their political content with comments about fashion and beauty, possibly to appear more authentic or in an attempt to build a following.”

Storm-2035 was also one of the threat activity clusters highlighted last week by Microsoft, which described it as an Iranian network “actively engaging U.S. voter groups on opposing ends of the political spectrum with polarizing messaging on issues such as the US presidential candidates, LGBTQ rights, and the Israel-Hamas conflict.”

Some of the phony news and commentary sites set up by the group include EvenPolitics, Nio Thinker, Savannah Time, Teorator, and Westland Sun. These sites have also been observed utilizing AI-enabled services to plagiarize a fraction of their content from U.S. publications. The group is said to be operational from 2020.

Microsoft has further warned of an uptick in foreign malign influence activity targeting the U.S. election over the past six months from both Iranian and Russian networks, the latter of which have been traced back to clusters tracked as Ruza Flood (aka Doppelganger), Storm-1516, and Storm-1841 (aka Rybar).

“Doppelganger spreads and amplifies fabricated, fake or even legitimate information across social networks,” French cybersecurity company HarfangLab said. “To do so, social networks accounts post links that initiate an obfuscated chain of redirections leading to final content websites.”

However, indications are that the propaganda network is shifting its tactics in response to aggressive enforcement, increasingly using non-political posts and ads and spoofing non-political and entertainment news outlets like Cosmopolitan, The New Yorker and Entertainment Weekly in an attempt to evade detection, per Meta.

The posts contain links that, when tapped, redirects users to a Russia war- or geopolitics-related article on one of the counterfeit domains mimicking entertainment or health publications. The ads are created using compromised accounts.

The social media company, which has disrupted 39 influence operations from Russia, 30 from Iran, and 11 from China since 2017 across its platforms, said it uncovered six new networks from Russia (4), Vietnam (1), and the U.S. (1) in the second quarter of 2024.

“Since May, Doppelganger resumed its attempts at sharing links to its domains, but at a much lower rate,” Meta said. “We’ve also seen them experiment with multiple redirect hops including TinyURL’s link-shortening service to hide the final destination behind the links and deceive both Meta and our users in an attempt to avoid detection and lead people to their off-platform websites.”

The development comes as Google’s Threat Analysis Group (TAG) also said this week that it had detected and disrupted Iranian-backed spear-phishing efforts aimed at compromising the personal accounts of high-profile users in Israel and the U.S., including those associated with the U.S. presidential campaigns.

The activity has been attributed to a threat actor codenamed APT42, a state-sponsored hacking crew affiliated with Iran’s Islamic Revolutionary Guard Corps (IRGC). It’s known to share overlaps with another intrusion set known as Charming Kitten (aka Mint Sandstorm).

“APT42 uses a variety of different tactics as part of their email phishing campaigns — including hosting malware, phishing pages, and malicious redirects,” the tech giant said. “They generally try to abuse services like Google (i.e. Sites, Drive, Gmail, and others), Dropbox, OneDrive and others for these purposes.”

The broad strategy is to gain the trust of their targets using sophisticated social engineering techniques with the goal of getting them off their email and into instant messaging channels like Signal, Telegram, or WhatsApp, before pushing bogus links that are designed to collect their login information.

The phishing attacks are characterized by the use of tools like GCollection (aka LCollection or YCollection) and DWP to gather credentials from Google, Hotmail, and Yahoo users, Google noted, highlighting APT42’s “strong understanding of the email providers they target.”

“Once APT42 gains access to an account, they often add additional mechanisms of access including changing recovery email addresses and making use of features that allow applications that do not support multi-factor authentication like application-specific passwords in Gmail and third-party app passwords in Yahoo,” it added.

Found this article interesting? Follow us on Twitter  and LinkedIn to read more exclusive content we post.