AI Remains a Wild Card in the War Against Disinformation

Source: Enrico01 via Alamy Stock Photo

COMMENTARY

Disinformation — information created and shared to mislead opinion or understanding — isn’t a new phenomenon. However, digital media and the proliferation of open source generative artificial intelligence (GenAI) tools like ChatGPT, DALL-E, and DeepSwap, coupled with mass dissemination capabilities of social media, are exacerbating challenges associated with preventing the spread of potentially harmful fake content.

Although in their infancy, these tools have begun shaping how we create digital content, requiring little in the way of skill or budget to produce convincing photo and video imitations of individuals or generate believable conspiratorial narratives. In fact, the World Economic Forum places disinformation amplified by AI as one of the most severe global risks over the next few years, including the possibilities for exploitation amid heightened global political and social tensions, and during critical junctures such as elections.

In 2024, as more than 2 billion voters across 50 countries have already headed to the polls or await upcoming elections, disinformation has driven concerns over its ability to shape public opinion and erode trust in the media and democratic processes. But while AI-generated content can be leveraged to manipulate a narrative, there is also potential for these tools to improve our capabilities to identify and protect against these threats.

Addressing AI-Generated Disinformation

Governments and regulatory authorities have introduced various guidelines and legislation to protect the public from AI-generated disinformation. In November 2023, 18 countries — including the US and UK — entered into a nonbinding AI Safety agreement, while in the European Union, an AI Act approved in mid-March limits various AI applications. The Indian government drafted legislation in response to a proliferation of deepfakes during elections cycle that compels social media companies to remove reported deepfakes or lose their protection from liability for third-party content.

Nevertheless, authorities have struggled to adapt to the shifting AI landscape, which often outpaces their ability to develop relevant expertise and reach consensus across multiple (and often opposing) stakeholders from government, civil, and commercial spheres.

Social media companies have also implemented guardrails to protect users, including increased scanning and removal of fake accounts, and steering users toward reliable sources of information, particularly around elections. Amid financial challenges, many platforms have downsized teams dedicated to AI ethics and online safety, creating uncertainty as to the impact this will have on platforms’ abilities and appetite to effectively stem false content in the coming years.

Meanwhile, technical challenges persist around identifying and containing misleading content. The sheer volume and rate at which information spreads through social media platforms — often where individuals first encounter falsified content — seriously complicates detection efforts; harmful posts can “go viral” within hours as platforms prioritize engagement over accuracy. Automated moderation has improved capabilities to an extent, but such solutions have been unable to keep up. For instance, significant gaps remain in automated attempts to detect certain hashtags, keywords, misspellings and non-English words.

Disinformation can be exacerbated when it is unknowingly disseminated by mainstream media or influencers who have not sufficiently verified its authenticity. In May 2023, the Irish Times apologized after gaps in its editing and publication process resulted in the publication of an AI-generated article. In the same month, while an AI-generated image on Twitter of an explosion at the Pentagon was quickly debunked by US law enforcement, it nonetheless prompted a 0.26% drop in the stock market.

What Can Be Done?

Not all applications of AI are malicious. Indeed, leaning into AI may help circumvent some limitations of human content moderation, decreasing reliance on human moderators to improve efficiency and reduce costs. But there are limitations. Content moderation using large language models (LLMs) is often overly sensitive in the absence of sufficient human oversight to interpret context and sentiment, blurring the line between preventing the spread of harmful content and suppressing alternative views. Continued challenges with biased training data and algorithms and AI hallucinations (occurring most commonly in image recognition tasks) have also contributed to difficulties in employing AI technology as a protective measure.

A further potential solution, already in use in China, involves “watermarking” AI-generated content to help identification. Though the differences between AI and human-generated content are often imperceptible to us, deep-learning models and algorithms within existing solutions can easily detect these variations. The dynamic nature of AI-generated content poses a unique challenge for digital forensic investigators, who need to develop increasingly sophisticated methods to counter adaptive techniques from malicious actors leveraging these technologies. While existing watermark technology is a step in the right direction, diversifying solutions will ensure continued innovation which can outpace, or at least keep up with, adversarial uses.

Boosting Digital Literacy

Combating disinformation also requires addressing users’ ability to critically engage with AI-generated content, particularly during election cycles. This requires improved vigilance in identifying and reporting misleading or harmful content. However, research shows that our understanding of what AI can do and our ability to spot fake content remains limited. Although skepticism is often taught from an early age in the consumption of written content, technological innovations now necessitate the extension of this practice to audio and visual media to develop a more discerning audience.

Testing Ground

As adversarial actors adapt and evolve their use of AI to create and spread disinformation, 2024 and its multitude of elections will be a testing ground for how effectively companies, governments, and consumers are able to combat this threat. Not only will authorities need to double down on ensuring sufficient protective measures to guard people, institutions, and political processes against AI-driven disinformation, but it will also become increasingly critical to ensure that communities are equipped with the digital literacy and vigilance needed to protect themselves where other measures may fail.

About the Author(s)

Associate, Strategic Intelligence, S-RM

Erin Drake is an associate in S-RM’s Strategic Intelligence team, where she leads on case management of regular and bespoke consulting projects. She joined the firm in 2017 and has worked on a variety of projects ranging from threat assessments to security risk assessments across several markets. This often entails the development of client-specific approach and methodological framework for high-level and detailed bespoke projects, to support clients in understanding and monitoring the security, political, regulatory, reputational, geopolitical, and macroeconomic threats present in their operating environment. Erin’s expertise includes global maritime security issues, political stability concerns in the commercial sector, and conflict analysis. Erin holds a master’s degree in international relations with a focus on global security issues like nuclear proliferation and multilateral diplomacy.

Global Cyber Threat Intelligence Lead, S-RM

Melissa DeOrio is Global Cyber Threat Intelligence Lead at S-RM, supporting clients on a variety of proactive cyber and cyber-threat intelligence services. Before joining S-RM, Melissa worked on US Federal Law Enforcement cyber investigations as a cyber targeter. In this role, Melissa utilized numerous cyber-investigative techniques and methodologies to investigate cyber threat actors and groups including open source intelligence techniques, cryptocurrency asset tracing as well as identifying and mapping threat actor tactics, techniques, and procedures (TTPs) to provide tactical and strategic intelligence reports. Melissa began her career in corporate intelligence, where she specialized in Turkish regional investigations, managed a global team of researchers, and played a role in the development and implementation of a new compliance program at a leading management consulting firm.