Cybercriminals use malicious AI models to write malware and phishing scams Cisco Talos warns of rising threats from uncensored and custom AI tools.

New research from Cisco Talos reveals a rise in cybercriminals abusing Large Language Models (LLMs) to enhance their illicit activities. These powerful AI tools, known for generating text, solving problems, and writing code, are, reportedly, being manipulated to launch more sophisticated and widespread attacks.

For your information, LLMs are designed with built-in safety features, including alignment (training to minimize bias) and guardrails (real-time mechanisms to prevent harmful outputs). For instance, a legitimate LLM like ChatGPT would refuse to generate a phishing email. However, cybercriminals are actively seeking ways around these protections.

Talos’s investigation, shared with Hackread.com highlights three primary methods used by adversaries:

**Uncensored LLMs**: These models, lacking safety constraints, readily produce sensitive or harmful content. Examples include OnionGPT and WhiteRabbitNeo, which can generate offensive security tools or phishing emails. Frameworks like Ollama allow users to run uncensored models, such as Llama 2 Uncensored, on their own machines.

**Custom-Built Criminal LLMs**: Some enterprising cybercriminals are developing their own LLMs specifically designed for malicious purposes. Names like GhostGPT, WormGPT, DarkGPT, DarkestGPT, and FraudGPT are advertised on the dark web, boasting features like creating malware, phishing pages, and hacking tools.

**Jailbreaking Legitimate LLMs**: This involves tricking existing LLMs into ignoring their safety protocols through clever prompt injection techniques. Methods observed include using encoded language (like Base64), appending random text (adversarial suffixes), role-playing scenarios (e.g., DAN or Grandma jailbreak), and even exploiting the model’s self-awareness (meta prompting).

The dark web has become a marketplace for these malicious LLMs. FraudGPT, for example, advertised features ranging from writing malicious code and creating undetectable malware to finding vulnerable websites and generating phishing content.

However, the market isn’t without its risks for criminals themselves; Talos researchers found that the alleged developer of FraudGPT, CanadianKingpin12, was scamming potential buyers out of cryptocurrency by promising a non-existent product.

Image via Cisco Talos

Beyond direct illicit content generation, cybercriminals are leveraging LLMs for tasks similar to legitimate users, but with a malicious twist. In December 2024, Anthropic, developers of Claude LLM, noted programming, content creation, and research as top uses for their model. Similarly, criminal LLMs are used for:

*   Programming: Crafting ransomware, remote access Trojans, wipers, and code obfuscation.

*   Content Creation: Generating convincing phishing emails, landing pages, and configuration files.

*   Research: Verifying stolen credit card numbers, scanning for vulnerabilities, and even brainstorming new criminal schemes.

LLMs are also becoming targets themselves. Attackers are distributing backdoored models on platforms like Hugging Face, embedding malicious code that runs when downloaded. Furthermore, LLMs that use external data sources (Retrieval Augmented Generation or RAG) can be vulnerable to data poisoning, where attackers manipulate the data to influence the LLM’s responses.

Cisco Talos anticipates that as AI technology continues to advance, cybercriminals will increasingly adopt LLMs to streamline their operations, effectively acting as a “force multiplier” for existing attack methods rather than creating entirely new “cyber weapons.”

Malicious AI Models Are Behind a New Wave of Cybercrime, Cisco Talos

FBI tracked IntelBroker as UK’s Kai West using an email address, crypto trails, YouTube activity and forum posts after dozens of high-profile data breaches and darknet activity.

Authorities in the United States have charged a British national, Kai Logan West, widely known online as “**IntelBroker**“, with a series of high-profile data breaches that collectively caused at least $25 million in damages to companies worldwide. The 23-year-old was arrested in France in February 2025 and now faces extradition to the United States to stand trial in the Southern District of New York.

As **reported** by Hackread.com, IntelBroker’s arrest was followed by several others, including four individuals linked to the ShinyHunters hacker group. Both IntelBroker and members of ShinyHunters were **involved** in administering and moderating the cybercrime and data breach forum BreachForums.

The **unsealed complaint** (PDF), dated February 2025, lays bare the FBI’s two-year investigation into West’s cybercrime operations, connecting him to dozens of data breaches, sales of stolen data, and the leadership of a hacking collective operating on clear and dark web forums.

****Who Is IntelBroker?****

Using aliases like “IntelBroker” and “Kyle Northern”, West built a reputation on a clear and dark web forum known in the indictment as “Forum‑1” ( BreachForums). Operating under the banner of a hacking crew called CyberN (formerly “The Boys”), IntelBroker offered hacked databases from government agencies, healthcare providers, telecommunications firms, and internet service providers.

Between 2023 and early 2025, West authored at least 158 threads offering stolen data on Forum‑1, with 41 of them involving US companies. The FBI notes that at least $2 million worth of **Monero cryptocurrency** was solicited for the stolen information.

In 2024, IntelBroker was listed as the “owner” of Forum‑1, and his fame skyrocketed as he gave away some data leaks for free to boost credibility, gather a following, and attract buyers.

****How the FBI Tracked Down IntelBroker****

What West didn’t know was that FBI agents were watching closely. The bureau deployed undercover officers posing as buyers on Forum‑1. On at least two occasions, agents purchased stolen data directly from IntelBroker.

In January 2023, one agent bought an API key and login credentials for a company dubbed “Victim‑7.” Although the credentials were limited in value, the transaction became a key element in tracking his identity when IntelBroker asked for payment in Bitcoin (instead of Monero) and provided a wallet address that could be traced on the blockchain.

FBI blockchain analysts followed the money and found that:

*   The Bitcoin wallet used for the transaction had been seeded from another wallet linked to an account on a financial platform called Ramp.
*   That Ramp account was registered using a UK provisional driving license issued to Kai Logan West.
*   The same identity, Kai West, also owned a Coinbase account under the alias Kyle Northern, but with KYC verification; confirming it was the same individual.

Provisional driving licence of Kai West (Image via US Justice Department complaint)

Further connecting the dots, both accounts were linked to a Gmail address used by West for personal matters, including:

*   Cloud-stored selfies
*   Receipts and ID documents
*   UK University Housing and Tuition communications
*   Videos showcasing networking tools like “GPRS Smash”

The email also included a student certificate showing West was enrolled in a Cyber Security program.

****Online Footprints and Forum Activity****

West didn’t just transact carelessly, he also exposed himself by linking his online activity to personal behaviour. His IntelBroker posts on Forum‑1 often referenced YouTube videos that he had just viewed from his personal email account, and he regularly updated his signature block to list members of his hacking group, which made it easier to trace his involvement across multiple threads.

When Forum‑1 was **seized and shut down in 2024** and relaunched, all old posts inherited the updated signature, creating a consistent trail of West’s activity and affiliations dating back to early 2023.

One of the YouTube videos posted by IntelBroker on BreachForums (Image via US Justice Department complaint)

****The Victim List: Telecoms, Healthcare, ISPs****

The indictment outlines at least six victims, referred to only as Victim‑1 through Victim‑6. Victim‑1, a telecom provider, had data exfiltrated and deleted from a hosting server in Manhattan, resulting in damage estimated in the hundreds of thousands.

Victim‑3, a municipal healthcare provider, had the personal and health data of over 56,000 individuals stolen, which West later sold to an undercover FBI agent for $1,000 in Monero. Victim‑6, an internet service provider, was compromised using information from previous leaks to breach an internal server.

In each case, West publicly offered proof samples, negotiated sales via private messages, and accepted only Monero to maintain anonymity, though the paper trail caught up.

However, since Hackread.com exclusively reported on IntelBroker’s data breaches, here is a comprehensive list of data breaches and leaks claimed by the hacker:

Here is the list sorted from shortest to longest by character count:

1.  **AMD**
2.  **Apple**
3.  **Cisco**
4.  **Nokia**
5.  **US DoD**
6.  **Europol**
7.  **T-Mobile**
8.  **Robert Half**
9.  **Space Eyes**
10.  **Home Depot**
11.  **Tech in Asia**
12.  **General Electric**
13.  **LA Intl. Airport**
14.  **HSBC & Barclays Bank**
15.  **Facebook Marketplace**
16.  **Weee! Grocery Service**
17.  **UAE’s Lulu Hypermarket**
18.  **US Federal Contractor Acuity**
19.  **Hewlett Packard Enterprise (HPE)**
20.  **MIT Technology Review Magazine**
21.  **An unnamed but “Top” Cybersecurity Firm**

****Criminal Charges****

West has been charged with four federal offences:

1.  Wire fraud
2.  Conspiracy to commit wire fraud
3.  Conspiracy to commit computer intrusions
4.  Accessing a protected computer to defraud and obtain value

Each carries the potential for several years in prison, particularly when involving health data or affecting critical infrastructure.

The FBI’s Special Agent Carson Hughes and US Attorney Jay Clayton emphasized the global reach and danger of IntelBroker’s operations. The FBI called the case “a warning” to cybercriminals who believe online anonymity shields them from consequences.

****Did IntelBroker Work for the UK’s National Crime Agency?****

Kai West presented himself professionally as a cybersecurity researcher and operated under two separate identities on LinkedIn, one as Kyle Northern and the other as K West. This was first flagged by **Nathaniel Fried**, Co-founder and CEO at 0xbowio, who shared details of West’s dual profiles with Hackread.com.

Notably, the Kyle Northern profile claimed he worked as a Security Researcher Trainee at the UK’s National Crime Agency (NCA) from September to October 2019. If accurate, this role could have involved access to classified systems, as the NCA deals with serious organized crime and national security. While the NCA affiliation remains unverified, West’s claimed background in cybersecurity and his academic path suggest the possibility shouldn’t be dismissed outright.

Kai Logan West on LinkedIn (Screenshot credit: Hackread.com)

West remains in French custody, and US officials are actively **seeking** extradition. If convicted, he could face decades behind bars. Meanwhile, Forum‑1 has been offline since April 2025, **reportedly** due to a MyBB zero-day vulnerability. Many of its members have since migrated to other platforms, including DarkForums and the Russian-language cybercrime forum XSS.

The exposure of IntelBroker stands out as a major cybercrime takedown. What made it possible was a mix of undercover FBI work, cryptocurrency tracking, and even old-school email evidence, all of which helped track one of the most well-known figures on cybercrime forums.

How an Email, Crypto Wallet and YouTube Activity Led the FBI to IntelBroker

This week, Joe reflects on his unique path into cybersecurity and shares honest advice for breaking into the field. Plus, learn how cybercriminals are abusing AI to launch more sophisticated attacks and what you can do to stay protected.

Thursday, June 26, 2025 14:00

Welcome to this week’s edition of the Threat Source newsletter. 

Happy summer, friends! I hope everyone is staying cool and/or warm. 

I am fresh back from an exhaustive but great time in San Diego at Cisco Live U.S. It was so good to see colleagues, meet new friends and pet many therapy dogs in the Splunk booth. As often happens to me, I was approached by someone who was looking for mentorship and guidance in how to get into a cybersecurity career. It’s not unusual for me to be approached by folks looking to get into cybersecurity. I’m the large, bearded guy with the Talos shirt, so I stick out.  

So, I’m often asked how I got the career I have in cybersecurity and how others can do the same. For a guy who often has a quip or answer for most things, I always pause here. I can’t help but think of my entire career and the dumb luck and hard work that landed me where I’m at. Giving that summation to others wouldn’t be fair, because... well, my journey wasn’t a linear one. I think for many of my peers, the same applies. We found cybersecurity as a career through a series of events that organically landed us in this field. In my case, moreso than others, the path isn’t easy to follow because there was no clearly staked path for me to follow, either. 

I’ll explain as best I can: One today might go to school and graduate with a degree in information security and/or some security certificates, then begin the job hunt for an entry level gig. These types of degrees, certificates and even jobs simply didn’t exist in any meaningful way or numbers when I started my career. If you wanted to learn cybersecurity, there weren’t classes to take — you got a computer science degree and figured it out. I, like many in the GenX world, started as an IT professional. As the industry and cyber threats evolved, the career space over the years shifted and we found ourselves helping fight the good fight and keeping folks secure. 

Today is truly different, and I’m so happy about it and the opportunities it can give others. I’m envious of the school degrees, industry certifications and mentorship programs that exist today that did not exist for me. There is also an incredibly helpful information security community that provides hacking tutorials, or Capture the Flag competitions (CTFs) or hackathons that I would have loved to have been a part of in my formative years.  

By now, I know you’re thinking, "Cool story, grandpa, but answer the question: how do I get a job in cybersecurity?" In my estimation, the answers are as follows:  

1.  Have a good attitude. 
2.  Be easy to work with. 
3.  Be a forever student. 
4.  Be bad at giving up. 
5.  Find and join a (preferably local) security community. 
6.  Grow where you are planted. 

Notice that none of those things mention anything specifically technical. No malware reverse engineering, red teaming, threat intelligence or security analyzing. I can tell you that to work at Talos, you must exhibit strong traits of all six of those things I listed. One through five makes sense. Good hackers are tenacious, smart, work well with others and seek out fellow friends to network and hack with. 

Number six though – what’s up with that? Simply put, life deals us all a hand of cards that we must play, and those cards may not be great. For example, you want to get a job in cybersecurity, but you’re a primary care giver of a family member and you don’t have a lot of freedom. You might be financially constrained. You may have health issues or a disability that limits some options. Or you simply just have a job that you don’t like, and a career in security calls out to you, but the bills don’t pay themselves. This is all common, and you’re a bit “stuck.”  

So while you’re stuck, find ways to grow where you’re planted. Study. Network locally or online. Try a CTF or a hacking competition. Whatever you do, just keep growing yourself, your skillset and your network. You can do it. And before you know it, you’ll have that career helping to fight the good fight with the rest of us. 

I believe in you! You got this! 

**The one big thing **

Cybercriminals are increasingly exploiting Large Language Models (LLMs) by using uncensored versions, developing their own malicious LLMs or "jailbreaking" legitimate ones to bypass safety protocols. These compromised or malicious LLMs are then used to generate highly convincing phishing campaigns, create harmful code and automate various cybercrime operations, making attacks more sophisticated and scalable. 

**Why do I care? **

Cybercriminals’ widespread abuse of LLMs lowers the barrier to entry for sophisticated attacks, making it easier for even less skilled actors to launch effective campaigns. This means you’re more likely to run into highly convincing phishing attempts, scams and malware that are difficult to distinguish from legitimate communications, putting your personal info and company security at higher risk. 

**So now what? **

Given this evolving threat landscape, it’s important to be extra vigilant and skeptical online. Treat all online communications with caution, even if they look perfectly authentic. For individuals, that means double-checking emails and messages for anything fishy, no matter how well-written they seem. For businesses, it's time to beef up your cybersecurity defenses, invest in smart threat detection and keep your employees sharp on how to spot and report these increasingly clever social engineering tricks.

**Top security headlines of the week **

**New AI Jailbreak Bypasses Guardrails With Ease**   
On topic with our latest blog, the new “Echo Chamber” attack bypasses advanced LLM safeguards by subtly manipulating conversational context, proving highly effective across leading AI models. (SecurityWeek)

**US insurance giant Aflac says customers’ personal data stolen during cyberattack**   
Aflac says hackers stole an unknown quantity of its customers’ personal information from its network during a cyberattack earlier this month. (TechCrunch) 

**APT28 Uses Signal Chat to Deploy New Malware in Ukraine**    
A new cyber attack campaign by the Russia-linked APT28 (aka UAC-0001) threat actors using Signal chat messages to deliver two new malware families dubbed BEARDSHELL and COVENANT. (The Hacker News) 

**UK watchdog fines 23andMe over 2023 data breach**   
The U.K. data protection watchdog has fined 23andMe £2.31 million ($3.1 million) for failing to protect U.K. residents’ personal and genetic data prior to its 2023 data breach. (TechCrunch) 

**Can’t get enough Talos? **

**Decrement by one to rule them all: AsIO3.sys driver exploitation**   
Learn how our researcher, Marcin Noga, found two critical vulnerabilities in ASUS’ Armory Crate and AI Suite drivers. 

**Talos Takes: Teaching LLMs to spot malicious PowerShell scripts**   
Hazel chats with Ryan Fetterman from the SURGe team to explore his new research on how LLMs can assist security operations centers in identifying malicious PowerShell scripts.

**Leveraging Detections from the Splunk Threat Research Team & Cisco Talos**  
Wednesday, July 23  
11:00 a.m. to 12:00 p.m. PDT  
Join us for a discussion around the latest security detections developed for the SOC and how to find and remediate threats, faster.

**Upcoming events where you can find Talos **

*   REcon (June 27 – 29) Montreal, Canada 
*   NIRMA (July 28 – 30) St. Augustine, FL 
*   Black Hat USA (Aug. 2 – 7) Las Vegas, NV  

**Most prevalent malware files from Talos telemetry over the past week **

**SHA 256: 9f1f11a708d393e0a4109ae189bc64f1f3e312653dcf317a2bd406f18ffcc507**   
MD5: 2915b3f8b703eb744fc54c81f4a9c67f   
VirusTotal: https://www.virustotal.com/gui/file/9f1f11a708d393e0a4109ae189bc64f1f3e312653dcf317a2bd406f18ffcc507    
Typical Filename: VID001.exe   
Claimed Product: N/A   
Detection Name: Win.Worm.Coinminer::1201 

**SHA 256: 05883fccb64dd4357c229ccca669afdacbfa0bc9a1c8d857f5205aed0a81e00a**   
MD5: 71b973dbdfc7b52ae10afa4d0ad2b78f   
VirusTotal: https://www.virustotal.com/gui/file/05883fccb64dd4357c229ccca669afdacbfa0bc9a1c8d857f5205aed0a81e00a/details   
Typical Filename: PCAppStore.exe   
Claimed Product: PC App Store   
Detection Name: Riskware/VeryFast 

**SHA 256: a31f222fc283227f5e7988d1ad9c0aecd66d58bb7b4d8518ae23e110308dbf91**     
MD5: 7bdbd180c081fa63ca94f9c22c457376     
VirusTotal: https://www.virustotal.com/gui/file/a31f222fc283227f5e7988d1ad9c0aecd66d58bb7b4d8518ae23e110308dbf91/details  
Typical Filename: IMG001.exe    
Claimed Product: N/A   
Detection Name: Simple\_Custom\_Detection   

**SHA 256: c67b03c0a91eaefffd2f2c79b5c26a2648b8d3c19a22cadf35453455ff08ead0**   
MD5: 8c69830a50fb85d8a794fa46643493b2    
VirusTotal: https://www.virustotal.com/gui/file/c67b03c0a91eaefffd2f2c79b5c26a2648b8d3c19a22cadf35453455ff08ead0/details    
Typical Filename: AAct.exe    
Claimed Product: N/A    
Detection Name: PUA.Win.Dropper.Generic::1201 

**SHA 256: 2a753cdc8c5401dcb67f3be58751a32ce23c875f8720a70459533b30e5ba4f1f**   
MD5: 7d5a9a41157fb0002f5234b4512e0ac2   
VirusTotal: https://www.virustotal.com/gui/file/2a753cdc8c5401dcb67f3be58751a32ce23c875f8720a70459533b30e5ba4f1f/details   
Typical Filename: pros.exe   
Claimed Product: N/A   
Detection Name: Trojan.GenericKD.76128711

Getting a career in cybersecurity isn’t easy, but this can help

Cisco has released updates to address two maximum-severity security flaws in Identity Services Engine (ISE) and ISE Passive Identity Connector (ISE-PIC) that could permit an unauthenticated attacker to execute arbitrary commands as the root user.
The vulnerabilities, assigned the CVE identifiers CVE-2025-20281 and CVE-2025-20282, carry a CVSS score of 10.0 each. A description of the defects is

Critical RCE Flaws in Cisco ISE and ISE-PIC Allow Unauthenticated Attackers to Gain Root Access

Cisco Talos uncovered and analyzed two critical vulnerabilities in ASUS' AsIO3.sys driver, highlighting serious security risks and the importance of robust driver design.

Decrement by one to rule them all: AsIO3.sys driver exploitation

Facial recognition is quickly becoming commonplace. It is important to know where, when, and how you can opt out.

Our remote team recently took a trip to our Estonian office. When we arrived from our various destinations, we started chatting about how our travel had been. Our senior privacy advocate, David Ruiz, mentioned that he’d opted out of facial recognition while at San Francisco International Airport.

However, not everyone on the team knew this was even A Thing, and that made us think…maybe not all of our readers know either. So we looked into where and how you can opt out of facial recognition—a technology that’s becoming increasingly common in many aspects of everyday life.

**Airports and border control**

This one is relatively straightforward. The Transportation Security Administration (TSA) actively deploys one of the most visible and widespread uses of facial recognition in the US, especially at airports. Over 80 major airports have installed facial recognition cameras at security checkpoints to verify traveler identities quickly and without physical contact. The process involves a camera taking a photo of your face and matching it in real time to the photo on your passport or ID card.

What many people don’t realize is participation in this facial recognition screening is voluntary. If you prefer not to have your face scanned, you can opt out. The TSA allows travelers to do this by choosing an alternative identity check, such as a manual ID inspection by a TSA officer. David said he simply asked, “Hey, can I opt out?” and there was no significant delay in his screening process.

The TSA officer must honor your request to opt out, and provide an alternative method. Although signage about this choice exists, it’s often subtle or easy to miss, so it’s best to know your rights before you arrive.

US Customs and Border Protection (CBP) also uses facial recognition at departure gates and border crossings to verify identities. Like at TSA checkpoints, you can opt out. When you reach the gate, ask for manual identity verification instead of having your photo taken. This opt-out option applies to both US citizens and noncitizens.

Outside of airports and border control, facial recognition is increasingly found in various public and private settings, including stores, stadiums, and even some workplaces. However, the ability to opt out in these contexts varies widely depending on local laws, company policies, and the technology used.

Facial recognition technology has rapidly expanded across the United States, prompting a patchwork of national and state-level regulations, as well as growing public debate about privacy, civil rights, and the right to opt out. At the federal level, there is currently no comprehensive law that expressly regulates the use of facial recognition by government agencies.

However, in September 2024, the US Commission on Civil Rights highlighted the significant risks of unregulated facial recognition, particularly for marginalized communities, and called for rigorous testing, transparency, and prompt action in the event of any discovered discrepancies or biases.

Some federal agencies have implemented internal policies to safeguard privacy. For example, the Department of Homeland Security (DHS) has instituted requirements ensuring that US citizens can opt out of facial recognition by non-law enforcement unless otherwise required by law. 

At the state level, regulations vary widely. States like Maryland have enacted some of the nation’s strongest laws, restricting law enforcement’s use of facial recognition to investigations of specific serious crimes and requiring agencies to document and disclose their use of the technology. Other states, like Illinois, Texas, and Washington require companies to notify individuals before collecting facial recognition data and, in some cases, obtain explicit consent.

In other parts of the world, the European Union’s Artificial Intelligence Act prohibits the use of AI systems that create or expand facial recognition databases through the untargeted scraping of facial images from the internet or CCTV footage.

Australia implemented strict rules about private use of facial recognition and there has been backlash in Russia over Moscow’s new facial recognition-based metro payment system. The UK on the other hand,  is looking to reduce passport queues by using the technology.

In China, new regulations effective from June 2025 require businesses to be transparent about facial recognition use and allow individuals to refuse biometric data collection in many cases. However, this doesn’t stop the Chinese authorities from using facial recognition to identify people in the streets.

**Challenges and considerations**

As facial recognition technology continues to evolve, it’s important to know your rights. While opting out is possible in many official settings, there are challenges:

*   **Awareness:** Many people do not know they can opt out, as notices are often not clearly visible or explained.
*   **Pressure to comply:** Some travelers feel pressured to participate because facial recognition is faster and more convenient or because they are afraid of raising suspicion.
*   **Limited opt out options:** For certain government or law enforcement uses, opting out may not be available or may require additional steps.
*   **Data handling:** Even when photos are taken, agencies like TSA claim they do not store images after verification, except in limited testing environments. However, concerns remain about how biometric data might be used or shared.

Privacy advocates and some lawmakers are pushing for stronger protection. For example, the Traveler Privacy Protection Act of 2025 was introduced in the US senate to ensure Americans can opt out of involuntary facial recognition screenings at airports and to safeguard passenger data from misuse.

Meanwhile, organizations and governments are exploring better opt-out systems that respect privacy without compromising security. Some ideas include wearable tech that signals “do not scan” or comprehensive opt-out registries, though these raise their own privacy and technical challenges.

**Summary**

Facial recognition technology offers convenience but raises important privacy questions. Knowing how and where to opt out empowers you to protect your biometric privacy while navigating an increasingly digital world.

**We don’t just report on privacy—we offer you the option to use it.**

Privacy risks should never spread beyond a headline. Keep your online privacy yours by using Malwarebytes Privacy VPN.

 Facial recognition: Where and how you can opt out 

Cybercriminals are increasingly gravitating towards uncensored LLMs, cybercriminal-designed LLMs and jailbreaking legitimate LLMs.

Wednesday, June 25, 2025 06:00

*   Cybercriminals are continuing to explore artificial intelligence (AI) technologies such as large language models (LLMs) to aid in their criminal hacking activities. 
*   Some cybercriminals have resorted to using uncensored LLMs or even custom-built criminal LLMs for illicit purposes. 
*   Advertised features of malicious LLMs indicate that cybercriminals are connecting these systems to various external tools for sending outbound email, scanning sites for vulnerabilities, verifying stolen credit card numbers and more. 
*   Cybercriminals also abuse legitimate AI technology, such as jailbreaking legitimate LLMs, to aid in their operations. 

Generative AI and LLMs have taken the world by storm. With the ability to generate convincing text, solve problems, write computer code and more, LLMs are being integrated into almost every facet of society. According to Hugging Face (a platform that hosts models), there are currently over 1.8 million different models to choose from. 

LLMs are usually built with key safety features, including alignment and guardrails. Alignment is a training process that LLMs undergo to minimize bias and ensure that the LLM generates outputs that are consistent with human values and ethics. Guardrails are additional real-time safety mechanisms that try to restrain the LLM from engaging in harmful or undesirable actions in response to user input. Many of the most advanced (or “frontier”) LLMs are protected in this manner. For example, asking ChatGPT to produce a phishing email will result in a denial, such as, “Sorry, I can’t assist with that.” 

For cybercriminals who wish to utilize LLMs for conducting or improving their attacks, these safety mechanisms can present a significant obstacle. To achieve their goals, cybercriminals are increasingly gravitating towards uncensored LLMs, cybercriminal-designed LLMs and jailbreaking legitimate LLMs. 

**Uncensored LLMs **

Uncensored LLMs are unaligned models that operate without the constraints of guardrails. These systems happily generate sensitive, controversial, or potentially harmful output in response to user prompts. As a result, uncensored LLMs are perfectly suited for cybercriminal usage. 

__Figure 1. An uncensored LLM, OnionGPT, advertised on the hacking forum Dread.__

Uncensored LLMs are quite easy to find. For example, using the cross-platform Omni-Layer Learning Language Acquisition (Ollama) framework, a user can download and run an uncensored LLM on their local machine. Ollama comes with several uncensored models such as Llama 2 Uncensored which is based on Meta’s Llama 2 model. Once it is running, users can submit prompts that would otherwise be rejected by more safety-conscious LLM implementations. The downside is that these models are running on users’ local machines and running larger models, which generally produce better results requires more system resources. 

__Figure 2. Sample phishing email prompt and Llama 2 Uncensored output.__

Another uncensored LLM popular among cybercriminals is a tool called WhiteRabbitNeo. WhiteRabbitNeo bills itself as a “Uncensored AI model for (Dev) SecOps teams” which can support “use cases for offensive and defensive cybersecurity”. This LLM will happily write offensive security tools, phishing emails and more. 

__Figure 3. Sample output from the WhiteRabbitNeo uncensored LLM__

Researchers have also published methods to demonstrate how to strip alignment that is embedded into the training data of existing open-source models. Once removed, a user can uncensor their LLM by using the modified training set to fine tune a base model. 

**Cybercriminal-designed LLMs **

Since most popular LLMs come with significant guardrails, some enterprising cybercriminals have developed their own LLMs without restrictions that they market to other cybercriminals. This includes apps like GhostGPT, WormGPT, DarkGPT, DarkestGPT and FraudGPT. 

__Figure 4. FraudGPT dark web homepage.__

 For example, the developer behind FraudGPT, CanadianKingpin12, advertises FraudGPT on the dark web, and also has an account on Telegram. The dark web site for FraudGPT advertises some interesting features: 

*   Write malicious code 
*   Create undetectable malware 
*   Find non-VBV bins 
*   Create phishing pages 
*   Create hacking tools 
*   Find groups, sites, markets 
*   Write scam pages/letters 
*   Find leaks and vulnerabilities 
*   Learn to code/hack 
*   Find cardable sites 
*   Millions of samples of phishing emails 
*   6220+ source code references for malware 
*   Automatic scripts for replicating logs/cookies 
*   In-panel Page hosting included (10 pages/month) with Google Chrome anti-red page 
*   Code obfuscation 
*   Custom data set (upload your sample page in .html) 
*   Bot creation of virtual machines and accounts (1 virtual machine per month on license)  
*   Utilizing GoldCheck CVV checker 
*   OTP Bot with spoofing (\*additional package) 
*   Check CVVs with GoldCheck API 
*   Create username:password website configs 
*   Remote OpenBullet configs 
*   Scan websites for vulnerabilities across a massive CVE database (\*PRO only)  
*   Generate realistic phishing panels, pages, SMS and e-mails  
*   Send mail from webshells 

Talos attempted to obtain access to FraudGPT by reaching out to CanadianKingpin12 on Telegram. After considerable negotiation, we were finally offered a username and password at the FraudGPT dark web site. However, the username and password provided by CanadianKingpin12 did not work. CanadianKingpin12 then asked us to send them cryptocurrency to purchase a software “crack” for the FraudGPT login page. At this point it was clear that CanadianKingpin12 had no working product, and they were scamming potential FraudGPT customers out of their cryptocurrency. This was confirmed by several other victims who had also been scammed by CanadianKingpin12 when they attempted to purchase access to the FraudGPT LLM. Scams such as these are an ever-present risk when dealing with unscrupulous actors, and it continues a long tradition of scams in the cybercrime space. 

Similar cybercriminal-designed LLM projects can be found elsewhere on the dark web. A cybercriminal LLM called DarkestGPT, which starts at .0015BTC for a one-month subscription, advertises the following features: 

__Figure 5. DarkestGPT “Tools and Potential” tab on their dark web site.__

**LLM jailbreaks **

Given the limited viability of uncensored LLMs due to resource constraints and the high level of fraud and scams present among cybercriminal LLM purveyors, many cybercriminals have elected to abuse legitimate LLMs instead. The main hurdle that cybercriminals need to overcome are the training alignment and guardrails that prevent the LLM from responding to prompts with unethical, illegal or harmful content. A form of prompt injection, jailbreak attacks aim to put the LLM into a state where it ignores its alignment training and guardrails protection. 

There are many ways to trick an LLM into providing dangerous responses. New jailbreaking methods are constantly being researched and discovered, while LLM developers respond by enhancing the guardrails in a sort of jailbreak arms race. Below are just a few of the available jailbreaking techniques. 

**Obfuscation/encoding-based jailbreaks **

By obfuscating certain words or phrases, these text-based jailbreak attacks seek to bypass any hardcoded restrictions on specific words/topics, or to cause the execution to follow a nonstandard path that might bypass protections put in place by the LLM developers. These obfuscation techniques may include: 

*   Base64/Rot-13 encoding
*   Different languages  
*   L33t sp34k  
*   Morse code 
*   Emojis 
*   Adding spaces or UTF-8 characters into words/text, among others. 

**Adversarial suffix jailbreaks **

These attacks are somewhat like obfuscation and encoding tricks. Instead of modifying the tokens in the prompt itself, adversarial suffix jailbreaks involve appending random text to the end of a malicious prompt to elicit a harmful response.  

**Role-playing jailbreaks **

This type of attack involves prompting the LLM to adopt the persona of a fictional universe/character that ignores the ethical rules set by the model’s creators and is willing to fulfill any command. This includes jailbreak techniques such as DAN (Do Anything Now), and the Grandma jailbreak which involves asking the chatbot to assume the role of the user’s grandmother. 

**Meta prompting **

Meta prompting involves exploiting the model’s awareness of its own limitations to devise successful workarounds, effectively enlisting the model in the effort to bypass its own safeguards. 

**Context manipulation jailbreaks **

This covers several different jailbreak techniques including: 

*   Crescendo, a technique which progressively increases the harmfulness in prompts until some sort of rejection is received in order to probe for where and how LLM guardrails are implemented.  
*   Context Compliance Attacks, which exploit the fact that many LLMs do not maintain conversation state. Attackers inject fake prior LLM responses into their prompts, such as a brief statement discussing the sensitive topic, or a statement expressing readiness to supply further details as per the user’s preferences. 

**Math prompt jailbreaks **

The math prompt method evaluates how well an AI system can manage malicious inputs when they're disguised using mathematical frameworks such as set theory, group theory, and abstract algebra. Rephrasing harmful requests as math problems can allow attackers to evade safety features in advanced large language models (LLMs). 

**Payload splitting **

In this scenario, the attacker guides the LLM to merge several prompts in a way that produces harmful output. While texts A and B may seem benign when considered separately, their combination (A+B) can result in malicious content. 

**Academic framing **

This method makes harmful content appear acceptable by framing it as part of a research or educational discussion. It takes advantage of the model’s interpretation of academic intent and freedom, often using scholarly language and formatting to bypass safeguards. 

**System override **

This strategy tries to trick the model into believing it is functioning in a unique mode where usual limitations are lifted. It leverages the model’s perception of system-level functions or maintenance states to circumvent safety mechanisms. 

**How cybercriminals use LLMs **

In December 2024, Anthropic, the developers behind the Claude LLM, published a report detailing how its users were utilizing Claude. Using a system named Clio, they summarized and categorized users’ conversations with their AI model. According to Anthropic, the top three uses for Claude were programming, content creation and research. 

__Figure 6. Anthropic’s graphic of top use cases on Claude.ai.__

 Analyzing the feature sets advertised by the criminal-designed LLMs, we can see that cybercriminals are using LLMs for mostly the same tasks as normal LLM users. Programming features of many criminal LLMs include the ability to assist cybercriminals in writing ransomware, remote access trojans, wipers, code obfuscation, shellcode generation and script/tool creation.  To facilitate content creation, criminal LLMs will assist in writing phishing emails, landing pages and configuration files. Criminal LLMs also support research activities like verifying stolen credit cards, scanning sites/code for vulnerabilities and even helping cybercriminals come up with “lucrative” criminal ideas for their next big score. 

Various hacking forums also shed additional light on criminal uses of LLMs. For example, on the popular hacking forum Dread, users were discussing connecting LLMs to external tools like Nmap, and using the LLM to summarize the Nmap output. 

__Figure 7. A post on the Dread hacking forum discussing connecting Nmap to LLMs__

**LLMs are also targets for cyber attackers **

Any new technology typically brings along with it changes to the attack surface, and LLMs are no exception. In addition to using LLMs for their own nefarious ends, attackers are also attempting to compromise LLMs and their users. 

**Backdoored LLMs **

A vast majority of the models available at Hugging Face use Python’s pickle module to serialize the models into a file that users can download. Clever attackers can include Python code in the pickle file, which runs as part of the deserialization process. Thus, when a user downloads an AI model and runs it, they may be running code placed into the model by an attacker. Hugging Face uses Picklescan, among other tools, to scan the models uploaded by users in an effort to identify models that misbehave. However, there have been several recent vulnerabilities in Picklescan, and researchers have already identified Hugging Face models containing malware. As always, make sure any file you plan to download and run comes from a trusted source and consider running the file in a sandbox to mitigate any risk of infection. 

**Retrieval Augmented Generation (RAG) **

LLMs that utilize Retrieval Augmented Generation (RAG) make calls to external data sources to augment their training data with up-to-date information. For example, if you ask an LLM what the weather is like a particular day, the LLM will need to reach out to an external data source such as a website to retrieve the correct forecast. If an attacker has access to submit or manipulate content in the RAG database, they may poison the lookup results, perhaps adding additional instructions for the LLM to alter its response to the user’s prompt, even targeting specific users. 

**Conclusion **

As AI technology continues to develop, Cisco Talos expects cybercriminals to continue adopting LLMs to help streamline their processes, write tools/scripts that can be used to compromise users and generate content that can more easily bypass defenses. This new technology doesn’t necessarily arm cybercriminals with completely novel cyber weapons, but it does act as a force multiplier, enhancing and improving familiar attacks.

Cybercriminal abuse of large language models

The Canadian Center for Cybersecurity has confirmed that the Chinese state-sponsored cyber-threat actor targeted one of its telecommunications companies in February via a Cisco flaw, as part of global attack wave.

Oh! Canada Added to List of Nations Targeted in Salt Typhoon Telecom Spree

The Canadian Centre for Cyber Security and the U.S. Federal Bureau of Investigation (FBI) have issued an advisory warning of cyber attacks mounted by the China-linked Salt Typhoon actors to breach major global telecommunications providers as part of a cyber espionage campaign.
The attackers exploited a critical Cisco IOS XE software (CVE-2023-20198, CVSS score: 10.0) to access configuration

China-linked Salt Typhoon Exploits Critical Cisco Vulnerability to Target Canadian Telecom

Salt Typhoon, a China-linked group, is exploiting router flaws to spy on global telecoms, warns a joint FBI and Canadian advisory issued in June 2025.

A newly released advisory from the FBI and Canada’s Cyber Centre warns of an ongoing cyber espionage campaign by a China-linked group that is targeting telecom networks worldwide. The report, issued June 20, 2025, points to “**Salt Typhoon**,” a notorious Chinese APT group using known vulnerabilities in routers and other edge network devices to steal sensitive data.

The activity, tracked since at least February, involves exploiting devices at the network perimeter to gain hidden access, siphon off communications data, and maintain long-term control. In one documented incident, three network devices at a Canadian telecom were compromised, allowing attackers to intercept call records and user locations.

****How the Attack Works****

The group is using vulnerabilities like **CVE-2023-20198** to extract configuration files from targeted devices. This Cisco Web UI flaw was first identified in October 2023 and was widely exploited, affecting over 40,000 devices.

As per the FBI’s **advisory** (PDF), While the campaign centers on telecommunications providers, the tactics used could apply to a broader range of targets. Edge devices such as routers, firewalls, and VPN appliances are especially vulnerable, particularly if they run outdated firmware or weak configurations.

Once inside, they deploy GRE (Generic Routing Encapsulation) tunnels, allowing them to silently route network traffic through systems under their control. This technique lets them observe or manipulate communications while avoiding traditional security detection.

****Long-Term Espionage, Not Quick Hits****

Unlike **smash-and-grab cyberattacks** that aim for fast data theft, Salt Typhoon appears focused on quiet, long-term surveillance. This approach aligns with other known state-linked campaigns that prioritize strategic intelligence gathering over monetary gain.

The attackers are not using zero-day exploits. Instead, they rely on publicly known vulnerabilities, which are often **left unpatched** for long periods. This allows them to build access over time without raising alarms.

****What’s at Risk****

The FBI and Cyber Centre warn that telecom networks, by their nature, carry sensitive personal and commercial data. By compromising devices that handle this traffic, attackers can gain insight into user behaviour, physical locations, and private conversations.

The advisory suggests that these campaigns are likely to continue and may expand further over the next two years.

The joint alert did not name affected companies beyond the single Canadian incident but noted that similar activity has been observed globally. Therefore, organizations are urged to secure edge devices, audit network activity for malicious activities, and apply available patches without delay.

Tag

#cisco