PRESS RELEASE

PITTSBURGH, PA – October 1, 2024 – ForAllSecure, the world's most advanced application security testing company, today announced it is changing its corporate name to Mayhem Security (“Mayhem”), signaling a new era of growth and opportunity aligned with its award-winning Mayhem Application Security platform.

Founded by a team of researchers from Carnegie Mellon, the company’s focus has evolved from research, development, and education to a product company centered around its Mayhem platform that quickly went from a Defense Advanced Research Project Agency (DARPA) Cyber Grand Challenge prototype to an in-demand commercialized AI-driven application security platform. Today, the Mayhem platform has been integrated into thousands of open-source projects, building a library of behavioral tests, identifying new zero-days, and helping defend against software supply chain threats. The name change follows record product achievements, with platform ARR rising 275% year over year and 78% of customers expanding their Mayhem footprint at or before their first subscription renewal.

“ForAllSecure has a long, successful history, from winning the DARPA Grand Challenge to dedicating ourselves to research and innovation in the cybersecurity industry through Mayhem Heroes, hackathons, and consulting,” said David Brumley, CEO of Mayhem. “Our new name and focus mark an important evolution for us as the Mayhem brand becomes synonymous with the platform that is transforming API security testing and has powered our growth. In fact, for several years, it was the majority of our revenue. Our new positioning is a natural next step as we continue the hard work of our dedicated researchers and hackers who will continue to push out innovative research and prototype new ways to defend software.” 

The past year has been a banner year, with the company achieving key innovation milestones. Most notably, Mayhem released Mayhem Dynamic software bill of materials (SBOM), which brings Mayhem’s runtime intelligence to the world of software composition analysis (SCA) and SBOM by looking at an application’s actual behavior to find only real, exploitable vulnerabilities, eliminating triage and investigations, and reducing false positives to increase developer velocity and minimize application risks. Mayhem re-architected its symbolic executor to test and triage 60% faster, released support for Windows-based applications, and launched a beta of automated harnessing for embedded systems. 

Under the name Mayhem Security, the company will continue to collaborate with the government and the industry to advance cybersecurity and revolutionize how organizations approach cybersecurity by automating the process of finding and fixing software vulnerabilities.

For more information, visit https://www.mayhem.security/.

You May Also Like

Introducing Mayhem: ForAllSecure Unveils New Name and Company Focus

An information disclosure vulnerability exists in open-webui version 0.3.8. The vulnerability is related to the embedding model update feature under admin settings. When a user updates the model path, the system checks if the file exists and provides different error messages based on the existence and configuration of the file. This behavior allows an attacker to enumerate file names and traverse directories by observing the error messages, leading to potential exposure of sensitive information.

**Exploitability Metrics**

Attack Vector: This metric reflects the context by which vulnerability exploitation is possible. This metric value (and consequently the resulting severity) will be larger the more remote (logically, and physically) an attacker can be in order to exploit the vulnerable system. The assumption is that the number of potential attackers for a vulnerability that could be exploited from across a network is larger than the number of potential attackers that could exploit a vulnerability requiring physical access to a device, and therefore warrants a greater severity.

Attack Complexity: This metric captures measurable actions that must be taken by the attacker to actively evade or circumvent existing built-in security-enhancing conditions in order to obtain a working exploit. These are conditions whose primary purpose is to increase security and/or increase exploit engineering complexity. A vulnerability exploitable without a target-specific variable has a lower complexity than a vulnerability that would require non-trivial customization. This metric is meant to capture security mechanisms utilized by the vulnerable system.

Attack Requirements: This metric captures the prerequisite deployment and execution conditions or variables of the vulnerable system that enable the attack. These differ from security-enhancing techniques/technologies (ref Attack Complexity) as the primary purpose of these conditions is not to explicitly mitigate attacks, but rather, emerge naturally as a consequence of the deployment and execution of the vulnerable system.

Privileges Required: This metric describes the level of privileges an attacker must possess prior to successfully exploiting the vulnerability. The method by which the attacker obtains privileged credentials prior to the attack (e.g., free trial accounts), is outside the scope of this metric. Generally, self-service provisioned accounts do not constitute a privilege requirement if the attacker can grant themselves privileges as part of the attack.

User interaction: This metric captures the requirement for a human user, other than the attacker, to participate in the successful compromise of the vulnerable system. This metric determines whether the vulnerability can be exploited solely at the will of the attacker, or whether a separate user (or user-initiated process) must participate in some manner.

**Vulnerable System Impact Metrics**

Confidentiality: This metric measures the impact to the confidentiality of the information managed by the VULNERABLE SYSTEM due to a successfully exploited vulnerability. Confidentiality refers to limiting information access and disclosure to only authorized users, as well as preventing access by, or disclosure to, unauthorized ones.

Integrity: This metric measures the impact to integrity of a successfully exploited vulnerability. Integrity refers to the trustworthiness and veracity of information. Integrity of the VULNERABLE SYSTEM is impacted when an attacker makes unauthorized modification of system data. Integrity is also impacted when a system user can repudiate critical actions taken in the context of the system (e.g. due to insufficient logging).

Availability: This metric measures the impact to the availability of the VULNERABLE SYSTEM resulting from a successfully exploited vulnerability. While the Confidentiality and Integrity impact metrics apply to the loss of confidentiality or integrity of data (e.g., information, files) used by the system, this metric refers to the loss of availability of the impacted system itself, such as a networked service (e.g., web, database, email). Since availability refers to the accessibility of information resources, attacks that consume network bandwidth, processor cycles, or disk space all impact the availability of a system.

**Subsequent System Impact Metrics**

Confidentiality: This metric measures the impact to the confidentiality of the information managed by the SUBSEQUENT SYSTEM due to a successfully exploited vulnerability. Confidentiality refers to limiting information access and disclosure to only authorized users, as well as preventing access by, or disclosure to, unauthorized ones.

Integrity: This metric measures the impact to integrity of a successfully exploited vulnerability. Integrity refers to the trustworthiness and veracity of information. Integrity of the SUBSEQUENT SYSTEM is impacted when an attacker makes unauthorized modification of system data. Integrity is also impacted when a system user can repudiate critical actions taken in the context of the system (e.g. due to insufficient logging).

Availability: This metric measures the impact to the availability of the SUBSEQUENT SYSTEM resulting from a successfully exploited vulnerability. While the Confidentiality and Integrity impact metrics apply to the loss of confidentiality or integrity of data (e.g., information, files) used by the system, this metric refers to the loss of availability of the impacted system itself, such as a networked service (e.g., web, database, email). Since availability refers to the accessibility of information resources, attacks that consume network bandwidth, processor cycles, or disk space all impact the availability of a system.

GHSA-mq92-jr35-ffpc: open-webui allows enumeration of file names and traversal of directories by observing the error messages

An Insecure Direct Object Reference (IDOR) vulnerability exists in open-webui/open-webui version v0.3.8. The vulnerability occurs in the API endpoint `http://0.0.0.0:3000/api/v1/memories/{id}/update`, where the decentralization design is flawed, allowing attackers to edit other users' memories without proper authorization.

GHSA-xcvc-5hgv-phqg: open-webui Insecure Direct Object Reference (IDOR) vulnerability

In version v0.3.8 of open-webui/open-webui, the endpoint /api/pipelines/upload is vulnerable to arbitrary file write and delete due to unsanitized file.filename concatenation with CACHE_DIR. This vulnerability allows attackers to overwrite and delete system files, potentially leading to remote code execution.

GHSA-54f4-v6v9-9q82: open-webui allows writing and deleting arbitrary files

PRESS RELEASE

ROCHESTER, N.Y., Oct. 9, 2024 /PRNewswire/ -- CYRISMA, an all-in-one risk management platform, announced its Series A financing today. Led by Blueprint Equity, with participation from SaaS Venture and Golden Ventures, the funding will accelerate CYRISMA's platform development, fuel customer success, and expand sales and marketing initiatives.

CYRISMA accelerates security programs for MSPs by providing a cost-effective, all-in-one solution to identify, prioritize, and remediate vulnerabilities, track compliance requirements, and manage AI security risk.

"In partnering with Blueprint Equity, we are excited to leverage their expertise and resources to further enhance our platform and support our customers," said Liam Downward, co-founder and CPO of CYRISMA. "This investment will allow us to continue delivering an affordable and comprehensive risk management solution, empowering MSPs to protect their clients effectively. Additionally, it enables us to enter into new markets, expanding our reach and increasing brand awareness."

Blueprint Equity's Sheldon Lewis, who will join CYRISMA's Board of Directors, commented, "With the rise of security threats for SMBs, there's been an increasing number of businesses outsourcing their cybersecurity to MSPs. This has accelerated demand in the market for strong, multi-tenant cybersecurity solutions for MSPs to best serve their clients." He added, "We were drawn to the breadth of the CYRISMA platform and their strong customer satisfaction. Oliver Downward, CEO, Liam, and the team have an unparalleled insight into the MSP market, and we're honored to partner with CYRISMA during their next phase of growth."

About CYRISMA 

CYRISMA is an all-in-one risk management platform for Managed Service Providers. With the rise in security threats and demand for cybersecurity services, CYRISMA provides MSPs with an effective, all-in-one solution to manage their cybersecurity initiatives for clients in a multi-tenant platform. To schedule a demo of CYRISMA, please visit https://www.cyrisma.com/.

About Blueprint Equity

Blueprint Equity provides expansion capital to high-growth, capital-efficient enterprise software and technology-enabled services businesses worldwide. Blueprint has $275 million of assets under management and is based in La Jolla, CA. For more information, please visit www.onblueprint.com.

CYRISMA Secures $7M Growth Equity Financing led by Blueprint Equity

The tack highlights bad actors' interest in trusted development and collaboration platforms — and their users.

Source: Tada Images via Shutterstock

Trusted and widely used software development and collaboration platforms like GitHub and GitLab have become both targets of and vehicles for a growing range of malicious activity.

The latest manifestations of that trend include a malware distribution campaign involving legitimate GitHub repositories and the availability this week of an exploit for a vulnerability that allows an attacker to gain access as any user of GitLab.

The first is an example of how attackers are exploiting the trusted reputation of platforms like GitHub to try and sneak malware past endpoint detection mechanisms. The GitLab vulnerability, meanwhile, highlights the growing exposure to organizations from exploits that give attackers access to code repositories and exfiltrate secrets and data, modify or inject code into software, and manipulate the CI/CD pipeline.

**Hosting Malware on Trusted GitHub Repos**

Researchers at Cofense this week reported a phishing campaign where a threat actor is attempting to direct targeted victims in the insurance and finance sectors to malware hosted on trusted GitHub repositories. The campaign involves the attacker sending victims tax-themed phishing emails containing a link to a password-protected archive containing Remcos, a remote access Trojan that cybercriminals and state-backed groups alike have used in various cyber-espionage and data theft attacks over the years.

What makes the campaign noteworthy, according to Cofense, is how the threat actor has managed to sneak the archive files containing the Remcos RAT into legitimate GitHub repositories belonging to trusted entities. Examples of such entities include His Majesty’s Revenue & Customs (HMRC), the UK's national tax authority; New Zealand's counterpart, InlandRevenue; and UsTaxes, an open source tax-filing platform.

In each instance, the attacker used GitHub comments to upload a malicious file containing Remcos RAT to the repositories of the respective entities.

Many GitHub repositories allow developers to comment on ongoing and collaborative software projects. The comments can cover a wide range of topics, including proposed code changes, documentation and bug-related issues, task creation clarification requests, task management and progress updates, and merge conflict resolution.

"GitHub comments are useful to a threat actor because malware can be attached to a comment in a GitHub repository without having to upload it to the source code files of that repository," Cofense security researcher Jacob Malimban wrote in a blog post. "This means that any organization's legitimate GitHub repository that allows comments can contain unapproved files outside of the vetted code." Unsanctioned files that someone might submit via GitHub comments end up in a subdirectory that is separate from the one containing the repository's vetted files, Malimban said. What is especially troubling is the fact that the link to the malicious file will continue to work even if the comment itself gets deleted.

**Multiple Incidents**

Other threat actors have noticed the opportunity as well. A recent case in point is the purveyor of the Redline Stealer, who earlier this year was spotted using no less than Microsoft's own GitHub repository to host the information stealing malware. In that campaign — as with the new Remcos RAT attacks that Cofense spotted — the threat actor uploaded the malware as a comment to Microsoft's GitHub vcpkg repository.

Emails with links to domains such as GitHub are effective at skirting secure email gateways because of their trusted reputation. Attackers can, in fact, directly link to their malware in such domains without the need to redirect users to other sites, or without requiring them to use other security bypass techniques like scanning QR codes, Cofense said.

The threat actor behind the new Remcos RAT could easily have targeted victims in other sectors as well. But they likely deliberately kept their focus narrow to test how effective the strategy of hosting malware on the GitHub repositories is before attacking others, Malimban surmised.

**Growing Threat Actor Interest**

Meanwhile, the new exploit for GitLab targets a critical authentication bypass vulnerability (CVE-2024-45409) affecting the Ruby-SAML and OmniAuth-SAML libraries that GitLab uses to enable SAML-based single sign-on. The exploit script gives attackers a way to abuse the vulnerability to access GitLab in the context of any user. The vulnerability affects all versions of GitLab Community Edition (CE) and Enterprise Edition (EE) below 16.11.10. The flaw is also present in multiple 17.x.x versions of GitLab.

The exploit is another sign of the growing researcher and threat actor interest in repositories like GitHub and GitLab and their users. Over the past year there have been multiple instances of attacks targeting repos on GitHub, like one involving cyber-extortion that Chilean cybersecurity firm CronUp reported in June and another involving the use of ghost accounts on GitHub to distribute malware. GitLab users have had their share of security scares to deal with as well, like CVE-2024-45409 and two other recent vulnerabilities (CVE-2024-6385 and CVE-2024-5655) that posed a major threat to the integrity of CI/CD pipelines.

**About the Author**

Jai Vijayan is a seasoned technology reporter with over 20 years of experience in IT trade journalism. He was most recently a Senior Editor at Computerworld, where he covered information security and data privacy issues for the publication. Over the course of his 20-year career at Computerworld, Jai also covered a variety of other technology topics, including big data, Hadoop, Internet of Things, e-voting, and data analytics. Prior to Computerworld, Jai covered technology issues for The Economic Times in Bangalore, India. Jai has a Master's degree in Statistics and lives in Naperville, Ill.

Hackers Hide Remcos RAT in GitHub Repository Comments

PRESS RELEASE

TEL AVIV, Israel, Oct. 09, 2024 (GLOBE NEWSWIRE) -- Pillar Security, a pioneering company in GenAI security solutions, today released the industry’s first "State of Attacks on GenAI" research based on real-world analysis of more than 2,000 AI applications. In sharp contrast to earlier opinion and theoretical risk surveys, this data-driven research is based on Pillar's telemetry data derived from data interactions that occurred in production AI-powered applications over the past three months.

Key findings from the report include:

*   High Success Rate of Data Theft: 90% of successful attacks resulted in the leakage of sensitive data
    
*   Alarming Bypass Rate: 20 percent of jailbreak attack attempts successfully bypassed GenAI application guardrails
    
*   Rapid Attack Execution: Adversaries require an average of just 42 seconds to execute an attack
    
*   Minimal Interaction Needed: Attackers needed only five interactions on average with GenAI applications to complete a successful attack
    
*   Widespread Vulnerabilities: Attacks exploited vulnerabilities at every stage of interaction with GenAI systems, underscoring the critical need for comprehensive security measures
    
*   Increase in Frequency and Complexity: the analyzed attacks reveal a clear increase in both the frequency and complexity of prompt injection attacks, with users employing more sophisticated techniques and making persistent attempts to bypass safeguards as time progresses
    

"The widespread adoption of GenAI in organizations has opened a new frontier in cybersecurity," said Dor Sarig, CEO and co-founder of Pillar Security. "Our report goes beyond theoretical risks and, for the first time, shines a light on the actual attacks occurring in the wild, offering organizations actionable insights to fortify their GenAI security posture."

Highlights among the many other insights in the fact-filled report are:

*   Top Jailbreak Techniques, which include Ignore Previous Instructions--attackers direct AI systems to disregard their initial programming--and Base64 Encoding--malicious prompts encoded to evade security filters
    
*   Primary Attacker Motivations are stealing sensitive data, proprietary business information and PII and circumventing content filters to produce disinformation, hate speech, phishing messages and malicious code, among others
    
*   Curated and detailed list analyzes top attacks observed in real-world production AI apps
    
*   Looking Ahead to 2025, Pillar projects the evolution from chatbots to copilots and autonomous agents, alongside the proliferation of small, locally deployed AI models. This new era of AI adoption democratizes access but further expands attack surfaces, introducing additional security challenges for organizations.
    

"As we move towards AI agents capable of performing complex tasks and making decisions, the security landscape becomes increasingly complex," explained Sarig. "Organizations must prepare for a surge in AI-targeted attacks by implementing tailored red-teaming exercises and adopting a 'secure by design' approach in their GenAI development process."

The report emphasizes the inadequacy of traditional static security measures in the face of evolving AI threats. "Static controls are no longer sufficient in this dynamic AI-enabled world," added Jason Harrison, Pillar Security CRO. "Organizations must invest in AI security solutions capable of anticipating and responding to emerging threats in real-time, while supporting their governance and cyber policies.”

Pillar’s complete research report on the State of Attacks on GenAI is available on their website. 

For more information on AI Security, please visit https://www.pillar.security/resources/buyer-guide.

To schedule a demo, please visit https://www.pillar.security/get-a-demo.

About Pillar Security  
Pillar Security provides a unified platform to secure the entire AI lifecycle from development through production to usage. The platform integrates seamlessly with existing controls and workflows, and provides proprietary risk detection models, comprehensive visibility, adaptive runtime protection, robust governance features and cutting-edge adversarial resistance. Pillar's detection and evaluation engines are continuously optimized by training on large datasets of real-world AI app interactions, providing the highest accuracy and precision of AI-related risks.

You May Also Like

90% of Successful Attacks Seen in the Wild Resulted in Leaked Sensitive Data

The security bugs were found susceptible to exploitation in connection to the previously disclosed, critical CVE-2024-8963 vulnerability in the security vendor's Cloud Services Appliance (CSA).

Source: Skorzewiak via Alamy Stock Photo

In the latest wrinkle of what seems to be an ongoing saga of vulnerability concerns, Ivanti is notifying customers of three additional vulnerabilities found in its Cloud Services Appliance (CSA) that are being exploited in the wild.

There is limited exploitation of the vulnerabilities (CVE-2024-9379, CVE-2024-9380, and CVE-2024-9381) according to the vendor, which are being chained individually with a previously disclosed zero-day vulnerability (CVE-2024-8963) found in Ivanti's CSA.

CVE-2024-9379 has a CVSS rating of 6.5 and allows a remote authenticated attacker with privileges to run SQL statements. CVE-2024-9380, with a CVSS score of 7.2, is an operating system command injection vulnerability in Ivanti CSA that can allow a remote authenticated attacker to obtain remote code execution with admin privileges. And lastly, CVE-2024-9381, carrying a CVSS score of 7.2, is a path traversal in Ivanti CSA before version 5.0 and allows a remote authenticated attacker to bypass restrictions with admin privileges.

The bugs were found on systems running CSA 4.6 patch 518 and prior, and there is no evidence of exploitation on any environments running CSA 5.0.

"Ivanti recommends reviewing the CSA for modified or newly added administrative users," said Ivanti in its user recommendations for checking compromised devices. "We also recommend reviewing EDR alerts, if you have installed EDR or other security tools on your CSA. As this is an edge device, Ivanti strongly recommends using a layered approach to security and installing an EDR tool on the CSA."

Should a user suspect that they have been compromised, its recommended they rebuild their CSA with version 5.0.

**About the Author**

3 More Ivanti Cloud Vulns Exploited in the Wild

Cyber pros are scrambling to stay up-to-date as the businesses they work for quickly roll out AI tools and keep expanding their cloud initiatives.

A major skills gap exists for security teams when it comes to artificial intelligence (AI) and cloud implementations, which happen to be two of the fastest-growing areas when it comes to enterprises' ongoing digital transformations.

According to O'Reilly's "2024 State of Security" report, nearly 39% of respondents on security teams reported that cloud computing is a space where more skills are needed but are difficult to find.

"Cloud security requires taking concepts like access control and least privilege, and applying them to servers and services that you'll never see and may only control through an API provided by your cloud vendor," wrote Mike Loukides, author of the report. "An error in any service can compromise all your infrastructure — that's why infrastructure as code is so important. In many respects, the game doesn't change, but the stakes become much higher."

Potential talent should prioritize skills like being able think in terms of securing hundreds or thousands of virtual instances, as well as being able to use or develop tools that can reach across multiple servers, services, and cloud providers.

AI, on the other hand, represents a whole new category of threats. Roughly 34% of respondents in the survey pointed to a lack of talent when it comes to AI skills, especially regarding attack avenues such as prompt injection. However, this space is so new that researchers are only beginning to understand the threats and vulnerabilities that AI poses — and even less is known about any possible solutions. 

"The security community is only beginning to catch up with the use and misuse of AI. In the coming years, we expect a surge in AI-specific research, training, and certification," Loukides wrote.

According to Mary Treseler, chief content officer of O'Reilly Media, those hiring in the cyber industry favor those who have a traditional computer science education in addition to experience in IT work such as system admin, help desk, and software development.

"It's possible to get a cybersecurity job without a degree, provided you have relevant work experience," Treseler says. "Certifications and experiences such as bug bounty hunting or capture-the-flag participation can supplement."

Also, some organizations, such as MITRE, are already providing tools to share data on real world AI incidents, such as the AI Incident Sharing initiative under MITRE ATLAS, to combat rising threats. The tool is anonymous in order to serve as a safe space to openly share the details of cyberattacks occurring across industries and government. It's modeled after traditional intelligence-sharing, so organizations can submit incident data through the site, after which they will be considered for membership. And in Europe, an effort is underway to promote AI literacy and awareness for staff dealing with the deployment of AI systems, via the EU Artificial Intelligence Pact.

**Security Up-Skilling: A Marathon, Not a Sprint**

Upskilling to eradicate gaps in cybersecurity talent is the easiest way forward for now, experts say.

"Our global survey underscores a security landscape in flux," Laura Baldwin, president of O'Reilly, said in a press release. "As cyber threats become increasingly sophisticated, it's clear that continuous, high-quality training is no longer optional; it's essential for safeguarding our digital future. Organizations must prioritize ongoing upskilling to stay ahead of evolving risks and build robust defenses."

Certifications, books, videos, and conferences can all be valuable resources to stay up-to-date with the latest need-to-know skills.

Some of the most popular certifications, according to Treseler are CISSP, which she notes is the most versatile and requires five years of work experience, CompTIA Security+, CEH, and CISM. These are all viable options that she says potential talent can trust will give them valuable expertise in the field, but also resources that higher-ups may be looking for when scouring for new candidates.

"Security is a challenge that will never go away," Loukides wrote. "Chances are, we'll invent new risks as quickly as we retire old ones. But we can do better at meeting the challenge."

Cloud, AI Talent Gaps Plague Cybersecurity Teams

Multimodal AI systems can help enterprise defenders weed out fraudulent emails, even if the system has not seen that type of message before.

Source: Peshkova via Shutterstock

Artificial intelligence (AI) models that work across different types of media and domains — so-called "multimodal AI" — can be used by attackers to create convincing scams. At the same time, defenders are finding multimodal AI equally useful at spotting fraudulent emails and not-safe-for-work (NSFW) materials.

A large language model (LLM) can accurately classify previously unseen samples of emails impersonating different brands with better than 97% accuracy, as measured by a metric known as the F1 score, according to researchers at cybersecurity firm Sophos, who presented their findings at the Virus Bulletin Conference on Oct. 4. While existing email-security and content-filtering systems can spot messages using brands that have been encountered before, multimodal AI systems can identify the latest attacks, even if the system is not trained on samples of similar emails.

While the approach will likely not be a feature in email-security products, it could be used as a late-stage filter by security analysts, says Ben Gelman, a senior data scientist at Sophos, which has joined other cybersecurity firms, such as Google, Microsoft, and Simbian, in exploring new ways of using LLMs and other generative AI models to augment and assist security analysts and to help speed up incident response.

"AI and cybersecurity are merging, and this whole AI-generated attack/AI generated defense \[approach\] is going to become natural in the cybersecurity space," he says. "It's a force multiplier for our analysts. We have a number of projects where we support our SOC analysts with AI-based tools, and it's all about making them more efficient and giving them all this knowledge and confidence at their fingertips."

**Understanding Attackers' Tactics**

Attackers have also started using LLMs to improve their email lures and attack code. Microsoft, Google, and OpenAI have all warned that nation-state groups appear to be using these public LLMs for various tasks, such as creating spear-phishing lures and code snippets used to scrape websites.

As part of their research, the Sophos team created a platform for automating the launch of an e-commerce scam campaign, or "scampaigns," to understand what sort of attacks could be possible with multimodal generative AI. The platform consisted of five different AI agents: a data agent for generating information about the products and services, an image agent for creating images, an audio agent for any sound needs, a UI agent for creating the custom code, and an advertising agent to create marketing materials. The customization potential for automated ChatGPT spear-phishing and scam campaigns could result in large-scale microtargeting campaigns, the Sophos researchers stated in its Oct. 2 analysis.

"\[W\]e can see that these techniques are particularly chilling because users may interpret the most effective microtargeting as serendipitous coincidences," the researchers stated. "Spear phishing previously required dedicated manual effort, but with this new automation, it is possible to achieve personalization at a scale that hasn't been seen before."

That said, Sophos has not yet encountered this level of AI usage in the wild.

Defenders should expect AI-assisted cyberattackers to have better quality social-engineering techniques and faster cycles of innovation, says Anand Raghavan, vice president of AI engineering at Cisco Security.

"It is not just the quality of the emails, but the ability to automate this has gone up an order of magnitude since the arrival of GPT and other AI tools," he says. "The attackers have gotten not just incrementally better, but exponentially better."

**Beyond Keyword Matching**

Using LLMs to process emails and turn them into text descriptions leads to better accuracy and can help analysts process emails that might have otherwise escaped notice, stated Younghoo Lee, a principal data scientist with Sophos's AI group, in research presented at the Virus Bulletin conference.

"\[O\]ur multimodal AI approach, which leverages both text and image inputs, offers a more robust solution for detecting phishing attempts, particularly when facing unseen threats," he stated in the paper accompanying his presentation. "The use of both text and image features proved to be more effective" when dealing with multiple brands.

The capability to process the context of the text in the email augments the multimodal capability to "understand" words and context from images, allowing a fuller understanding of an email, says Cisco's Raghavan. LLMs' ability to focus not just on pinpointing suspicious language but also on dangerous contexts — such as emails that urge a user to take a business-critical action — make them very useful in assisting analysis, he says.

Any attempt to compromise workflows that have to do with money, credentials, sensitive data, or confidential processes should be flagged.

"Language as a classifier also very strongly enables us to reduce false positives by identifying what we call critical business workflows," Raghavan says. "If an attacker is interested in compromising your organization, there are four kinds of critical business workflows, \[and\] language is the predominant indicator for us to determine \[whether\] an email is concerning or not."

So why not use LLMs everywhere? Cost, says Sophos's Gelman.

"Depending on LLMs to do anything at massive scale is usually way too expensive relative to the gains that you're getting," he says. "One of the challenges of multimodal AI is that every time you add a mode like images, you need way more data, you need way more training time, and — when the text and the image models conflict — you need a better model and potentially better training" to decide between the two.

**About the Author**

Veteran technology journalist of more than 20 years. Former research engineer. Written for more than two dozen publications, including CNET News.com, Dark Reading, MIT's Technology Review, Popular Science, and Wired News. Five awards for journalism, including Best Deadline Journalism (Online) in 2003 for coverage of the Blaster worm. Crunches numbers on various trends using Python and R. Recent reports include analyses of the shortage in cybersecurity workers and annual vulnerability trends.

Tag

#vulnerability