PRESS RELEASE

NEW YORK, Oct. 9, 2024 /PRNewswire/ -- OpenGradient, a leading decentralized AI infrastructure company, has raised $8.5M in seed funding. With a mission to accelerate open-source AI, OpenGradient democratizes model ownership, streamlines AI deployment, and enables universal access to AI.

OpenGradient's end-to-end decentralized platform offers scalable AI compute on-chain, ensuring verifiability and attribution while enhancing security for AI deployment and hosting. Its multi-layer technology stack includes an EVM-compatible blockchain with its innovative heterogeneous AI compute architecture (HACA) and a proprietary library of feature-rich tools to support and scale secure AI workflows on-chain.

OpenGradient empowers developers to build and deploy optimized decentralized applications (dApps) with ease. OpenGradient's web portal and SDK allow Web2 developers to leverage its AI stack without the complexities of blockchain while enjoying the security and composability benefits of a decentralized infrastructure.

CEO and Co-Founder, Matthew Wang, said,

"Developers today face significant hurdles when developing secure software for custom AI models, often grappling with developing a variety of complex cryptographic or hardware solutions into their stack. OpenGradient revolutionizes this approach, offering near-instantaneous model deployment and secure inference of AI models in mere seconds."

The $8.5 million seed round included participation from a16z Crypto Startup Accelerator (a16z CSX), SV Angel, Coinbase Ventures, SALT Fund, Symbolic Capital, Foresight Ventures, and angel investors, including Balaji Srinivasan (ex-Coinbase CTO), Illia Polosukhin(NEAR Founder), Sandeep Nailwal (Polygon Founder), and more. OpenGradient is also currently participating in the Fall 2024 a16z CSX program in New York City.

Principal at SV Angel, Mike Liu, said,

"OpenGradient is ushering in a new era for decentralized AI by developing infrastructure to support the next generation of intelligent applications. They've built a truly disruptive tech stack at the intersection of AI and blockchain, and we're excited to support such an innovative and talented team."

The funding will be used to continue building out the decentralized infrastructure, deploying solutions and tools for AI and Web3 developers, and advancing applied ML research to grow the open-source AI ecosystem on blockchain. OpenGradient's testnet is expected to be available to developers in Q4 2024.

About OpenGradient

OpenGradient is a leading decentralized platform unlocking scalable AI compute on-chain, empowering developers to build secure, intelligent, and optimized decentralized applications. We provide on-chain AI model hosting, permissionless composability, secure inference execution, and seamless access to accelerate open-source AI and foster next-gen applications. Headquartered in New York City, the team comprises global talent with deep expertise in AI/ML, blockchain, crypto, and Web3 ecosystem. Learn more at https://opengradient.ai and follow us on X and LinkedIn.

OpenGradient Raises $8.5M to Decentralize AI Infrastructure and Accelerate Secure, Open-Source AI

The tack highlights bad actors' interest in trusted development and collaboration platforms — and their users.

Source: Tada Images via Shutterstock

Trusted and widely used software development and collaboration platforms like GitHub and GitLab have become both targets of and vehicles for a growing range of malicious activity.

The latest manifestations of that trend include a malware distribution campaign involving legitimate GitHub repositories and the availability this week of an exploit for a vulnerability that allows an attacker to gain access as any user of GitLab.

The first is an example of how attackers are exploiting the trusted reputation of platforms like GitHub to try and sneak malware past endpoint detection mechanisms. The GitLab vulnerability, meanwhile, highlights the growing exposure to organizations from exploits that give attackers access to code repositories and exfiltrate secrets and data, modify or inject code into software, and manipulate the CI/CD pipeline.

**Hosting Malware on Trusted GitHub Repos**

Researchers at Cofense this week reported a phishing campaign where a threat actor is attempting to direct targeted victims in the insurance and finance sectors to malware hosted on trusted GitHub repositories. The campaign involves the attacker sending victims tax-themed phishing emails containing a link to a password-protected archive containing Remcos, a remote access Trojan that cybercriminals and state-backed groups alike have used in various cyber-espionage and data theft attacks over the years.

What makes the campaign noteworthy, according to Cofense, is how the threat actor has managed to sneak the archive files containing the Remcos RAT into legitimate GitHub repositories belonging to trusted entities. Examples of such entities include His Majesty’s Revenue & Customs (HMRC), the UK's national tax authority; New Zealand's counterpart, InlandRevenue; and UsTaxes, an open source tax-filing platform.

In each instance, the attacker used GitHub comments to upload a malicious file containing Remcos RAT to the repositories of the respective entities.

Many GitHub repositories allow developers to comment on ongoing and collaborative software projects. The comments can cover a wide range of topics, including proposed code changes, documentation and bug-related issues, task creation clarification requests, task management and progress updates, and merge conflict resolution.

"GitHub comments are useful to a threat actor because malware can be attached to a comment in a GitHub repository without having to upload it to the source code files of that repository," Cofense security researcher Jacob Malimban wrote in a blog post. "This means that any organization's legitimate GitHub repository that allows comments can contain unapproved files outside of the vetted code." Unsanctioned files that someone might submit via GitHub comments end up in a subdirectory that is separate from the one containing the repository's vetted files, Malimban said. What is especially troubling is the fact that the link to the malicious file will continue to work even if the comment itself gets deleted.

**Multiple Incidents**

Other threat actors have noticed the opportunity as well. A recent case in point is the purveyor of the Redline Stealer, who earlier this year was spotted using no less than Microsoft's own GitHub repository to host the information stealing malware. In that campaign — as with the new Remcos RAT attacks that Cofense spotted — the threat actor uploaded the malware as a comment to Microsoft's GitHub vcpkg repository.

Emails with links to domains such as GitHub are effective at skirting secure email gateways because of their trusted reputation. Attackers can, in fact, directly link to their malware in such domains without the need to redirect users to other sites, or without requiring them to use other security bypass techniques like scanning QR codes, Cofense said.

The threat actor behind the new Remcos RAT could easily have targeted victims in other sectors as well. But they likely deliberately kept their focus narrow to test how effective the strategy of hosting malware on the GitHub repositories is before attacking others, Malimban surmised.

**Growing Threat Actor Interest**

Meanwhile, the new exploit for GitLab targets a critical authentication bypass vulnerability (CVE-2024-45409) affecting the Ruby-SAML and OmniAuth-SAML libraries that GitLab uses to enable SAML-based single sign-on. The exploit script gives attackers a way to abuse the vulnerability to access GitLab in the context of any user. The vulnerability affects all versions of GitLab Community Edition (CE) and Enterprise Edition (EE) below 16.11.10. The flaw is also present in multiple 17.x.x versions of GitLab.

The exploit is another sign of the growing researcher and threat actor interest in repositories like GitHub and GitLab and their users. Over the past year there have been multiple instances of attacks targeting repos on GitHub, like one involving cyber-extortion that Chilean cybersecurity firm CronUp reported in June and another involving the use of ghost accounts on GitHub to distribute malware. GitLab users have had their share of security scares to deal with as well, like CVE-2024-45409 and two other recent vulnerabilities (CVE-2024-6385 and CVE-2024-5655) that posed a major threat to the integrity of CI/CD pipelines.

**About the Author**

Jai Vijayan is a seasoned technology reporter with over 20 years of experience in IT trade journalism. He was most recently a Senior Editor at Computerworld, where he covered information security and data privacy issues for the publication. Over the course of his 20-year career at Computerworld, Jai also covered a variety of other technology topics, including big data, Hadoop, Internet of Things, e-voting, and data analytics. Prior to Computerworld, Jai covered technology issues for The Economic Times in Bangalore, India. Jai has a Master's degree in Statistics and lives in Naperville, Ill.

Hackers Hide Remcos RAT in GitHub Repository Comments

PRESS RELEASE

TEL AVIV, Israel, Oct. 09, 2024 (GLOBE NEWSWIRE) -- Pillar Security, a pioneering company in GenAI security solutions, today released the industry’s first "State of Attacks on GenAI" research based on real-world analysis of more than 2,000 AI applications. In sharp contrast to earlier opinion and theoretical risk surveys, this data-driven research is based on Pillar's telemetry data derived from data interactions that occurred in production AI-powered applications over the past three months.

Key findings from the report include:

*   High Success Rate of Data Theft: 90% of successful attacks resulted in the leakage of sensitive data
    
*   Alarming Bypass Rate: 20 percent of jailbreak attack attempts successfully bypassed GenAI application guardrails
    
*   Rapid Attack Execution: Adversaries require an average of just 42 seconds to execute an attack
    
*   Minimal Interaction Needed: Attackers needed only five interactions on average with GenAI applications to complete a successful attack
    
*   Widespread Vulnerabilities: Attacks exploited vulnerabilities at every stage of interaction with GenAI systems, underscoring the critical need for comprehensive security measures
    
*   Increase in Frequency and Complexity: the analyzed attacks reveal a clear increase in both the frequency and complexity of prompt injection attacks, with users employing more sophisticated techniques and making persistent attempts to bypass safeguards as time progresses
    

"The widespread adoption of GenAI in organizations has opened a new frontier in cybersecurity," said Dor Sarig, CEO and co-founder of Pillar Security. "Our report goes beyond theoretical risks and, for the first time, shines a light on the actual attacks occurring in the wild, offering organizations actionable insights to fortify their GenAI security posture."

Highlights among the many other insights in the fact-filled report are:

*   Top Jailbreak Techniques, which include Ignore Previous Instructions--attackers direct AI systems to disregard their initial programming--and Base64 Encoding--malicious prompts encoded to evade security filters
    
*   Primary Attacker Motivations are stealing sensitive data, proprietary business information and PII and circumventing content filters to produce disinformation, hate speech, phishing messages and malicious code, among others
    
*   Curated and detailed list analyzes top attacks observed in real-world production AI apps
    
*   Looking Ahead to 2025, Pillar projects the evolution from chatbots to copilots and autonomous agents, alongside the proliferation of small, locally deployed AI models. This new era of AI adoption democratizes access but further expands attack surfaces, introducing additional security challenges for organizations.
    

"As we move towards AI agents capable of performing complex tasks and making decisions, the security landscape becomes increasingly complex," explained Sarig. "Organizations must prepare for a surge in AI-targeted attacks by implementing tailored red-teaming exercises and adopting a 'secure by design' approach in their GenAI development process."

The report emphasizes the inadequacy of traditional static security measures in the face of evolving AI threats. "Static controls are no longer sufficient in this dynamic AI-enabled world," added Jason Harrison, Pillar Security CRO. "Organizations must invest in AI security solutions capable of anticipating and responding to emerging threats in real-time, while supporting their governance and cyber policies.”

Pillar’s complete research report on the State of Attacks on GenAI is available on their website. 

For more information on AI Security, please visit https://www.pillar.security/resources/buyer-guide.

To schedule a demo, please visit https://www.pillar.security/get-a-demo.

About Pillar Security  
Pillar Security provides a unified platform to secure the entire AI lifecycle from development through production to usage. The platform integrates seamlessly with existing controls and workflows, and provides proprietary risk detection models, comprehensive visibility, adaptive runtime protection, robust governance features and cutting-edge adversarial resistance. Pillar's detection and evaluation engines are continuously optimized by training on large datasets of real-world AI app interactions, providing the highest accuracy and precision of AI-related risks.

You May Also Like

90% of Successful Attacks Seen in the Wild Resulted in Leaked Sensitive Data

The bill is broken up into several pieces, including ransomware reporting and securing smart devices, among other objectives.

Source: David Coleman | Have Camera Will Travel via Alamy Stock Photo

Australia introduced a new cybersecurity law into parliament, known as the Cyber Security Bill 2024, its first effort to codify security standards for smart devices, ransomware reporting, and coordination of significant cybersecurity incidents. 

The bill calls for creation of a Cyber Incident Review Board to implement post-incident reviews of significant cybersecurity incidents, as well as restrictions on how incident information can be provided to the National Cyber Security Coordinator and shared with other countries.

Under Australia's Security of Critical Infrastructure (SOCI) Act of 2018, the bill will also implement reforms by reinforcing government assistance measures to manage the impact of cyber incidents on critical infrastructure and simplify the information shared between industry and government. 

"We need a framework that enhances protections for victims of cyber incidents and encourages them to engage with government, and we need a framework that enables us to learn lessons from significant cybersecurity incidents so that we can be better prepared going forward," stated Tony Burke, Minister of Home Affairs of Australia, when introducing the bill.

The proposed regulatory framework is part of the country's goal to become a leader in cybersecurity by 2030, an objective that was introduced in 2023 through the Australian Cybersecurity Strategy, an estimated seven-year effort.

**About the Author**

Australia Intros Its First National Cyber Legislation

A stealthy new underground offering uses sophisticated adversary-in-the-middle (AitM) techniques to convincingly serve up "Microsoft" login pages of various kinds, with dynamic enterprise branding.

Source: Matthijs Kuijpers via Alamy Stock Photo

A phishing-as-a-service (PhaaS) kit dubbed Mamba 2FA is targeting Microsoft 365 users using a variety of convincing adversary-in-the-middle (AitM) disguises.

According to the Sekoia Threat Detection & Research (TDR) team, the kit, which goes for $250 per month on underground cybercrime forums, can present a number of faux login pages to unsuspecting users. It can imitate OneDrive, a SharePoint Online secure link, or a generic Microsoft sign-in page; or it can show the user a purported voicemail retrieval link that redirects to a sign-in page after a click.

In all cases, it dynamically reflects enterprise targets' branding, including logos and background image.

According to Sekoia, Mamba 2FA slithers past two-factor authentication (2FA) methods that use one-time codes and app notifications; supports Entra ID, AD FS, third-party SSO providers, and consumer Microsoft accounts; and harvests credentials and cookies that are instantly sent to the attacker via a Telegram bot.

"Mamba 2FA has been advertised on Telegram since at least March," according to a Sekoia analysis this week. "However, according to data from public URL and file analysis sandboxes, the kit has been used in phishing campaigns since November 2023. The operator of the service had a long-standing presence on ICQ until this messaging platform shut down in June 2024, and this may be where Mamba 2FA was primarily sold before shifting to Telegram."

**About the Author**

Tara Seals has 20+ years of experience as a journalist, analyst and editor in the cybersecurity, communications and technology space. Prior to Dark Reading, Tara was Editor in Chief at Threatpost, and prior to that, the North American news lead for Infosecurity Magazine. She also spent 13 years working for Informa (formerly Virgo Publishing), as executive editor and editor-in-chief at publications focused on both the service provider and the enterprise arenas. A Texas native, she holds a B.A. from Columbia University, lives in Western Massachusetts with her family and is on a never-ending quest for good Mexican food in the Northeast.

Mamba 2FA Cybercrime Kit Targets Microsoft 365 Users

The security bugs were found susceptible to exploitation in connection to the previously disclosed, critical CVE-2024-8963 vulnerability in the security vendor's Cloud Services Appliance (CSA).

Source: Skorzewiak via Alamy Stock Photo

In the latest wrinkle of what seems to be an ongoing saga of vulnerability concerns, Ivanti is notifying customers of three additional vulnerabilities found in its Cloud Services Appliance (CSA) that are being exploited in the wild.

There is limited exploitation of the vulnerabilities (CVE-2024-9379, CVE-2024-9380, and CVE-2024-9381) according to the vendor, which are being chained individually with a previously disclosed zero-day vulnerability (CVE-2024-8963) found in Ivanti's CSA.

CVE-2024-9379 has a CVSS rating of 6.5 and allows a remote authenticated attacker with privileges to run SQL statements. CVE-2024-9380, with a CVSS score of 7.2, is an operating system command injection vulnerability in Ivanti CSA that can allow a remote authenticated attacker to obtain remote code execution with admin privileges. And lastly, CVE-2024-9381, carrying a CVSS score of 7.2, is a path traversal in Ivanti CSA before version 5.0 and allows a remote authenticated attacker to bypass restrictions with admin privileges.

The bugs were found on systems running CSA 4.6 patch 518 and prior, and there is no evidence of exploitation on any environments running CSA 5.0.

"Ivanti recommends reviewing the CSA for modified or newly added administrative users," said Ivanti in its user recommendations for checking compromised devices. "We also recommend reviewing EDR alerts, if you have installed EDR or other security tools on your CSA. As this is an edge device, Ivanti strongly recommends using a layered approach to security and installing an EDR tool on the CSA."

Should a user suspect that they have been compromised, its recommended they rebuild their CSA with version 5.0.

**About the Author**

3 More Ivanti Cloud Vulns Exploited in the Wild

The EU AI Act provides a governance, risk, and compliance (GRC) framework that helps organizations take a risk-based approach to using AI.

Source: Vitor Miranda via Alamy Stock Photo

COMMENTARY

As artificial intelligence (AI) becomes increasingly prevalent in business operations, organizations must adapt their governance, risk, and compliance (GRC) strategies to address the privacy and security risks this technology poses. The European Union's AI Act provides a valuable framework for assessing and managing AI risk, offering insights that can benefit companies worldwide.

The EU AI Act applies to providers and users of AI systems in the EU, as well as those putting AI systems on the EU market or using them within the EU. Its primary goal is to ensure that AI systems are safe and respect fundamental rights and values, including privacy, nondiscrimination, and human dignity.

The EU AI Act categorizes AI systems into four risk levels. On one end of the spectrum, AI systems that pose clear threats to safety, livelihoods, and rights are deemed an Unacceptable Risk. On the other end, AI systems classified as Minimal Risk are largely unregulated, though subject to general safety and privacy rules.

The classifications to study for GRC management are High Risk and Limited Risk. High Risk denotes AI systems where there is a significant risk of harm to individuals' health, safety, or fundamental rights. Limited Risk AI systems pose minimal threat to safety, privacy, or rights but remain subject to transparency obligations.

The EU AI Act allows organizations to take a risk-based approach when assessing AI. The framework helps establish a logical approach for AI risk assessments, particularly for High and Limited Risk activities.

**Requirements for High-Risk AI Activities**

High-Risk AI activities can include credit scoring, AI-driven recruitment, healthcare diagnostics, biometric identification, and safety-critical systems in transportation. For these and similar activities, the EU AI Act mandates the following stringent requirements:

1.  Risk management system: Implement a comprehensive risk management system throughout the AI system's life cycle.
    
2.  Data governance: Ensure proper data governance with high-quality datasets to prevent bias.
    
3.  Technical documentation: Maintain detailed documentation of the AI system's operations.
    
4.  Transparency: Provide clear communication about the AI system's capabilities and limitations.
    
5.  Human oversight: Enable meaningful human oversight for monitoring and intervention.
    
6.  Accuracy and robustness: Ensure the AI system maintains appropriate accuracy and robustness.
    
7.  Cybersecurity: Implement state-of-the-art security mechanisms to protect the AI system and its data.
    

**Requirements for Limited and Minimal Risk AI Activities**

While Limited and Minimal Risk activities don't require the same level of scrutiny as High-Risk systems, they still warrant careful consideration.

1.  Data assessment: Identify the types of data involved, its sensitivity, and how it will be used, stored, and secured.
    
2.  Data minimization: Ensure that only essential data is collected and processed.
    
3.  System integration: Evaluate how the AI system will interact with other internal or external systems.
    
4.  Privacy and security: Apply traditional data privacy and security measures.
    
5.  Transparency: Implement clear notices that inform users of AI interaction or AI-generated content.
    

**Requirements for All AI Systems: Assessing Training Data**

The assessment of AI training data is crucial for risk management. Key considerations for the EU AI Act include ensuring that you have the necessary rights to use the data for AI training purposes, as well as implementing strict access controls and data segregation measures for sensitive data.

In addition, AI systems must protect authors' rights and prevent unauthorized reproduction of protected IP. They also have to maintain high-quality, representative datasets and mitigate potential biases. Finally, they must maintain clear records of data sources and transformations for traceability and compliance purposes.

**How to Integrate AI Act Guidelines Into Existing GRC Strategies**

While AI presents new challenges, many aspects of the AI risk assessment process build on existing GRC practices. Organizations can start by applying traditional due-diligence processes for systems that handle confidential, sensitive, or personal data. Then, focus on these AI-specific considerations:

1.  AI capabilities assessment: Evaluate the AI system's actual capabilities, limitations, and potential impacts.
    
2.  Training and management: Assess how the AI system's capabilities are trained, updated, and managed over time.
    
3.  Explainability and interpretability: Ensure that the AI's decision-making process can be explained and interpreted, especially for High-Risk systems.
    
4.  Ongoing monitoring: Implement continuous monitoring to detect issues, such as model drift or unexpected behaviors.
    
5.  Incident response: Develop AI-specific incident response plans to address potential failures or unintended consequences.
    

By adapting existing GRC strategies and incorporating insights from frameworks like the EU AI Act, organizations can navigate the complexities of AI risk management and compliance effectively. This approach not only helps mitigate potential risks but also positions companies to leverage AI technologies responsibly and ethically, thus building trust with customers, employees, and regulators alike.

As AI continues to evolve, so, too, will the regulatory landscape. The EU AI Act serves as a pioneering framework, but organizations should stay informed about emerging regulations and best practices in AI governance. By proactively addressing AI risks and embracing responsible AI principles, companies can harness the power of AI while maintaining ethical standards and regulatory compliance.

**About the Author**

General Counsel, Secureframe

Kyle McLaughlin serves as General Counsel at Secureframe, an all-in-one platform for continuous security compliance. With a proven track record counseling prominent tech companies including Cisco and Duo Security, he specializes in IP issues, security, compliance, and privacy (holding CIPP/E certification). McLaughlin earned his JD from Wayne State University Law School and completed his bachelor's degree in Political Science and English at the University of Michigan.

Risk Strategies Drawn From the EU AI Act

Cyber pros are scrambling to stay up-to-date as the businesses they work for quickly roll out AI tools and keep expanding their cloud initiatives.

A major skills gap exists for security teams when it comes to artificial intelligence (AI) and cloud implementations, which happen to be two of the fastest-growing areas when it comes to enterprises' ongoing digital transformations.

According to O'Reilly's "2024 State of Security" report, nearly 39% of respondents on security teams reported that cloud computing is a space where more skills are needed but are difficult to find.

"Cloud security requires taking concepts like access control and least privilege, and applying them to servers and services that you'll never see and may only control through an API provided by your cloud vendor," wrote Mike Loukides, author of the report. "An error in any service can compromise all your infrastructure — that's why infrastructure as code is so important. In many respects, the game doesn't change, but the stakes become much higher."

Potential talent should prioritize skills like being able think in terms of securing hundreds or thousands of virtual instances, as well as being able to use or develop tools that can reach across multiple servers, services, and cloud providers.

AI, on the other hand, represents a whole new category of threats. Roughly 34% of respondents in the survey pointed to a lack of talent when it comes to AI skills, especially regarding attack avenues such as prompt injection. However, this space is so new that researchers are only beginning to understand the threats and vulnerabilities that AI poses — and even less is known about any possible solutions. 

"The security community is only beginning to catch up with the use and misuse of AI. In the coming years, we expect a surge in AI-specific research, training, and certification," Loukides wrote.

According to Mary Treseler, chief content officer of O'Reilly Media, those hiring in the cyber industry favor those who have a traditional computer science education in addition to experience in IT work such as system admin, help desk, and software development.

"It's possible to get a cybersecurity job without a degree, provided you have relevant work experience," Treseler says. "Certifications and experiences such as bug bounty hunting or capture-the-flag participation can supplement."

Also, some organizations, such as MITRE, are already providing tools to share data on real world AI incidents, such as the AI Incident Sharing initiative under MITRE ATLAS, to combat rising threats. The tool is anonymous in order to serve as a safe space to openly share the details of cyberattacks occurring across industries and government. It's modeled after traditional intelligence-sharing, so organizations can submit incident data through the site, after which they will be considered for membership. And in Europe, an effort is underway to promote AI literacy and awareness for staff dealing with the deployment of AI systems, via the EU Artificial Intelligence Pact.

**Security Up-Skilling: A Marathon, Not a Sprint**

Upskilling to eradicate gaps in cybersecurity talent is the easiest way forward for now, experts say.

"Our global survey underscores a security landscape in flux," Laura Baldwin, president of O'Reilly, said in a press release. "As cyber threats become increasingly sophisticated, it's clear that continuous, high-quality training is no longer optional; it's essential for safeguarding our digital future. Organizations must prioritize ongoing upskilling to stay ahead of evolving risks and build robust defenses."

Certifications, books, videos, and conferences can all be valuable resources to stay up-to-date with the latest need-to-know skills.

Some of the most popular certifications, according to Treseler are CISSP, which she notes is the most versatile and requires five years of work experience, CompTIA Security+, CEH, and CISM. These are all viable options that she says potential talent can trust will give them valuable expertise in the field, but also resources that higher-ups may be looking for when scouring for new candidates.

"Security is a challenge that will never go away," Loukides wrote. "Chances are, we'll invent new risks as quickly as we retire old ones. But we can do better at meeting the challenge."

Cloud, AI Talent Gaps Plague Cybersecurity Teams

Multimodal AI systems can help enterprise defenders weed out fraudulent emails, even if the system has not seen that type of message before.

Source: Peshkova via Shutterstock

Artificial intelligence (AI) models that work across different types of media and domains — so-called "multimodal AI" — can be used by attackers to create convincing scams. At the same time, defenders are finding multimodal AI equally useful at spotting fraudulent emails and not-safe-for-work (NSFW) materials.

A large language model (LLM) can accurately classify previously unseen samples of emails impersonating different brands with better than 97% accuracy, as measured by a metric known as the F1 score, according to researchers at cybersecurity firm Sophos, who presented their findings at the Virus Bulletin Conference on Oct. 4. While existing email-security and content-filtering systems can spot messages using brands that have been encountered before, multimodal AI systems can identify the latest attacks, even if the system is not trained on samples of similar emails.

While the approach will likely not be a feature in email-security products, it could be used as a late-stage filter by security analysts, says Ben Gelman, a senior data scientist at Sophos, which has joined other cybersecurity firms, such as Google, Microsoft, and Simbian, in exploring new ways of using LLMs and other generative AI models to augment and assist security analysts and to help speed up incident response.

"AI and cybersecurity are merging, and this whole AI-generated attack/AI generated defense \[approach\] is going to become natural in the cybersecurity space," he says. "It's a force multiplier for our analysts. We have a number of projects where we support our SOC analysts with AI-based tools, and it's all about making them more efficient and giving them all this knowledge and confidence at their fingertips."

**Understanding Attackers' Tactics**

Attackers have also started using LLMs to improve their email lures and attack code. Microsoft, Google, and OpenAI have all warned that nation-state groups appear to be using these public LLMs for various tasks, such as creating spear-phishing lures and code snippets used to scrape websites.

As part of their research, the Sophos team created a platform for automating the launch of an e-commerce scam campaign, or "scampaigns," to understand what sort of attacks could be possible with multimodal generative AI. The platform consisted of five different AI agents: a data agent for generating information about the products and services, an image agent for creating images, an audio agent for any sound needs, a UI agent for creating the custom code, and an advertising agent to create marketing materials. The customization potential for automated ChatGPT spear-phishing and scam campaigns could result in large-scale microtargeting campaigns, the Sophos researchers stated in its Oct. 2 analysis.

"\[W\]e can see that these techniques are particularly chilling because users may interpret the most effective microtargeting as serendipitous coincidences," the researchers stated. "Spear phishing previously required dedicated manual effort, but with this new automation, it is possible to achieve personalization at a scale that hasn't been seen before."

That said, Sophos has not yet encountered this level of AI usage in the wild.

Defenders should expect AI-assisted cyberattackers to have better quality social-engineering techniques and faster cycles of innovation, says Anand Raghavan, vice president of AI engineering at Cisco Security.

"It is not just the quality of the emails, but the ability to automate this has gone up an order of magnitude since the arrival of GPT and other AI tools," he says. "The attackers have gotten not just incrementally better, but exponentially better."

**Beyond Keyword Matching**

Using LLMs to process emails and turn them into text descriptions leads to better accuracy and can help analysts process emails that might have otherwise escaped notice, stated Younghoo Lee, a principal data scientist with Sophos's AI group, in research presented at the Virus Bulletin conference.

"\[O\]ur multimodal AI approach, which leverages both text and image inputs, offers a more robust solution for detecting phishing attempts, particularly when facing unseen threats," he stated in the paper accompanying his presentation. "The use of both text and image features proved to be more effective" when dealing with multiple brands.

The capability to process the context of the text in the email augments the multimodal capability to "understand" words and context from images, allowing a fuller understanding of an email, says Cisco's Raghavan. LLMs' ability to focus not just on pinpointing suspicious language but also on dangerous contexts — such as emails that urge a user to take a business-critical action — make them very useful in assisting analysis, he says.

Any attempt to compromise workflows that have to do with money, credentials, sensitive data, or confidential processes should be flagged.

"Language as a classifier also very strongly enables us to reduce false positives by identifying what we call critical business workflows," Raghavan says. "If an attacker is interested in compromising your organization, there are four kinds of critical business workflows, \[and\] language is the predominant indicator for us to determine \[whether\] an email is concerning or not."

So why not use LLMs everywhere? Cost, says Sophos's Gelman.

"Depending on LLMs to do anything at massive scale is usually way too expensive relative to the gains that you're getting," he says. "One of the challenges of multimodal AI is that every time you add a mode like images, you need way more data, you need way more training time, and — when the text and the image models conflict — you need a better model and potentially better training" to decide between the two.

**About the Author**

Veteran technology journalist of more than 20 years. Former research engineer. Written for more than two dozen publications, including CNET News.com, Dark Reading, MIT's Technology Review, Popular Science, and Wired News. Five awards for journalism, including Best Deadline Journalism (Online) in 2003 for coverage of the Blaster worm. Crunches numbers on various trends using Python and R. Recent reports include analyses of the shortage in cybersecurity workers and annual vulnerability trends.

AI-Augmented Email Analysis Spots Latest Scams, Bad Content

​​​With careful planning, ongoing evaluation, and a commitment to treat cybersecurity as a core business function, SMBs can transform their vulnerabilities into strengths​​.

Source: Josie Elias via Alamy Stock Photo

COMMENTARY

Small and medium-sized businesses (SMBs) increasingly have become prime targets for cybercriminals. While large corporations often dominate headlines when breaches occur, the reality is that SMBs are at even greater risk. Almost 70% of SMBs reported experiencing at least one cyberattack in the past year. The reasons are clear: SMBs often operate with limited budgets, inadequate cybersecurity tools, and a shortage of skilled cybersecurity professionals. These factors make them particularly vulnerable to the sophisticated and evolving threats of today's cyber environment.  

SMBs are the lifeblood of our economy, and their drive and determination are truly inspiring. The businesses I interact with are exceptionally skilled and consistently deliver outstanding services and products to their customers. I must remind myself, however, that SMBs are not inherently technology companies. Because of budget challenges, they are often considered "soft targets" by threat actors.  

These smaller businesses just want their IT to work seamlessly and securely. Yet, when it comes to mitigating threats like cyber breaches, they are at a disadvantage. While many SMBs understand the importance of cybersecurity, they often need help prioritizing, implementing, and maintaining effective defenses due to limited resources — both financial and technical — compared with larger organizations.  

**Understanding the Landscape **

The range of cyber threats facing SMBs is broad and constantly evolving. Common attack vectors include phishing, ransomware, denial of service, social engineering, and session hijacking, to name a few. Each threat can cause significant harm — whether through intellectual property theft, financial extortion, or reputational damage.  

The most successful cyberattacks exploit the gaps in an organization's cyber-risk strategy. For SMBs, these gaps frequently are the result of constrained resources, limited access to skilled talent, and a reactive approach to cybersecurity. In my conversations with customers and business partners, it's clear that while the concern for cyber-risk is universal, SMBs are often the least equipped to address these risks independently.  

**People, Process, and Technology: A Comprehensive Approach **

To effectively address cyber threats, SMBs must adopt a holistic approach that focuses on three essential components: people, process, and technology.  

**1\. People: Bridging the Skills Gap **

One of the most significant challenges SMBs face is the lack of skilled cybersecurity professionals. Even the best technology and processes can fall short without the right talent. SMBs must assess their current workforce's skills and identify gaps. Addressing these gaps is crucial, whether through training existing employees, hiring new talent, or partnering with external cybersecurity firms.  

In many cases, it may be more practical for SMBs to engage with a trusted partner to supplement their in-house capabilities. Many of the customers I speak with utilize cybersecurity-focused consultancies for short- and mid-term implementations, or rely on managed service providers (MSPs). Additionally, leveraging software-as-a-service (SaaS) solutions can be a cost-effective way to access advanced security tools without requiring extensive in-house expertise. These services often have guaranteed service levels, ensuring that experienced professionals manage critical security functions.  

**2\. Process: Defining Cyber Resilience **

While each organization has unique technical requirements, the need for a well-defined cyber-resilience strategy is universal. SMBs must develop processes tailored to their specific needs and adapt to changing business demands. A one-size-fits-all approach will not suffice. Instead, SMBs should consider standard frameworks like ITIL, Agile, and DevOps as baselines for developing their cybersecurity strategies, as these frameworks can help streamline processes and strengthen the overall cybersecurity posture.  

A key takeaway from my conversations with successful SMBs is the importance of designing sustainable business processes. Cyber resilience is an ongoing journey, not a static goal requiring continuous improvement and adaptability. Every organization must regularly evaluate and update processes to keep pace with evolving needs and emerging threats. By embracing a dynamic approach to process development, SMBs can stay ahead of the curve and maintain robust defenses.  

**3\. Technology: Choosing the Right Tools **

Technology is the cornerstone of any cybersecurity strategy. Given the wide range of available tools, SMBs must carefully select the solutions that best meet their specific needs. Whether focusing on network security, data protection, or identity management, the chosen technology must be both practical and scalable.  

SMBs should focus on ensuring their technology stack aligns with their cybersecurity strategy. This means evaluating on-premises and cloud-based solutions while carefully managing access to sensitive data. The objective is to choose technology that not only addresses immediate security concerns but also strengthens long-term resilience.  

**Engaging Leadership and Industry **

A critical aspect of any successful cybersecurity program is the involvement of leadership at every level of the organization. From my discussions with business leaders who have established robust cyber resilience programs, one common theme emerges: Cybersecurity is a serious priority across the organization. It's not merely the IT department's responsibility but a critical business imperative that affects reputation, financial health, and legal compliance.  

To secure this level of commitment, SMBs must involve their leadership teams in developing and overseeing cybersecurity strategies. This entails conducting regular assessments of the program's effectiveness, incorporating feedback from both cybersecurity professionals and business leaders. When leadership is actively involved, it sends a clear message that cybersecurity is a priority, fostering a culture of security throughout the organization.  

Another critical factor is the willingness to seek external expertise. Successful SMBs often look beyond their internal resources, utilizing market analysis, user groups, vendor forums, and industry contacts to inform their cybersecurity strategies. For SMBs with limited staff and experience, these external resources offer valuable insights and support critical to the success of their programs. 

**Conclusion: A Proactive Path Forward **

Cybersecurity is not a one-time effort — it's an ongoing commitment that requires vigilance, adaptability, and strategic investment. For SMBs, the path to cyber resilience may be challenging, but it is achievable with the right approach. By focusing on the critical areas of people, processes, and technology, and engaging leadership at all levels, SMBs can develop robust defenses that safeguard their assets, reputation, and future growth.  

Ultimately, it's not just about preventing attacks. It’s about building a resilient organization that can thrive in an increasingly digital and complex business environment. As threats evolve, SMBs must continuously adapt their strategies and solutions to protect their businesses. Through careful planning, ongoing evaluation, and a commitment to treat cybersecurity as a core business function, SMBs can transform their vulnerabilities into strengths and secure their place in the digital economy.  

**About the Author**

CEO, One Identity

Mark Logan serves as Chief Executive Officer of One Identity and joined the company in June 2022. He is responsible for driving the overall growth strategy, go-to-market and P&L for One Identity. Mark is a seasoned executive with experience developing enterprise software companies to achieve successful growth. He is a proven leader with nearly two decades of C-suite experience at companies such as Emptoris (now part of IBM), Attunity, and most recently LogRhythm. Having been in the cyber security space for several years, Mark believes One Identity is in the right market at the right time to provide critically important cybersecurity protection for enterprises, public sector organizations and federal agencies.

Source

DARKReading