A research tool by the company found a vulnerability in the SQLite open source database, demonstrating the "defensive potential" for using LLMs to find vulnerabilities in applications before they're publicly released.

Source: Krot Studio via Alamy Stock Photo

Google has discovered its first real-world vulnerability using an artificial intelligence (AI) agent that company researchers are designing expressly for this purpose. The discovery of a memory-safety flaw in a production version of a popular open source database by the company's Big Sleep large language model (LLM) project is the first of its kind, and it has "tremendous defensive potential" for organizations, the Big Sleep team wrote in a recent Project Zero blog.

Big Sleep — the work of a collaboration between the company's Project Zero and Deep Mind groups — discovered an exploitable stack buffer underflow in SQLite, a widely used open source database engine.

Specifically, Big Sleep discovered a pattern in the code of a publicly released version of SQLite that creates a potential edge case that needs to be handled by all code that uses the field, the researchers noted. A function in the code failed to correctly handle the edge case, "resulting in a write into a stack buffer with a negative index when handling a query with a constraint on the 'rowid' column," thus creating an exploitable flaw, according to the post.

Google reported the bug to SQLite developers in early October. They fixed it on the same day and before it appeared in an official release of the database, so users were not affected.

Related:News Desk 2024: Hacking Microsoft Copilot Is Scary Easy

**Inspired by AI Bug-Hunting Peers**

"We believe this is the first public example of an AI agent finding a previously unknown exploitable memory-safety issue in widely used real-world software," the Big Sleep team wrote in the post. While this may be true, it's not the first time an LLM-based reasoning system autonomously found a flaw in the SQLite database engine, Google acknowledged.

An LLM model called Atlantis from a group of AI experts called Team Atlanta discovered six zero-day flaws in SQLite3 and even autonomously identified and patched one of them during the AI Cyber Challenge organized by ARPA-H, DARPA, and the White House, the team revealed in a blog post in August.

In fact, the Big Sleep team used one of the Team Atlanta discoveries — of "a null-pointer dereference" flaw in SQLite —  to inspire them to use AI "to see if we could find a more serious vulnerability," according to the post.

**Software Review Goes Beyond Fuzzing**

Google and other software development teams already use a process called fuzz-testing, colloquially known as "fuzzing," to help find flaws in applications before release. Fuzzing is an approach that targets the software with deliberately malformed data — or inputs — to see if it will crash so they can investigate and fix the cause.

Related:Privacy Anxiety Pushes Microsoft Recall AI Release Again

In fact, Google earlier this year released an AI-boosted fuzzing framework as an open source resource to help developers and researchers improve how they find software vulnerabilities. The framework automates manual aspects of fuzz-testing and uses LLMs to write project-specific code to boost code coverage.

While fuzzing "has helped significantly" to reduce the number of flaws in production software, developers need a more powerful approach "to find the bugs that are difficult (or impossible) to find" in this way, such as variants for previously found and patched vulnerabilities, the Big Sleep team wrote.

"As this trend continues, it's clear that fuzzing is not succeeding at catching such variants, and that for attackers, manual variant analysis is a cost-effective approach," the team wrote in the post.

Moreover, variant analysis is a better fit for current LLMs because its provides them with a starting point —  such as the details of a previously fixed flaw — for a search, and thus removes a lot of ambiguity from AI-based vulnerability testing, according to Google. In fact, at this point in the evolution of LLMs, lack of this type of starting point for a search can cause confusion, they noted.

Related:OWASP Releases AI Security Guidance

"We're hopeful that AI can narrow this gap," the Big Sleep team wrote. "We think that this is a promising path towards finally turning the tables and achieving an asymmetric advantage for defenders."

**Glimpse Into the Future**

Google Big Sleep is still in its research phase, and using AI-based automation to identify software flaws overall is a new discipline. However, there already are tools available that developers can use to get a jump on finding vulnerabilities in software code before public release.

Late last month, researchers at Protect AI released Vulnhuntr, a free, open source static code analyzer tool that can find zero-day vulnerabilities in Python codebases using Anthropic's Claude artificial intelligence (AI) model.

Indeed, Google's discovery shows promising progress for the future of using AI to help developers troubleshoot software before letting flaws seep into production versions.

"Finding vulnerabilities in software before it's even released means that there's no scope for attackers to compete: the vulnerabilities are fixed before attackers even have a chance to use them," Google's Big Sleep team wrote.

**About the Author**

Elizabeth Montalbano is a freelance writer, journalist, and therapeutic writing mentor with more than 25 years of professional experience. Her areas of expertise include technology, business, and culture. Elizabeth previously lived and worked as a full-time journalist in Phoenix, San Francisco, and New York City; she currently resides in a village on the southwest coast of Portugal. In her free time, she enjoys surfing, hiking with her dogs, traveling, playing music, yoga, and cooking.

Google: Big Sleep AI Agent Puts SQLite Software Bug to Bed

Feeling creative? Submit your caption and our panel of experts will reward the winner with a $25 Amazon gift card.

When you let go of that trapeze, you really want your teammates to be ready to catch you. Send us a cybersecurity-related caption to describe the above scene and our favorite will win its wordsmith a $25 Amazon gift card.

Here are four convenient ways to submit your ideas before the Nov. 27, 2024 deadline:

• Email \[email protected\] with the subject line "Edge November Toon."

• Via social media: X, Facebook, and LinkedIn. (If you win, we'll respond to you on the same platform, requesting your email address.)

**Last Month's Winner**

Shout out — and a $25 Amazon gift card — to Joseph Ayella, a recurring winner, for sending us the winning caption for last month's "And For My Next Trick..." cartoon.

"I call this trick the woman-in-the-middle attack."

Some noteworthy contenders included: "All we could get was half a FTE for security assessments," "And with this SIEM solution, we could cut a half of our SOC analysts," and "I'm glad he's not the hacker." A big thank you to all who participated.

**About the Author**

John Klossner has been drawing technology cartoons for more than 15 years. His work regularly appears in Computerworld and Federal Computer Week. His illustrations and cartoons have also been published in The New Yorker, Barron's, and The Wall Street Journal.

Name That Edge Toon: Aerialist's Choice

The true measure of our cybersecurity prowess lies in our capacity to endure.

Source: Lasse Kristensen via Alamy Stock Photo

COMMENTARY

In July, the industry witnessed one of the largest technology outages in recent history, with estimates of $5.4 billion in damages. When CrowdStrike distributed a Rapid Response Content Channel Update with an exception-handling logic flaw, it opened the door for constructive conversations about automatic updates — when to use them, when not to use them, whether they make us more or less secure. It's time to reflect and ask: What is the cost of our relentless pursuit of innovation, software currency, and speed to market? How can we reprioritize to reestablish the balance in the C-I-A triad?

IT and security teams are under enormous pressure to stay ahead of threats. However, teams must not sacrifice the right checks and balances for speed. The CrowdStrike incident serves as a reminder to the industry that even the most secure and trusted systems can fail, and it's time to revisit how teams test and deploy critical updates.  

**The C-I-A Triad: Rebalancing Priorities**

The C-I-A triad is a foundational pillar of cybersecurity, representing the Confidentiality (security), Integrity (accuracy), and Availability of technology platforms. For too long, the cybersecurity community — vendors and customers alike — have fixated on the C in this triad. However, the C-I-A triad is supposed to represent the full scope of a cybersecurity program. With the main focus on privacy and data security, the industry over emphasized security — and in doing so, added speed to the equation. Teams are now responding faster and deploying updates quicker to stay ahead of emerging threats and day-to-day attacks, but that's leading to mistakes and improper testing.  

Meanwhile, the I and A were relegated to secondary status — even outsourced to other technology teams. Integrity — the accuracy, completeness, and consistency of the ecosystem and underlying data — was compromised in the name of speed. Availability also suffered as the focus shifted to rapid recovery rather than ensuring uptime and reliability, all for the sake of rapid innovation and response to perceived threats.  

If the CrowdStrike event has taught us anything, it is that now is the time for both vendors and customers to recommit themselves to recognizing the integral importance of and essential need to rebalance all three pillars of the C-I-A triad. In doing so, teams can build more resilient systems.  

**The Shift From Software to Critical Infrastructure**

Leaders need to undertake three key shifts to achieve the essential checks and balance systems inherent to the C-I-A triad.  

1\. Transparency: Vendors must be more transparent with their product updates and give customers more control over how updates are applied. Customers should be able to manually update, deploy updates in stages, and remain on a prior stable version as a matter of policy.  

In the case of the CrowdStrike event, the complex update caused the outage. First, the team deployed a configuration file in February. Later, in July, it deployed a Rapid Response Content Update. As part of that update, a configuration content validator, using the prior configuration file, attempted to apply the update, but due to the "logic bug" in the exception handling routines, the staggered update resulted in the infamous "blue screen of death" for many Windows servers and workstations. These channel updates are often a series of staged updates, all occurring at once. How many of CrowdStrike's customers understood this nuance of the update strategy? It's unclear, but they had limited control over the update and were unable to stage it so it could be certified and tested before affecting the entirety of the enterprise.  

2\. Reevaluate vendor testing: Platforms such as CrowdStrike have transformed to become a core component of critical infrastructure. Security vendors frequently push automatic updates to improve security, but this can also mean speeding through the "trust but verify; walk before you run; test test test" cycles. While speed matters, this incident should force teams to take a closer look at how they deploy updates, ensure integrity and availability, and maintain business resiliency.  

IT and security teams must reevaluate overreliance on vendor testing and automatic updates. Even small teams can have the flexibility to choose when to update without incurring substantial overhead. The update is automatic — but the time and place to update can be chosen. Leaders should consider implementing staggered updates, using staging and testing environments to certify and assess the viability and stability of the update. More credence and consideration should be given to the value of updating now versus waiting to give more ability to ensure that the integrity and availability won't be compromised by the update.  

3\. Improve testing environments: Companies must ensure that cybersecurity teams have adequate testing environments available for certifying and testing security updates and implementations. The same diligence given to IT and development teams must be applied to cybersecurity.  

Security is no longer software; it's a foundational component of critical infrastructure. As seen with the CrowdStrike event, banks, transportation, manufacturing, and financial markets can all be devastated by a failure of the security ecosystem. As the industry continues to see convergence of solutions to a few vendors, it's important to make these platforms more resilient.   

The true measure of our cybersecurity prowess lies in our capacity to endure. Teams should embrace those proven patterns of change management that have served us well in the past, but also evolve and expand in scope to accommodate new technology and new potential threats. Vendors must empower customers with greater control and flexibility in how and why they deploy our solutions and updates. Technology and security practitioners, in turn, must use this moment as a clarion call to rethink priorities and recommit to balancing and counterbalancing the security, integrity, and availability drivers that empower our security tools.   

This creates a durable security future, regains and rebuilds essential fiduciary trust, and ensures that teams can rise to every threat while never again falling into complacency, valuing speed and ease at the expense of everything else. 

**About the Author**

CISO, Silverfort

John Paul Cunningham is the CISO of Silverfort. He has a passion for managing security and risk within firms, and has built and led information security, risk management, comprehensive technology organizations, and developed customer-facing organizations built on customer and organizational success. His sphere of influence extends from the board room to the break room, where he brings business and technology together to deliver solutions that propel the organization forward to achieve its goals.

Can Automatic Updates for Critical Infrastructure Be Trusted?

The large-scale operation took advantage of open repositories, hardcoded credentials in source code, and other cloud oversights.

Source: Tithi Luadthong via Alamy Stock Photo

Earlier this week, researchers uncovered a major cybercriminal operation, dubbed EmeraldWhale, after the attackers dumped more than 15,000 credentials into a stolen, open AWS S3 bucket in a massive Git repository theft campaign. The incident is a reminder to tighten up cloud configurations and review source code for mistakes like the inclusion of hardcoded credentials.

Over the course of the onslaught, EmeraldWhale targeted Git configurations in order to steal credentials, cloned more than 10,000 private repositories, and extracted cloud credentials from source code. 

The campaign used a variety of private tools to abuse misconfigured Web and cloud services, according to the Sysdig Threat Research Team, which discovered the global operation. Phishing is the primary tool the campaign used to steal the credentials, which can be worth hundreds of dollars per account on the Dark Web. The operation also makes money by selling its target lists on underground marketplaces for others to engage in the same activity.

**EmeraldWhale's First Breach**

The researchers were initially monitoring Sysdig TRT cloud honeypot when it observed a ListBuckets call using a compromised account — an S3 bucket dubbed s3simplisitter.

The bucket belonged to an unknown account and was publicly exposed. After launching an investigation, the researchers found evidence of a multifaceted attack, including Web scraping of Git files in open repositories. A massive scanning campaign occurred between August and September, according to the researchers, affecting servers with exposed Git repository configuration files, which can contain hardcoded credentials.

"As security professionals, we cannot afford to be complacent, particularly when it comes to keeping sensitive secrets, API tokens, and authentication credentials out of our source code," Naomi Buckwalter, director of product security at Contrast Security, wrote in an emailed statement to Dark Reading. "Not only should infosec professionals be on the front lines educating their development teams on how to securely store, manage, and access secrets, they should also regularly scan their source code for hard coded credentials and monitor credential usage for anomalous activity."

**Always Have Your Guard Up**

In general, Git directories contain "all information required for version control, including the complete commit history, configuration files, branches, and references."

"If the .git directory is exposed, attackers can retrieve valuable data about the repository's history, structure, and sensitive project information," added the researchers. "This includes commit messages, usernames, email addresses, and passwords or API keys if the repository requires them or if they were committed."

The incident is clear reminder that it's critical for businesses and organizations to have visibility on all services and get a clear view on potential attack surfaces in order to consistently manage them and mitigate threats.

"Many breaches occur because internal services are inadvertently exposed to the public Internet, making them easy targets for malicious actors," Victor Acin, head of threat intel at Outpost24, wrote in an emailed statement to Dark Reading.

Acin recommended that enterprises implement a "proper external attack surface management (EASM) platform" to keep track of potential misconfigurations and shadow IT.

And even when private repositories are supposedly secure, it's worth adding additional protections and ensuring that information is locked down.

EmeraldWhale's Massive Git Breach Highlights Config Gaps

PRESS RELEASE

LAS VEGAS, Oct. 30, 2024 /PRNewswire/ -- MONEY 20/20 --  AU10TIX, a global technology leader in identity verification and management, today released its Q3 2024 Global Identity Fraud Report at Money 20/20 USA in Las Vegas. Drawing insights from millions of transactions processed around the globe from July to September 2024, the report uncovers significant trends in large-scale organized identity fraud. This quarter, automated bot attacks targeting social media platforms surged in the lead-up to next week's US presidential election, with the sector accounting for 28% of all attacks in Q3, up from just 3% in Q1. The report also reveals a faster-than-anticipated leap in the sophistication of AI-powered impersonation bots and deepfake technology, including synthetic selfies that have shown the ability to bypass some leading verification systems.

Rapid developments in AI technology have enabled the industrialization of identity fraud, with bad actors launching automated mega-attacks of thousands of false identities targeting payments, crypto and social media companies all over the world. In Q3, AU10TIX observed an especially high number of bot attacks combined with deepfake content in attempts to open fake social media accounts at scale. Some of the attacks appeared to incorporate advanced randomized GenAI elements, marking a new level of automated detection evasion. This rapid growth began in March 2024 and peaked in September, with outlets like Associated Press reporting on bots spreading disinformation and manipulating public perception in the weeks leading up to the US election.

Another major Q3 development involved the debut of synthetic selfies. Selfies have historically been one of the least-used methods of fraud, due to the difficulty of outsmarting facial matching tech. However, fraudsters are now exploiting extremely sophisticated technology to create 100% deepfaked "selfies" that match the synthetic IDs they use to trick automated KYC processes.

"Fraudsters are evolving faster than ever, leveraging AI to scale and execute their attacks, especially in the social media and payments sectors," said Dan Yerushalmi, CEO of AU10TIX. "While companies are using AI to bolster security, criminals are weaponizing the same technology to create synthetic selfies and fake documents, making detection almost impossible. The only way to detect this type of fraud is by analyzing behavior at the traffic level, as AU10TIX does with our Serial Fraud Monitor. We are committed to continually advancing our detection methods to protect customers against this rapid evolution of fraud tactics, using a combination of advanced AI, biometric verification, and deepfake detection."

In addition to the increase in synthetic selfies, AU10TIX reported a 20% rise in the use of "image template" attacks by bad actors. This trend suggests that AI is being used to rapidly create variations of synthetic identities—including photos, document numbers, and other personal information—using the same ID template. The regional hotspots driving this change are APAC and North America; the surges have not been observed in other regions.

On a more positive note, fraud rates in the payments sector dropped from 52% in Q2 to 39% in Q3, which AU10TIX attributes to increased self-regulation and law enforcement interventions. Although the payments industry continued to be the most targeted segment, discouraged fraudsters shifted their attention to less regulated sectors like the crypto market, which accounted for 31% of Q3 attacks.

AU10TIX's Q3 2024 Global Identity Fraud Report offers three actionable insights to help organizations protect against identity fraud:

*   Adopt Behavior-Based Detection: Traditional document verification alone isn't enough. To catch today's sophisticated fraudsters, organizations must analyze user behavior and traffic patterns.
    
*   Use AI to Combat AI: As fraudsters deploy AI to scale attacks, it is critical to leverage AI-driven solutions to keep pace with these threats.
    
*   Be Ready for Synthetic Selfies: The rise in synthetic selfies means that relying on traditional KYC methods may no longer be effective. Organizations must enhance their detection systems to spot these new forms of fraud.
    

About AU10TIX   
AU10TIX plays a pivotal role in establishing trust between individuals/companies and digital systems. Founded in 2002, it is the global leader in identity verification and management, protecting the world's largest brands against advanced fraud. The company's future-proof product portfolio helps businesses provide frictionless customer onboarding and verification in 4-8 seconds while staying ahead of emerging threats and evolving regulatory requirements. AU10TIX offers the world's only 100% automated global identity management system, as well as the industry's only solution that can detect organized mass attacks by analyzing traffic patterns and cross-checking data in a consortium of more than 60 major companies. With its deep roots in airport security, AU10TIX has authenticated billions of identities and prevented over $18 billion in identity fraud. AU10TIX is a subsidiary of ICTS International N.V. (OTCQB: ICTSF). Connect with AU10TIX on LinkedIn and on X at @AU10TIXLimited. For more information, visit AU10TIX.com.

AU10TIX Q3 2024 Global Identity Fraud Report Detects Skyrocketing Social Media Attacks

PRESS RELEASE

GZIRA, Malta, Oct. 30, 2024 /PRNewswire/ -- SOFTSWISS, a leading software supplier in the iGaming industry, joins Cybersecurity Awareness Month. The company is enhancing its cybersecurity efforts by leveraging the expertise of white-hat hackers and independent security researchers, inviting them to participate in its private Bug Bounty Program.

The program requires experts to identify vulnerabilities and ensure the highest level of security for SOFTSWISS clients. To ensure that only significant cases that meet specific requirements are reported, SOFTSWISS launched a private program with invitation-only access. 

Invitation-only programs encourage white hat hackers to pay close attention to their terms and requirements, resulting in higher-quality reports. While public programs can offer broader perspectives but tend to generate more irrelevant reports, the private launch ensures that the focus remains on critical security issues. 

The Bug Bounty Program offers various financial incentives, depending on the severity and complexity of the discovered vulnerabilities. This program serves as an additional layer of defence, providing external, unbiased assessments from highly skilled security experts. 

"We care about the security of our clients and their players, which is why it is important for us not only to pay close attention to protecting the products we develop but also to constantly recheck our applications with the help of skilled external experts and enthusiasts. In collaboration with the expert community, we can ensure a reliable level of data protection and uninterrupted operations for all our clients," said Evgeny Zaretskov, Group Chief Information Security Officer at SOFTSWISS. "SOFTSWISS is proud to set a new standard for cybersecurity by leveraging crowdsourced expertise. In this ever-evolving landscape, even a minor bug can lead to significant losses for operators. The Bug Bounty Program is an extra measure to protect our existing and future clients."

The program has started with two products in the SOFTSWISS portfolio. It is conducted in a dedicated test environment, which operates independently of the iGaming project systems, ensuring no disruption to player experience or platform performance.

Moving forward, the SOFTSWISS cybersecurity team plans to broaden the frames, adding more products and refining requirements to maintain the highest standards of security across all platforms.

SOFTSWISS is an international tech company with over 15 years of experience developing innovative solutions for iGaming. The company revolutionised the industry by introducing the world's first Bitcoin-optimised solution for iGaming. 

You May Also Like

SOFTSWISS Expands Bug Bounty Program

The Recall AI tool will be available to Copilot+ PC subscribers in December, and can be used to record images of every interaction on the device for review later. Critics say this introduces major privacy and security concerns along with useful functionality.

Source: GK Images via Alamy Stock Photo

Microsoft has made the decision to once again delay the release of its new artificial Intelligence tool, Recall, while the company works through trying to make sure all of the handy data it delivers can't be abused by adversaries.

The Recall tool will be part of the suite of services delivered through Microsoft's AI Assistant software, Copilot+. Recall's job, once it's rolled out, will be to gather "snapshots" of each action on the PC to be accessible later through an easy search. The software will be able to "recall" the exact moment the user saw a website, used an app, or interacted with a document.

Compelling use cases aside, information security professionals have balked at Recall's ability to keep its snapshots secure from would-be threat actors. For its part, Microsoft has taken these cybersecurity concerns seriously. In June, Microsoft announced it had added new privacy and security features to Recall just days ahead of its intended rollout date. That release was ultimately pushed back to October in order to take extra steps to shore up the tool's security. Now, the release date has been pushed back again.

"We are committed to delivering a secure and trusted experience with Recall," according to a statement about the delay from Brandon LeBlanc, senior product manager for Windows. "To ensure we deliver on these important updates, we’re taking additional time to refine the experience before previewing it with Windows Insiders. Originally planned for October, Recall will now be available for preview with Windows Insiders on Copilot+ PCs by December."

Related:News Desk 2024: Hacking Microsoft Copilot Is Scary Easy

**Microsoft Pledges to Secure Recall**

In late September, David Weston, Microsoft's vice president of enterprise and OS security, detailed the company's commitment to the security of Recall data, stressing the tool is opt-in only, encrypted, and includes malware protection; and, its data is protected in a virtualization-based security (VBS) enclave inaccessible by even admin and kernel users without biometric authentication.

"Using VBS Enclaves with Windows Hello enhanced sign-in security allows data to be briefly decrypted while you use the Recall feature to search. Authorization will time out and require the user to authorize access for future sessions," Weston wrote. "This restricts attempts by latent malware trying to 'ride along' with a user authentication to steal data."

Weston further assured those concerned about Recall's security that: in-private browsing information is never saved by Recall; users have an option to filter out specific sites or apps from Recall recording; content filtering keeps data like credit card and Social Security numbers from being stored; users can delete stored information by date, content, app, or website; and an icon clearly shows when snapshots are being saved, so users can easily pause the function.

Related:Cybersecurity Job Market Stagnates, Dissatisfaction Abounds

"Recall’s secure design and implementation provides a robust set of controls against known threats," Weston added. "Microsoft is committed to making the power of AI available to everyone, while retaining security and privacy against even the most sophisticated attacks."

**Is Microsoft Eyeing Claude's 'Computer Use' Feature?**

It appears Microsoft is taking the warnings from the cybersecurity community about Recall's potential business risks seriously, Bugcrowd founder Casey Ellis tells Dark Reading. Redmond might also have its eye on a recent release of a similar tool in Anthropic's Claude AI before rolling out Recall, he adds.

"After the initial reaction to Recall — and some of the security and privacy concerns raised by how it was implemented — Microsoft appears to be hastening slowly here," Ellis says. "I wouldn’t be surprised if they’re taking the opportunity to learn from how the market responds to and uses Anthropic’s 'computer use' feature, which is very similar to Recall from a privacy, security, and functionality standpoint."

Related:Noma Launches With Plans to Secure Data, AI Life Cycle

Released just days ago, the computer use feature allows the latest version of Claude to interact with a computer in the same way as a human. Claude's new feature, like Recall, ingests screenshots from Internet-connected computers. And in its Oct. 22 announcement of the release, Anthropic admitted the tool does indeed come with inherent cybersecurity risks.

"In this spirit, our Trust & Safety teams have conducted extensive analysis of our new computer-use models to identify potential vulnerabilities," the release announcement said. "One concern they've identified is prompt injection — a type of cyberattack where malicious instructions are fed to an AI model, causing it to either override its prior directions or perform unintended actions that deviate from the user's original intent."

Anthropic added that it hopes to work out this and other issues in its public beta phase, which will certainly be of keen interest to Microsoft as it works through its Recall release.

Claude, according to Anthropic, will not use this user-submitted data to train its own AI model. But when it comes to Microsoft, security consultant John Bambenek isn't so sure Recall will adhere to the same standard.

"AI systems require tons of data, which means Microsoft wants all the data on how users are interacting with their computers," Bambenek says. "I am not sure the feature is terribly useful for end users, however, it certainly is for training future models. It has enormous privacy implications, so hopefully the delay is useful in terms of minimizing the risks and potential harms to end users."

While Microsoft security teams and Anthropic's Claude feature testing move forward, Patrick Harr, CEO of SlashNext Email Security, warns these tools remain vulnerable to cyberattack.

"We continually see phishing and socially engineered attacks from professional groups, mimicking support staff that target company users either through email, other messaging apps, or even bot calls to provide remote access to their desktops," Harr says. "Once accessed into Recall, the threat actors have perfect timeline and information about that user that can be exploited. Proceed with caution until this update is done."

Privacy Anxiety Pushes Microsoft Recall AI Release Again

OWASP has released guidance materials addressing how to respond to deepfakes, AI security best practices, and how to secure open source and commercial generative AI applications.

Source: Jozef Sedmak via Alamy Stock Photo

The Open Worldwide Application Security Project (OWASP) has announced new security guidance materials to help organizations identify and manage the risks associated with the adoption, deployment, and management of large language models (LLMs) and generative artificial intelligence (GenAI) applications. 

The guidance is part of the OWASP Top 10 for LLM Application Security Project, a global, community-led open source project. Since its inception in 2023, the group has released research, guidance, and resource materials to help organizations develop a comprehensive strategy encompassing governance, collaboration, and practical tools.

*   The "Guide for Preparing and Responding to Deepfake Events" illustrates the problems posed by "hyper-realistic digital forgeries." An outgrowth of the AI Cyber Threat Intelligence initiative, this resource combines practical and pragmatic defense strategies to help organizations stay secure as deepfake technology improves. 
    
*   The "Center of Excellence Guide" helps businesses establish best practices and frameworks for creating AI security practices. The guidance helps organizations establish systems for risk management and interdepartmental coordination among security, legal, data science, and operations teams, as well as how to develop and enforce security policy and educate staff on AI security.
    
*   The "AI Security Solution Landscape Guide" is a broad reference on how to secure both open source and commercial LLM and GenAI applications. It categorizes existing and emerging security products and gives guidance on how to think about risks identified in the Top 10 list.
    

The project brings together more than 500 cybersecurity and AI experts from companies and organizations around the world to identify LLM vulnerabilities and mitigations. In early 2024, the project expanded its focus to include strategic stakeholders, like CISOs and compliance officers, in addition to developers, data scientists, and other security practitioners.

"We're two years into the generative AI boom, and attackers are using AI to get smarter and faster," said Steve Wilson, project lead for the OWASP Top 10 for LLM Project, in a statement. "Security leaders and software developers need to do the same. Our new resources arm organizations with the tools they need to stay ahead of these increasingly sophisticated threats."

**About the Author**

Jennifer Lawinski is a writer and editor with more than 20 years experience in media, covering a wide range of topics including business, news, culture, science, technology and cybersecurity. After earning a Master's degree in Journalism from Boston University, she started her career as a beat reporter for The Daily News of Newburyport. She has since written for a variety of publications including CNN, Fox News, Tech Target, CRN, CIO Insight, MSN News and Live Science. She lives in Brooklyn with her partner and two cats.

OWASP Releases AI Security Guidance

The sophisticated Chinese cyberattacks of today rest on important groundwork laid during the pandemic and before.

Source: Cinematic via Alamy Stock Photo

Chinese threat actors are operating at a higher level today than ever before, thanks to years of trial-and-error-style attacks against mass numbers of edge devices.

Networking devices are a known favorite of China's advanced persistent threats (APT), and why wouldn't they be? Sitting on the outer banks of an enterprise network, they not only allow threat actors a way in, they also double as useful nodes for botnets. They offer opportunities for lateral movement, they often store sensitive data, and network defenders have a harder time seeing into and securing them than they do other kinds of network computers. 

Over time, Chinese APTs have been improving on their edge attack capabilities. Since 2018, Sophos has traced a distinct evolution in tactics: from naive, low-level attacks came more sophisticated campaigns against massive numbers of devices, followed by a period of more targeted attacks against specific organizations.

**The First Salvo in a Long Cyber War**

On Dec. 4, 2018, Sophos analysts discovered a suspicious device running network scans against Cyberoam, a Sophos subsidiary based in India. In some ways the attack was run of the mill, using commodity malware and common living-off-the-land (LotL) tactics.

Other evidence, though, suggested that this was something different. For example, the attacker utilized a novel technique to pivot from on-premises devices to the cloud, via an overly permissive identity and access management (IAM) configuration to the Amazon Web Services Systems Manager (AWS SM).

Related:Dark Reading News Desk Live From Black Hat USA 2024

"AWS SM was quite a new technology, and it was quite a subtle misconfiguration," Sophos chief information security officer (CISO) Ross McKerchar recalls. "That was one of the first indicators that we were up against an interesting adversary." 

Later, the attackers deployed a novel rootkit called Cloud Snooper. Cloud Snooper was so stealthy that two third-party consultancies missed it in their analysis, before Sophos eventually picked up on its presence.

The goal of the attack, it seemed, was to collect information useful for future attacks against edge devices. It was a harbinger of what was to come.

**A Five-Year Evolution in Chinese TTPs**

Chinese cyber threats blossomed from roughly 2020 to 2022, as attackers focused on identifying and breaching edge devices en masse.

It worked thanks to the large quantity of devices in the wild that have Internet-facing portals. Typically, these interfaces are designed for internal use. With COVID-19, though, more and more companies were allowing employees to connect from the open Web. This provided a window for hackers with the right kind of credentials or vulnerabilities to get in.

Related:The Overlooked Importance of Identifying Riskiest Users

It helped, too, that around that same time — July 2021 — China's Cyberspace Administration passed the Regulations on the Management of Network Product Security Vulnerability Information rules. These mandates forced cybersecurity researchers to report vulnerabilities to the country's Ministry of Industry and Information Technology (MIIT) before disclosing to any other parties. "It was designed to co-opt the whole country — private citizens included — into being assets for PRC objectives," McKerchar says. Sophos argues with medium confidence that two notable campaigns during this period were facilitated by vulnerabilities responsibly disclosed by researchers at universities in the Chinese city of Chengdu.

Chinese APTs weren't only interested in using compromised devices to attack the companies from whence they came. With varying degrees of success, they would often try to incorporate the devices into broader operational relay box networks (ORBs). These ORBs, in turn, offered higher-level threat actors more sophisticated infrastructure from which to launch more advanced attacks and hide any trace of their origin.

Related:FBI, Partners Disrupt RedLine, Meta Stealer Operations

**What's Happening Now**

After this noisy period, around the middle of 2022, Chinese APTs shifted yet again. Ever since, they've been focused on much more deliberate and targeted attacks against organizations of high value: government agencies, military contractors, research and development firms, critical infrastructure providers, and the like.

These attacks follow no single pattern, involving known and zero-day vulnerabilities, userl and and UEFI bootkits, and whatever other elements pair with active, hands-on-keyboard-type attacks. They almost certainly wouldn't be as sophisticated as they are, though, without all of the years of trial and error that occurred before. Evidence to that is just how effective these threat actors are at overcoming cybersecurity defenses. In recent years, they've demonstrated an ability to sabotage hotfixes for vulnerable devices, and block evidence of their activity from reaching Sophos analysts.

"There's a clear arc of moving to stealthier and stealthier persistence in the activity that we've uncovered," McKerchar says.

He explains how "the first malware, whilst it was bespoke for our devices, it wasn't really trying to hide. They were just banking on nobody looking. In the second wave of attacks they learned a bunch of lessons, remarkably quickly. The malware wasn't explicitly trying to hide, it was just smaller, and naturally able to blend in a bit more. Then after that, they started kind of pulling out more interesting tactics: Trojan class files, memory-resident malware, rootkits, bootkits."

He concludes, "It'd be hard to speculate on what's next, except \[that\] they're going to be improving again."

**About the Author**

Nate Nelson is a freelance writer based in New York City. Formerly a reporter at Threatpost, he contributes to a number of cybersecurity blogs and podcasts. He writes "Malicious Life" -- an award-winning Top 20 tech podcast on Apple and Spotify -- and hosts every other episode, featuring interviews with leading voices in security. He also co-hosts "The Industrial Security Podcast," the most popular show in its field.

Chinese APTs Cash In on Years of Edge Device Attacks

Misconfigurations, weak authentication, and logic flaws are among the main drivers of API security risks at many organizations.

Source: Who is Danny via Shutterstock

Security vulnerabilities in the application programming interfaces (APIs) powering modern digital services and applications have emerged as a major threat to enterprise systems and data.

A recent report from Wallarm shows a 21% increase in API-related flaws between this year's second and third quarters. Nearly one-third (32%) were associated with cloud infrastructure and cloud-native applications and services. In addition to the increased volume, a high proportion of the vulnerabilities that Wallarm reviewed had severity scores of 7.5 or higher, indicating growing risk for organizations from API use.

"In Q3 we saw API breaches driven by authentication and authorization issues, leaked API data, and classic injection attacks," says Wallarm founder and CEO Ivan Novikov.

Significantly, while many of the vulnerabilities in OWASP's list of Top 10 API vulnerabilities are server-focused, Wallarm's data shows an uptick in client-side flaws, like OAuth misconfiguration and cross-site issues, Novikov says.

"It's concerning because defenders are resource-constrained and need to focus on the most important types of attacks," he says.

A continuing enterprise emphasis on API integration and functionality — over security — is exacerbating the issue. Just 37% of organizations have formally incorporated security testing into their API life cycle management practices, a study by Postman found earlier this year.

"APIs are now a top target for malicious actors, making security and observability critical," the report noted.

What are the major contributors to API security risks? And what should organizations be doing to mitigate them?

**Misconfigured APIs**

Many API security issues in recent years have stemmed from relatively easily avoidable misconfigurations. Common examples include inadequate authentication and authorization, lack of input validation, improper rate limiting, inadequate logging and monitoring, and the exposure of sensitive data through error messages. Such misconfigurations can have severe consequences.

For instance, Broken Object Level Authorization — when an API does not properly validate user access to resources — can allow attackers to manipulate object IDs to access unauthorized data, says Ankit Sobti, co-founder and CTO of Postman. Similarly, Broken User Authentication vulnerabilities — when an API fails to enforce proper authentication — often allow attackers to bypass authentication checks and gain unauthorized access to endpoints.

Organizations can mitigate these issues by implementing security best practices, such as strict authorization checks, role-based access control, multifactor authentication, server-side data filtering, and reviewing API responses for unnecessary data.

"Without proper rate limiting, APIs become vulnerable to abuse through techniques like brute-force attacks or denial-of-service attacks, which can overwhelm the service," Sobti stresses.

The vast majority of API-related breaches over the past few years have resulted from poor posture governance, says Nick Rago, field CTO at Salt Security. In many instances, "the barrier to breach was pretty low, and the attacker did not need any herculean effort to take advantage of a misconfigured API."

Rago attributes the problem to a lack of proper oversight over API development and management.

Building a governance framework around the creation of a corporate posture standard is an important first step, he says. To alleviate risks, organizations need to implement capabilities for discovering API assets, assessing their security posture, and remediating noncompliance as needed, Rago adds.

**Badly Designed APIs**

Poorly designed APIs are another major driver of API security incidents; these APIs do everything they are supposed to do, except in a manner that an adversary can take advantage of, Rago says.

"Think of APIs that return more information than an application needs or APIs that can be scraped for information over time," he explains.

Other examples include APIs that use unvalidated SQL inputs, expose implementation details, are too complex and bloated, handle errors in an insecure manner, or have inconsistent naming and structure.

In addition, a poorly designed API can sometimes ignore business logic inconsistencies, Rago says. Examples include ecommerce APIs that allow users to manipulate prices or make modifications that enable overly permissive access to accounts and transactions. Imperva's "State of API Security 2024" report, in fact, identified business logic abuse as last year's top attack on APIs. These attacks accounted for 27% of all API related attacks in 2023, an increase of some 10% over the prior year.

"Both abuse of badly designed APIs or attacks leveraged against a business logic vulnerability can be addressed by leveraging specialized behavioral threat protection that can decipher not just anomalous usage but discern malicious intent behind an API consumer," Rago notes.

As Imperva vice president of API security Lebin Cheng wrote in an op-ed earlier this year, poorly made API design decisions can have a lasting impact on organizations and their customers. APIs, for instance, could cause serious performance bottlenecks if developers fail to consider scalability requirements when designing them. Similarly, by focusing mostly on business needs, developers can often overlook common security issues, such as buffer overflow errors, during design time, Cheng wrote. 

"The issue of poor API design is further compounded by the fact that there are no strict standards for how APIs should be designed," he said. "This leaves it up to individual developers to determine the best way to implement and develop APIs, which means that poor design decisions can easily slip through the cracks."

**Lack of Visibility**

APIs have emerged as a top attack vector for threat actors because of their near ubiquitous use. Imperva's research showed organizations, on average, have 613 API endpoints per account. The security vendor found API traffic to account for 71% of all Web traffic in 2023, with the average enterprise making 1.5 billion API calls per year.

Despite the proliferating use and corresponding risk exposure, many organizations don't have enough visibility over their APIs.

"New methods are required to discover and test APIs," says Kimm Yeo, solutions manager at Black Duck.

Organizations need to start thinking about API security in a more proactive manner, Yeo advocates. That means implementing capabilities to discover and inspect APIs earlier in the software development life cycle, she says. The goal should be to ensure APIs and applications are continuously tested before they get into production.

"Today's API security solutions largely focus on implementing API discovery during production, \[where\] any critical alerts produced are difficult to trace to the code," she says. This can make it impossible for developers to fix identified issues, Yeo adds.

The pressing issue for most organizations is their lack of inventory of all externally facing APIs, says Krishna Vishnubhotla, vice president of product strategy at Zimperium.

"It's critical to act quickly as bad actors are exploiting this gap," he says. "The first step is to urgently discover and inventory all these public APIs, followed by immediate measures to secure them.

**Inadequate Security Testing**

Many organizations are failing to prioritize API security adequately and often underestimate the unique risks APIs pose. Postman's survey found just 37% of organizations currently do automated scanning and regular penetration tests to try and catch API vulnerabilities earlier in the development life cycle. Relatively few have integrated security testing and checks in their API development process or centralized API monitoring capabilities.

Organizations that embrace API-first strategies — where APIs are a priority focus during the software planning, design, architecture, and development process — are seeing better success on the API security front, says Salt Security's Rago.

"Those organizations typically enforce 'spec-first development,' meaning an API must be 'blueprinted' with Swagger or OAS and approved before a line of code is written," he says. "You need to blueprint the hospital first and validate its construction against the plan before you let patients in. Seems obvious, but in most organizations that is still not the way it works."

API risks fall under two big categories: access and availability, Wallarm's Novikov says. Attackers either gain access to something they shouldn't or they can take an API offline by impacting its availability.

"There are lots of technical details about how they might accomplish these objectives, but they all bubble up to these two outcomes," Novikov says.

At a high level, the key protections against these risks are strong authentication and authorization across all API endpoints, he says.

"That means knowing all the APIs you have, which should require authentication, strictly checking authorization on the server side, and implementing advanced rate limiting to slow attackers down," he advises. "These mitigations are best practices, but that doesn't mean they're common practices."

**About the Author**

Jai Vijayan is a seasoned technology reporter with over 20 years of experience in IT trade journalism. He was most recently a Senior Editor at Computerworld, where he covered information security and data privacy issues for the publication. Over the course of his 20-year career at Computerworld, Jai also covered a variety of other technology topics, including big data, Hadoop, Internet of Things, e-voting, and data analytics. Prior to Computerworld, Jai covered technology issues for The Economic Times in Bangalore, India. Jai has a Master's degree in Statistics and lives in Naperville, Ill.

Source

DARKReading