Single sign-on systems from several Big Tech companies are being incorporated into deepfake generators, WIRED found. Discord and Apple have started to terminate some developers’ accounts.

Major technology companies, including Google, Apple, and Discord, have been enabling people to quickly sign up to harmful “undress” websites, which use AI to remove clothes from real photos to make victims appear to be “nude” without their consent. More than a dozen of these deepfake websites have been using login buttons from the tech companies for months.

A WIRED analysis found 16 of the biggest so-called undress and “nudify” websites using the sign-in infrastructure from Google, Apple, Discord, Twitter, Patreon, and Line. This approach allows people to easily create accounts on the deepfake websites—offering them a veneer of credibility—before they pay for credits and generate images.

While bots and websites that create nonconsensual intimate images of women and girls have existed for years, the number has increased with the introduction of generative AI. This kind of “undress” abuse is alarmingly widespread, with teenage boys allegedly creating images of their classmates. Tech companies have been slow to deal with the scale of the issues, critics say, with the websites appearing highly in search results, paid advertisements promoting them on social media, and apps showing up in app stores.

“This is a continuation of a trend that normalizes sexual violence against women and girls by Big Tech,” says Adam Dodge, a lawyer and founder of EndTAB (Ending Technology-Enabled Abuse). “Sign-in APIs are tools of convenience. We should never be making sexual violence an act of convenience,” he says. “We should be putting up walls around the access to these apps, and instead we're giving people a drawbridge.”

The sign-in tools analyzed by WIRED, which are deployed through APIs and common authentication methods, allow people to use existing accounts to join the deepfake websites. Google’s login system appeared on 16 websites, Discord’s appeared on 13, and Apple’s on six. X’s button was on three websites, with Patreon and messaging service Line’s both appearing on the same two websites.

WIRED is not naming the websites, since they enable abuse. Several are part of wider networks and owned by the same individuals or companies. The login systems have been used despite the tech companies broadly having rules that state developers cannot use their services in ways that would enable harm, harassment, or invade people’s privacy.

After being contacted by WIRED, spokespeople for Discord and Apple said they have removed the developer accounts connected to their websites. Google said it will take action against developers when it finds its terms have been violated. Patreon said it prohibits accounts that allow explicit imagery to be created, and Line confirmed it is investigating but said it could not comment on specific websites. X did not reply to a request for comment about the way its systems are being used.

In the hours after Jud Hoffman, Discord vice president of trust and safety, told WIRED it had terminated the websites’ access to its APIs for violating its developer policy, one of the undress websites posted in a Telegram channel that authorization via Discord was “temporarily unavailable” and claimed it was trying to restore access. That undress service did not respond to WIRED’s request for comment about its operations.

**Rapid Expansion**

Since deepfake technology emerged toward the end of 2017, the number of nonconsensual intimate videos and images being created has grown exponentially. While videos are harder to produce, the creation of images using “undress” or “nudify” websites and apps has become commonplace.

“We must be clear that this is not innovation, this is sexual abuse,” says David Chiu, San Francisco’s city attorney, who recently opened a lawsuit against undress and nudify websites and their creators. Chiu says the 16 websites his office’s lawsuit focuses on have had around 200 million visits in the first six months of this year alone. “These websites are engaged in horrific exploitation of women and girls around the globe. These images are used to bully, humiliate, and threaten women and girls,” Chiu alleges.

Harmful 'Nudify' Websites Used Google, Apple, and Discord Sign-On Systems

In addition to its long-standing password spraying attacks, Microsoft says Iran-backed hacker group Peach Sandstorm—or APT 33—has developed custom malware dubbed “Tickler.”

The Iranian government-backed hacking group known as APT 33 has been active for more than 10 years, conducting aggressive espionage operations against a diverse array of public and private sector victims around the world, including critical infrastructure targets. And while the group is particularly known for strategic but technically simple attacks like “password spraying,” it has also dabbled in developing more sophisticated hacking tools, including potentially destructive malware tailored to disrupt industrial control systems. Now, findings from Microsoft released on Wednesday indicate that the group is continuing to evolve its techniques with a new multistage backdoor.

Microsoft Threat Intelligence says that the group, which it calls Peach Sandstorm, has developed custom malware that attackers can use to establish remote access into victim networks. The backdoor, which Microsoft named “Tickler” for some reason, infects a target after the hacking group gains initial access via password spraying or social engineering. Beginning in April and as recently as July, the researchers observed Peach Sandstorm deploying the backdoor against victims in sectors including satellite, communications equipment, and oil and gas. Microsoft also says that the group has used the malware to target federal and state government entities in the United States and the United Arab Emirates.

“The Tickler malware isn’t necessarily a big step up in tactics, techniques, and procedures for this threat actor, but it does represent a clear and active development focus on taking action on objectives," Sherrod DeGrippo, Microsoft's director of threat intelligence, told WIRED in a statement.

The researchers observed Peach Sandstorm deploying Tickler and then manipulating victim Azure cloud infrastructure using the hackers’ Azure subscriptions to gain full control of target systems. Microsoft says that it has notified customers who were impacted by the targeting.

The gang has also continued its low-tech password spraying attacks, according to Microsoft, in which hackers attempt to access many target accounts by guessing leaked or common passwords until one lets them in. Peach Sandstorm has been using this technique to gain access to target systems both to infect them with the Tickler backdoor and for other types of espionage operations. Since February 2023, the researchers say they have observed the hackers “carrying out password spray activity against thousands of organizations.” And in April and May 2024, Microsoft observed Peach Sandstorm using password spraying to target United States and Australian organizations that are in the space, defense, government, and education, sectors.

“Peach Sandstorm also continued conducting password spray attacks against the educational sector for infrastructure procurement and against the satellite, government, and defense sectors as primary targets for intelligence collection,” Microsoft wrote.

The researchers say that, in addition to this activity, the gang has been continuing its social engineering operations on the Microsoft-owned professional social network LinkedIn, which they say date back to at least November 2021 and have continued into mid-2024. Microsoft observed the group setting up LinkedIn profiles that purport to be students, software developers, and talent acquisition managers who are supposedly based in the US and Western Europe.

“Peach Sandstorm primarily used \[these accounts\] to conduct intelligence gathering and possible social engineering against the higher education, satellite sectors, and related industries,” Microsoft wrote. “The identified LinkedIn accounts were subsequently taken down.”

Microsoft's DeGrippo points out that while the new campaigns are noteworthy, Peach Sandstorm has targeted the space industry before.

“This is not the first time Peach Sandstorm has shown interest in satellite-related targeting. This threat actor had \[previously\] pursued organizations in the satellite, defense, and pharmaceutical sectors around the globe,” DeGrippo says. “This backdoor is custom malware with multiple iterations. It shows a focus and commitment to leveraging malware for specific objectives.”

Iranian hackers have been prolific and aggressive on the international stage for years and have shown no signs of slowing down. Earlier this month, reports surfaced that a different Iranian group has been targeting the 2024 US election cycle, including attacks against both the Trump and Harris campaigns.

_Updated at 5:35 pm ET, August 28, 2024: Added comments from Microsoft's director of threat intelligence._

Notorious Iranian Hackers Have Been Targeting the Space Industry With a New Backdoor

French authorities detained Durov to question him as part of a probe into a wide range of alleged violations—including money laundering and CSAM—but it remains unclear if he will face charges.

French prosecutors gave preliminary information in a press release on Monday about the investigation into Telegram CEO Pavel Durov, who was arrested suddenly on Saturday at Paris’ Le Bourget airport. Durov has not yet been charged with any crime, but officials said that he is being held as part of an investigation “against person unnamed” and can be held in police custody until Wednesday.

The investigation began on July 8 and involves wide-ranging charges related to alleged money laundering, violations related to import and export of encryption tools, refusal to cooperate with law enforcement, and “complicity” in drug trafficking, possession and distribution of child pornography, and more.

The investigation was initiated by “Section J3” cybercrime prosecutors and has involved collaboration with France’s Centre for the Fight against Cybercrime (C3N) and Anti-Fraud National Office (ONAF), according to the press release. “It is within this procedural framework in which Pavel Durov was questioned by the investigators,” Paris prosecutor Laure Beccuau wrote in the statement.

Telegram did not respond to multiple requests for comment about the investigation but asserted in a statement posted to the company's news channel on Sunday that Durov has “nothing to hide.”

“Given the existence of several preliminary investigations in France concerning Telegram in relation to the protection of minors' rights and in cooperation with other French investigation units—for instance, on cyber harassment—the arrest of Durov, does not seem to me like a highly exceptional move,” says Cannelle Lavite, a French lawyer who specializes in free-speech matters.

Lavite notes that Durov is a French citizen who was arrested in French territory with an arrest warrant issued by French judges. She adds that the list of charges involved in the investigation is “extensive,” a wide net that she says is not entirely surprising in the context of “France's ambiguous legislative arsenal” meant to balance content moderation and free speech.

Durov is a controversial figure for his leadership of Telegram, in large part because he has not typically cooperated with calls to moderate the platform's content. In some ways, this has positioned him as a free-speech defender against government censorship, but it has also made Telegram a haven for hate speech, criminal activity, and abuse. Additionally, the platform is often billed as a secure communication tool, but much of it is open and accessible by default.

“Telegram is not primarily an encrypted messenger; most people use it almost as a social network, and they’re not using any of its features that have end-to-end encryption,” says John Scott-Railton, senior researcher at Citizen Lab. “The implication there is that Telegram has a wide range of abilities and access to potentially do content moderation and respond to lawful requests. This puts Pavel Durov very much in the center of all kinds of potential governmental pressure.”

On top of all of this, many researchers have questioned whether Telegram's end-to-end encryption is durable when users do elect to enable it.

French president Emmanuel Macron said in a social media post on Monday that “France is deeply committed to freedom of expression and communication … The arrest of the president of Telegram on French soil took place as part of an ongoing judicial investigation. It is in no way a political decision.”

News of Durov's arrest is fueling concerns, though, that the move could threaten Telegram's stability and undermine the platform. The case seems poised, too, to have implications in long-standing debates around the world about social media moderation, government influence, and use of privacy-preserving end-to-end encryption.

Lavite says the case certainly invokes debates about “the balance between the right to encrypted communication and free speech on the one hand, and users' protection—content moderation—on the other hand.” But she notes that there is a lot of information about the investigation that is unknown and “a lot of blurry zones still.”

On Monday afternoon, Telegram seemed to be receiving a download boost from the situation, moving from 18th to 8th place in Apple's US App Store apps ranking. Global iOS downloads were up by 4 percent, and in France the app was number one in the App Store social network category and number three overall.

Telegram CEO Pavel Durov’s Arrest Linked to Sweeping Criminal Investigation

Durov has reportedly been detained in France over Telegram’s alleged failure to adequately moderate illegal content on the messaging app. His arrest sparked backlash and left some associates asking, what now?

“Civil society has had a complicated relationship with Telegram over the years,” says Natalia Krapiva, a lawyer at the digital rights group Access Now. “We have defended Telegram against attempts by authoritarian regimes to block and coerce the platform into providing encryption keys, but we have also been raising alarms about Telegram’s lack of human rights policies, reliable channel of communication, and remedy for its users.” Krapiva stresses that French authorities may try to force Durov to provide Telegram’s encryption keys to decrypt private messages, “which Russia has already tried to do in the past.”

The hashtag #FreePavel has been spreading online, including via X’s CEO, Elon Musk, who has posted numerous times about Durov’s arrest. “POV: It’s 2030 in Europe and you’re being executed for liking a meme,” he wrote on Saturday night in response to a post about the Telegram CEO’s detention. “The need to protect free speech has never been more urgent,” Robert F. Kennedy Jr., who on Friday endorsed Donald Trump for US president, wrote on X, where he referred to Telegram as “uncensored” and “encrypted.”

While Telegram is frequently described as an encrypted messaging app, messages are not end-to-end encrypted by default, and senior executives previously told WIRED that they view the platform as a social network. This is largely due to Channels—a one-to-many broadcast feature that allows unlimited subscribers to view posts.

One of the posts that has gained the most traction on X was by right-wing former Fox News journalist Tucker Carlson, who alluded to the oft-repeated but debatable story that Durov left Russia because the government tried to take over his company. “But in the end, it wasn’t Putin who arrested him for allowing the public to exercise free speech. It was a western country,” Carlson wrote in a post that has so far been viewed at least 5.7 million times. Carlson also linked to an hourlong interview he did with Durov earlier this year, one of the first and only interviews the Telegram CEO has given in recent years.

In Durov’s absence, Telegram’s future looks uncertain to some: “I am in shock, and everyone close to Pavel feels the same,” says Georgy Lobushkin, former head of PR at VK, a social network Durov cofounded, who is still in regular contact with Durov. “Nobody was prepared for this situation.” Asked if he worried about Telegram’s future and who could run the company in Durov’s absence, Lobushkin says: “\[I\] worry a lot.”

TF1Info, which first broke the news in France of Durov’s arrest, reported that it was “beyond doubt” Durov would remain in custody during the investigation. “Pavel Durov will end up in pretrial detention, that's for sure,” one unnamed investigator told reporters.

“No one in Telegram was prepared for such a scenario,” says Anton Rozenberg, who worked with Durov from the early days of VK in 2007, before working for Telegram from 2016 to 2017. Rozenberg foresaw Durov acquiring the best legal defense money could buy. “But without him, the messenger may have huge problems with management, all crucial decisions, and even payments,” he added, given Durov’s personal involvement in running the company. Rozenberg saw no obvious replacement for Durov, who makes key decisions on nearly all matters at Telegram—financing, development strategies, product design, monetization, and content moderation policy.

For now, everything can be expected to continue as normal, says Elies Campo, who directed Telegram’s growth, business, and partnerships from 2015 to 2021. “Depending on how long this is going to last, it’s like a government, right? There’s this structure, there’s self-momentum.” Campo adds that the company’s staff is small enough—around 60 employees—that the infrastructure won’t be affected.

The challenge, Campo concedes, would be if Durov needs to be physically present to pay providers—something Rozenberg also flagged.

“As far as I know, Pavel did the payments,” Campo says. “So what's going to happen when there needs to be some payments for infrastructure providers, or providers in terms of connectivity—and he's still under arrest?”

Pavel Durov’s Arrest Leaves Telegram Hanging in the Balance

Plus: The US intelligence community formally blames Iran for Trump campaign hack, aircraft-tracking platform FlightAware says a “configuration error” exposed sensitive user data, and more.

The United States Defense Department has ideas about a dramatic strategy for defending Taiwan against a Chinese military offensive that would involve deploying an “unmanned hellscape" consisting of thousands of drones buzzing around the island nation. Meanwhile, the US National Institute of Standards and Technology announced a red-team hacking competition this week with the AI ethics nonprofit Humane Intelligence to find flaws and biases in generative AI systems.

WIRED took a closer look at the Telegram channel and website known as Deep State that uses public data and secret intelligence to power its live-tracker map of Ukraine’s evolving front line. Protesters went to Citi Field in New York on Wednesday to raise awareness about the serious privacy risks of deploying facial recognition systems at sporting venues. The technology has increasingly been implemented at stadiums and arenas across the country with little oversight. And Amazon Web Services updated its instructions for how customers should implement authentication in its Application Load Balancer, after researchers found an implementation issue that they say could expose misconfigured web apps.

But wait, there's more! Each week, we round up the security and privacy news we didn’t cover in depth ourselves. Click the headlines to read the full stories. And stay safe out there.

**The US Navy Is Short on Uniform Pants and Won’t Restock For Months**

US Navy officials confirmed to Military.com this week that pants for the standard Navy Working Uniform (NWU) are out of stock at Navy Exchanges and are in perilously low supply across the sea service's distribution channels. The Navy's Exchange Service Command is “experiencing severe shortages of NWU trousers” both in stores and online, according to spokesperson Courtney Williams. Sailors have been noticing out-of-stock notifications online, which state that pants are “not available for purchase in any size.” Williams said that current stock around the world is at 13 percent and that the top priority right now is providing pants to new recruits at Recruit Training Command in Illinois, the Naval Academy Preparatory School in Rhode Island, and the officer training schools.

The shortage seems to have resulted from issues with the Defense Logistics Agency's pants pipeline. Military.com reports that signs currently inside Navy Exchanges say the shortage is “due to Defense Logistics Agency vendor issues.” Williams said the Command has “been in communication with DLA on a timeline for the uniform's production and supply chain.”

Mikia Muhammad, a spokesperson for the Defense Logistics Agency, told Military.com that the first pants restocks are scheduled for October, but these supplies will go to recruits and training programs. She said that Navy exchanges should expect “full support” beginning in January.

**US Government Formally Accuses Iran of Hacking Trump Campaign**

A joint statement on Monday by the FBI, the Office of the Director of National Intelligence, and the Cybersecurity and Infrastructure Security Agency formally accused Iran of conducting a hack-and-leak operation against Donald Trump's presidential campaign. Trump himself had accused Iran in a social media post on August 10, following a report from Microsoft on August 9 about Iranian hackers targeting US political campaigns. The Iranian government denies the accusation.

“The \[Intelligence Community\] is confident that the Iranians have through social engineering and other efforts sought access to individuals with direct access to the presidential campaigns of both political parties,” the US agencies wrote. “Such activity, including thefts and disclosures, are intended to influence the US election process.”

Politico reported on August 10 that Iran had breached the Trump campaign, and an entity calling itself “Robert” had contacted the publication offering alleged stolen documents. The same entity also contacted The New York Times and The Washington Post hawking similar documents.

**FlightAware Says Customer Data Including Social Security Numbers Was “Exposed”**

The popular flight-tracking service FlightAware said this week that a “configuration error” in its systems exposed personal customer data, including names, email addresses, and even some Social Security numbers. The company discovered the exposure on July 25 but said in a breach notification to the attorney general of California that the situation may date as far back as January 2021. The company is mandating that all affected users reset their account passwords.

The company said in its public statement that the exposed data includes “user ID, password, and email address. Depending on the information you provided, the information may also have included your full name, billing address, shipping address, IP address, social media accounts, telephone numbers, year of birth, last four digits of your credit card number, information about aircraft owned, industry, title, pilot status (yes/no), and your account activity (such as flights viewed and comments posted).” It also said in its disclosure to California, “Additionally, our investigation has revealed that your Social Security Number may have been exposed.”

**US Law Enforcement Has Been Leaning on Trove of Encrypted Messages First Obtained by EU Cops**

Since European law enforcement agencies hacked the end-to-end encrypted phone company Sky in 2021, the communications they compromised have been used as evidence in numerous EU investigations and criminal cases. But a review of court records by 404 Media and Court Watch showed this week that US agencies have also been leaning on the trove of roughly half a billion chat messages. US law enforcement has used the data in multiple drug-trafficking prosecutions, particularly to pursue alleged smugglers who transport cocaine with commercial ships and speedboats.

The US Navy Has Run Out of Pants

The Telegram channel and website Deep State uses public data and insider intelligence to power its live tracker of Ukraine’s ever-shifting front line.

Over time, Deep State has added more advanced features and quirks to the map. A toolbar in the bottom-left corner offers the option to enable different layers, including weather patterns, fortifications, and gamma radiation levels in case of nuclear disaster. Users can simulate the effect of different weapons, calculating the range and potential damage of everything from self-propelled howitzers and ballistic missiles to Patriot air defense systems and nuclear explosions. A hidden Easter egg summons an animation of Baby Yoda that, when poked, uses the Force to destroy Russian units.

The map soon became too much for Mykula and Pohorilyi to manage alone; they now enlist the help of more than 100 paid employees and volunteers. Their methods have also evolved. They still use open source intelligence to verify new information, but also acquire data directly from frontline military units whom they’ve developed relationships with. In some cases, the authority of a single source whom they’ve learned to trust is enough, though Mykula admits there have been occasional errors. In other cases, when multiple sources contradict one another, they wait until definitive evidence emerges. Propaganda is rife on both sides, and Mykula insists that Deep State will take no part in it. “We want to win,” he says. “Propaganda will not win.”

Mykula and Pohorilyi do, however, oblige when Ukrainian military commanders request delays to map updates that may compromise their activities. They also receive some government funding for an alternate version of the map available only to verified members of the military. The government funding also goes toward other intelligence activities that Ruslan refuses to discuss; most of their funding comes from public donations.

Late in the first year of the war, Mykula and Pohorilyi learned that their map was helping another, unexpected group of users: Russian soldiers. The map’s designer had added a function that would display instructions to surrender if a user tried to access from a Russian IP address. Then, in October 2022, in an interview with a popular Ukrainian blogger, a Russian POW testified that he had used Deep State’s map for this exact purpose.

The success of Deep State’s map has attracted more users to their original Telegram channel, which now has more than 700,000 subscribers. It publishes its own original reports of the war, all available through a free app, which other established Ukrainian media organizations sometimes refer to. But the map remains the most popular product, used by Ukrainians at home and abroad to track the front line that, at the time of writing, creeps further toward their office in Kyiv every day.

Both Mykula and Pohorilyi approach their work with a stern dedication that belies their youth and inexperience. “We don’t want to disappoint our audience because our projects have become critical for Ukrainians,” Mykula says. “If you compare us to other maps, you will see that Ukrainians don’t go to check on them. They come to us.”

_This story first appeared in the September/October 2024 edition of WIRED UK_.

When War Came to Their Country, They Built a Map

Protesters took to Citi Field Wednesday to raise awareness of the facial recognition systems that have become common at major league sporting venues.

Thousands of people lined up outside Citi Field in Queens, New York, on Wednesday to watch the Mets face off with the Orioles. But outside the ticketing booth, a handful of protesters handed out flyers. They were there to protest a recent Major League Baseball program, one that’s increasingly common in professional sports: using facial recognition on fans.

Facial recognition companies and their customers argue that these systems save time, and therefore money, by shortening lines at stadium entrances. However, skeptics argue that the surveillance tools are never totally secure, make it easier for police to get information about fans, and fuel “mission creep” where surveillance technology becomes more common or even required.

The MLB’s facial recognition program, dubbed Go-Ahead Entry, lets participating fans go on a separate security line, usually shorter than the other queues. Fans download the MLB Ballpark app, submit a selfie, and have their face matched at an in-person camera kiosk at a stadium’s entrance.

Six MLB teams are participating in Go-Ahead Entry, including the Philadelphia Phillies, Cincinnati Reds, Houston Astros, Kansas City Royals, San Francisco Giants, and Washington Nationals.

Some MLB teams, including the Mets, have their own facial recognition programs for express entry. The Mets have been using the facial recognition company Wicket for its Mets Entry Express program since 2021. The Cleveland Guardians, similarly, have been using technology from the company Clear at its ballpark, Progressive Field, since 2019.

Jeff Boehm, Wicket's chief operating officer, tells WIRED in an email that the company believes in “the responsible use of biometric technology to improve the event experience,” which includes taking “data security and privacy very seriously.”

Boehm adds: “As with many new technologies, there is misinformation out there about how the technology is being used. Contrary to some of these claims, Wicket's use is always 100% opt-in (and users can opt-out at any time) and we are not scanning people's faces without their consent. The data is not shared or sold to any third parties.”

Neither the Mets nor MLB immediately responded to WIRED’s requests for comment.

The National Football League has also started using Wicket facial recognition for express entry. NFL spokesperson Brian McCarthy said in an X post that the league-wide program, at least currently, is only available to "team/game-day personnel, vendors, and media”—not fans. The Cleveland Browns and Tennessee Titans, however, do have facial recognition entry systems that fans can use. (The news of the NFL’s expanded use of face recognition still caused confusion on Facebook and X, where some people thought facial recognition would be required at the stadiums for all 32 NFL teams.)

At Citi Field on Wednesday, the Mets Entry Express Line was used scarcely, perhaps five people every five minutes or so. There was never a line. The main security lines, though longer in comparison, took only about five minutes.

A group of privacy advocates stand outside Citi Field on Wednesday to warn fans about the growing use of facial recognition systems at sports events.Photograph: Caroline Haskins

Stadiums Are Embracing Face Recognition. Privacy Advocates Say They Should Stick to Sports

The AI ethics nonprofit Humane Intelligence and the US National Institute of Standards and Technology are launching a series of contests to get more people probing for problems in generative AI systems.

At the 2023 Defcon hacker conference in Las Vegas, prominent AI tech companies partnered with algorithmic integrity and transparency groups to sic thousands of attendees on generative AI platforms and find weaknesses in these critical systems. This “red-teaming” exercise, which also had support from the US government, took a step in opening these increasingly influential yet opaque systems to scrutiny. Now, the ethical AI and algorithmic assessment nonprofit Humane Intelligence is taking this model one step further. On Wednesday, the group announced a call for participation with the US National Institute of Standards and Technology, inviting any US resident to participate in the qualifying round of a nationwide red-teaming effort to evaluate AI office productivity software.

The qualifier will take place online and is open to both developers and anyone in the general public as part of NIST's AI challenges, known as Assessing Risks and Impacts of AI, or ARIA. Participants who pass through the qualifying round will take part in an in-person red-teaming event at the end of October at the Conference on Applied Machine Learning in Information Security (CAMLIS) in Virginia. The goal is to expand capabilities for conducting rigorous testing of the security, resilience, and ethics of generative AI technologies.

“The average person utilizing one of these models doesn’t really have the ability to determine whether or not the model is fit for purpose,” says Theo Skeadas, chief of staff at Humane Intelligence. “So we want to democratize the ability to conduct evaluations and make sure everyone using these models can assess for themselves whether or not the model is meeting their needs.”

The final event at CAMLIS will split the participants into a red team trying to attack the AI systems and a blue team working on defense. Participants will use the AI 600-1 profile, part of NIST's AI risk management framework, as a rubric for measuring whether the red team is able to produce outcomes that violate the systems' expected behavior.

“NIST's ARIA is drawing on structured user feedback to understand real-world applications of AI models,” says Humane Intelligence founder Rumman Chowdhury, who is also a contractor in NIST's Office of Emerging Technologies and a member of the US Department of Homeland Security AI safety and security board. “The ARIA team is mostly experts on sociotechnical test and evaluation, and \[is\] using that background as a way of evolving the field toward rigorous scientific evaluation of generative AI.”

Chowdhury and Skeadas say the NIST partnership is just one of a series of AI red team collaborations that Humane Intelligence will announce in the coming weeks with US government agencies, international governments, and NGOs. The effort aims to make it much more common for the companies and organizations that develop what are now black-box algorithms to offer transparency and accountability through mechanisms like “bias bounty challenges,” where individuals can be rewarded for finding problems and inequities in AI models.

“The community should be broader than programmers,” Skeadas says. “Policymakers, journalists, civil society, and nontechnical people should all be involved in the process of testing and evaluating of these systems. And we need to make sure that less represented groups like individuals who speak minority languages or are from nonmajority cultures and perspectives are able to participate in this process.”

The US Government Wants You—Yes, You—to Hunt Down Generative AI Flaws

Amazon has updated its instructions for how customers should more securely implement AWS's traffic-routing service known as Application Load Balancer, but it's not clear everyone will get the memo.

A vulnerability related to Amazon Web Service's traffic-routing service known as Application Load Balancer could have been exploited by an attacker to bypass access controls and compromise web applications, according to new research. The flaw stems from a customer implementation issue, meaning it isn't caused by a software bug. Instead, the exposure was introduced by the way AWS users set up authentication with Application Load Balancer.

Implementation issues are a crucial component of cloud security in the same way that the contents of an armored safe aren't protected if the door is left ajar. Researchers from the security firm Miggo found that, depending on how Application Load Balancer authentication was set up, an attacker could potentially manipulate its handoff to a third-party corporate authentication service to access the target web application and view or exfiltrate data.

The researchers say that looking at publicly reachable web applications, they have identified more than 15,000 that appear to have vulnerable configurations. AWS disputes this estimate, though, and says that “a small fraction of a percent of AWS customers have applications potentially misconfigured in this way, significantly fewer than the researchers' estimate.” The company also says that it has contacted each customer on its shorter list to recommend a more secure implementation. AWS does not have access or visibility into its clients' cloud environments, though, so any exact number is just an estimate.

The Miggo researchers say they came across the problem while working with a client. This “was discovered in real-life production environments,” Miggo CEO Daniel Shechter says. “We observed a weird behavior in a customer system—the validation process seemed like it was only being done partially, like there was something missing. This really shows how deep the interdependencies go between the customer and the vendor.”

To exploit the implementation issue, an attacker would set up an AWS account and an Application Load Balancer, and then sign their own authentication token as usual. Next, the attacker would make configuration changes so it would appear their target's authentication service issued the token. Then the attacker would have AWS sign the token as if it had legitimately originated from the target's system and use it to access the target application. The attack must specifically target a misconfigured application that is publicly accessible or that the attacker already has access to, but would allow them to escalate their privileges in the system.

Amazon Web Services says that the company does not view token forging as a vulnerability in Application Load Balancer because it is essentially an expected outcome of choosing to configure authentication in a particular way. But after the Miggo researchers first disclosed their findings to AWS at the beginning of April, the company made two documentation changes geared at updating their implementation recommendations for Application Load Balancer authentication. One, from May 1, included guidance to add validation before Application Load Balancer will sign tokens. And on July 19, the company also added an explicit recommendation that users set their systems to receive traffic from only their own Application Load Balancer using a feature called “security groups.”

AWS spokesperson Patrick Neighorn tells WIRED in a statement: “It is incorrect to call this an authentication and authorization bypass of AWS Application Load Balancer (ALB) or any other AWS service because the technique relies on a bad actor already having direct connectivity to a misconfigured customer application that does not authenticate requests. We recommend customers configure their applications to only accept requests from their ALB by using security groups and by following the ALB security best practices.”

Taken together, the two changes AWS made effectively address the attack path the Miggo researchers proposed. But since they involve changing how AWS customers have set up their own systems, the fixes aren't like a software patch that a developer can push out to all users. Instead, AWS users with vulnerable configurations must hear about the updated guidance, realize it is relevant to their configurations, and make the changes themselves.

Under what's known as the Shared Responsibility Model, such situations often sit in the gray area between what a cloud platform provider should address for its customers and what users need to be in charge of managing themselves. And while this ambiguity has existed for years, there are some cases where there is still no obvious single solution for making sure that every cloud customer has the exact security setup they need and intend.

An AWS Configuration Issue Could Expose Thousands of Web Apps

The US Defense Department’s grand strategy for protecting Taiwan from a massive Chinese military offensive involves flooding the zone with thousands of drones.

It has become conventional wisdom among the halls of the United States government that China will launch a full-scale invasion of Taiwan within the next few years. And when that happens, the US military has a relatively straightforward response in mind: Unleash hell.

Speaking to The Washington Post on the sidelines of the International Institute for Strategic Studies’ annual Shangri-La Dialogue in June, US Indo-Pacific Command chief Navy Admiral Samuel Paparo colorfully described the US military’s contingency plan for a Chinese invasion of Taiwan as flooding the narrow Taiwan Strait between the two countries with swarms of thousands upon thousands of drones, by land, sea, and air, to delay a Chinese attack enough for the US and its allies to muster additional military assets in the region.

“I want to turn the Taiwan Strait into an unmanned hellscape using a number of classified capabilities,” Paparo said, “so that I can make their lives utterly miserable for a month, which buys me the time for the rest of everything.”

Cheap, easily weaponizable drones have transformed battlefields from Ukraine to the Middle East in recent years, and the US military is rapidly adapting to this new uncrewed future. While Paparo isn’t the first to invoke the image of a robotic “hellscape” with regards to Taiwan (his predecessor, Admiral John Aquilino had previously used the term in August 2023), his comments offer the most vivid description of the Defense Department’s plan for dealing with Chinese aggression toward the US ally. In recent months, new details have started to draw out the contours of what, exactly, this “hellscape” would look like.

**A Great Wall of Drones**

China has undertaken a major military buildup ahead of what American defense leaders see as an imminent attempt to annex Taiwan, which Beijing sees as a breakaway province. According to a June analysis from the Center for Strategic & International Studies, the People’s Liberation Army Navy now boasts the largest maritime force on the planet, with 234 warships to the US Navy’s fleet of 219; testifying before the Senate Armed Services Committee in March, then-INDOPACOM boss Aquilino stated that the PLA Air Force also now has the largest number of warplanes in the region (not counting uncrewed systems), with designs on soon surpassing the US and Russian air fleets.

While the exact size of the Chinese military’s arsenal of uncrewed vehicles is difficult to estimate, the country has become the leading exporter of armed combat drones around the world over the past decade (along with Turkey), according to data from the Stockholm International Peace Research Institute. And when it comes to the consumer drones that are converted into weapons of war by soldiers on the front lines, Chinese drone giant Da-Jiang Innovations (better known in the US as DJI) controls three-quarters of that market, according to The Wall Street Journal. While the US and China may appear locked in an military drone arms race, the latter currently possess a significant advantage.

“China has essentially copied all of the large and medium high-altitude drones the US has and produced what amount to cheaper versions of the MQ-9 Reaper or the \[RQ-4\] Global Hawk,” Stacie Pettyjohn tells WIRED. A senior fellow and director of defense programs at the Center for a New American Security, Pettyjohn is the lead author of the June report “Swarms Over the Strait” on the role of drones in a future conflict over Taiwan. “Potentially more concerning is the smaller drones that don’t have to fly as far and can be launched from mainland China, of which the Chinese military has many.”

Simply put, China has a lot of drones and can make a lot more drones quickly, creating a likely advantage during a protracted conflict. “This stands in contrast to American and Taiwanese forces, who do not have large inventories of drones or the right mix of drones to successfully defeat a Chinese invasion,” Pettyjohn and her coauthors write in the CNAS report.

Apart from beefing up Taiwan’s counter-drone defenses, the Pentagon’s “hellscape” plan proposes that the US military make up for this growing gap by producing and deploying what amounts to a massive screen of autonomous drone swarms designed to confound enemy aircraft, provide guidance and targeting to allied missiles, knock out surface warships and landing craft, and generally create enough chaos to blunt (if not fully halt) a Chinese push across the Taiwan Strait. Networked drones will not just strike adversaries but also provide critical intelligence, surveillance, and reconnaissance functions to fill the gaps between satellite imaging and crewed overflights, ostensibly allowing the US and its allies to develop a more complete picture of the battlefield as it evolves.

“A 2020 Rand Corporation report concludes that hundreds of networked, low-cost drones could guide American long-range anti-ship missiles toward a Chinese invasion fleet and would be a key capability needed to defeat China and successfully defend Taiwan,” the CNAS report says. “Similarly, proponents of air denial strategy argue that using ‘sufficiently large numbers of smaller, cheaper weapons,’ including ground-based air defenses and drone swarms, ‘in a distributed way’ would prevent China from gaining air superiority.”

As Pettyjohn points out to WIRED, the concept of Taiwan using densely layered defenses to inflict incredibly high losses on an invading Chinese force isn’t new, with past visions of a “porcupine strategy” built on missiles and mines as deterrents to an outright invasion. But the incorporation of massive drone fleets adds a new layer to the fight. Indeed, war games conducted by the US Air Force and defense think tanks like Rand over the past several years have pointed to the critical role drone swarms could play in potentially thwarting an invasion of Taiwan.

According to the CNAS report, this strategy appears to have been sharpened by recent lessons from Russia’s invasion of Ukraine, where the Ukrainian military has successfully deployed drones against the superiorly numbered and equipped Russian force to disrupt enemy formations, destroy armored vehicles, and even neutralize surface combatants. Look no further than the Black Sea, where Ukrainian forces have managed to destroy 26 Russian vessels and forced Moscow’s vaunted Black Sea Fleet to a safe harbor hundreds of miles away using missiles, kamikaze UAVs, and explosive-laden drone boats.

The CNAS report makes several recommendations for how the Pentagon can best employ drones to defend Taiwan, emphasizing the need to build a “diverse” fleet of UAVs encompassing “a mix of higher-end and cheaper systems” (the large and expensive Reaper versus low-cost single-use kamikaze drones, for example), investing in the development of autonomous drone boats for attacking larger surface warships, and pre-positioning short- and medium-range drones on Taiwan for a rapid, immediate response to a Chinese invasion.

“In addition to acquiring ‘good enough’ long-range drones for target acquisition and strike, the United States should have a smaller number of stealthy drones that can conduct surveillance in highly contested airspace and provide targeting information for standoff missile strikes,” the report says, adding that “affordable kamikaze drones with relatively simple autonomy could overwhelm the Chinese navy’s air defenses and damage or destroy the invasion fleet.”

**Engage the Replicator**

With a potential invasion looming, the Pentagon has kicked efforts to make its vision of a “hellscape” into high gear. Last August, deputy secretary of defense Kathleen Hicks announced the department’s new Replicator initiative, designed to build and field “attritable autonomous systems”—DOD-speak for disposable, AI-enabled drones—“at scale of multiple thousands, in multiple domains” within the next 18 to 24 months. As of March, the Pentagon had earmarked $1 billion across its fiscal-year 2024 and 2025 budgets for its first round of Replicator systems, as USNI News reported.

Replicator appears to be doing its job already. In May, the Pentagon announced its first tranche of fresh capabilities, which includes the accelerated fielding of more than 1,000 of defense contractor AeroVironment’s Switchblade-600 loitering munitions—a man-portable missile that circles over targets before dive-bombing them at the right moment—in the next year, as well as the procurement of uncrewed “interceptor” surface vessels under the department’s new Production-Ready, Inexpensive, Maritime Expeditionary (Prime) effort. According to a DOD solicitation released in January, the Prime drone boats will purportedly be capable of “autonomously transiting hundreds of miles through contested waterspace, loitering in an assigned operating area while monitoring for maritime surface threats, and then sprinting to interdict a noncooperative, maneuvering vessel.” The first Replicator systems were already deployed to the Indo-Pacific, according to Hicks, with some military units training with cheap drones produced under the initiative as of August.

"This is just the beginning," Admiral Christopher Grady, vice chairman of the Joint Chiefs of Staff, said in a May statement. "Replicator is helping us jump-start the delivery of critical capabilities at scale.

Replicator isn’t the US military’s only effort to incorporate uncrewed weapons platforms into its formations. The Army has asked for $120.6 million as part of its fiscal-year 2025 budget request for Low Altitude Stalking and Strike Ordnance (Lasso) semiautonomous loitering munitions to outfit infantry brigade combat teams with the capability. As of April, the Marine Corps has selected three defense contractors (AeroVironment, defense upstart Anduril, and Teledyne FLIR) to compete for a potential $249 million contract to furnish Marines with so-called Organic Precision Fires-Light kamikaze drone swarms (to say nothing of the Corps’ push to acquire Long Range Attack Missile air-launched loitering munition). US Special Operations Command, an early adopter of loitering munitions, now wants to outfit its fleet of aircraft with air-launched systems. And as for the maritime realm, the Marine Corps has been experimenting with uncrewed surface vessels bristling with Uvision Hero-120 kamikaze drone launchers, while the Navy has been eyeing missile-hauling drone boats as potential escorts for transport ships, among other lethal initiatives, after years of experimenting with uncrewed surface vessels as waterborne sensor nodes.

Beyond expanding its arsenal of uncrewed systems, the US is also working to bolster Taiwan’s own drone capabilities. In June, the State Department announced the approval of a $360 million weapons sale to Taipei that included 291 ALTIUS 600M-V kamikaze drones produced by Anduril and 720 Switchblade-300 loitering munitions. As the CNAS report notes, the integration of these one-way attack drones into Taiwanese military formations, when deployed in conjunction with explosive-laden drone boats and anti-ship missiles, could potentially prevent Chinese warships from ever reaching the country’s shores in a similar manner to how Ukrainian forces have denied the Russian military control over the Black Sea.

“Taiwan needs a lot of these systems and needs them quickly to incorporate them into broader tactics and formations” as effectively as the Ukrainians have, Pettyjohn says.

And this is just the beginning: The Taiwanese government plans to procure nearly 1,000 additional AI-enabled attack drones in the next year, according to Taipei Times, with long-standing plans to expand indigenous production of homegrown capabilities to prevent backlogs in weapons transfers from the United States—and, more importantly, ease reliance on Chinese-made commercial off-the-shelf parts. (Although, as Pettyjohn points out, the Taiwanese defense community itself isn’t totally unified around the “hellscape” plan in the first place.)

Access to commercial drones “is where Taiwan is most disadvantaged” because of DJI’s relative dominance of the market, Pettyjohn says, noting that “even if Taiwan had Chinese drones available to them, they would have to hack in to each system to ensure they can’t be tracked by DJI or don’t have similar vulnerabilities.”

“Consider that for most of the first-person-view kamikaze drones used in Ukraine right now, all of those components are sourced from China,” she adds. “Even Ukraine has tried to wean itself off Chinese sources and hasn’t found anything at a comparable price point.”

**Mass-Producing Hell**

Planning a “hellscape" of hundreds of thousands of drones is one thing, but actually making it a reality is another. An April 2023 assessment from the Rand Corporation indicated that rising demand for weaponized drones would likely “strain” the capacity of the existing US defense industrial base. Similarly, a separate CNAS report from June 2023 argued that the war in Ukraine (and the US government’s role as a major provider of security assistance to Kyiv) has “shed light on serious deficiencies” in the Pentagon’s ability to rapidly scale production of “key weapons” like precision-guided munitions compared to Russia—a problem echoed in the most recent CNAS report’s assessment of the US government’s approach to Taiwan’s defense.

“Ukraine consistently has pioneered new approaches to drone warfare, but Russia has rapidly adapted and scaled drone production in a way that Ukraine cannot match,” the June 2024 CNAS report says. “Technological and tactical innovations are necessary but not sufficient. Mass production of an affordable mix of drones is also needed to support a large and likely protracted conflict.”

The report adds that the US defense industrial based may not be “ currently capable of producing the quantities of drones needed for a war with China.”

Like Russia, China’s autocratic regime has enabled the country’s defense industrial base to rapidly accelerate weapons R&D and production, so far that Beijing is “heavily investing in munitions and acquiring high-end weapons systems and equipment five to six times faster than the United States,” as a March comparison from CSIS put it. By contrast, the US defense industrial ecosystem has over the past several decades consolidated into a handful of large “prime” contractors like Lockheed Martin and Raytheon, a development that threatens to not only stifle innovation but hamstring the production of critical systems needed for the next big war.

“Overall, the US defense industrial ecosystem lacks the capacity, responsiveness, flexibility, and surge capability to meet the US military’s production and war-fighting needs,” the CSIS report says. “Unless there are urgent changes, the United States risks weakening deterrence and undermining its war-fighting capabilities.”

To that end, the latest CNAS report recommends that the Pentagon and Congress work to foster both the commercial and military drone industrial base “to scale production and create surge capacity” to quickly replace drones lost in a future conflict. While the Pentagon has, with regards to Ukraine, relied on multi-year and large-lot procurement programs to source munitions from large “primes” and “\[provide\] industry with the stability it needs to expand production capacity,” as the 2023 CNAS report put it, the Replicator initiative is explicitly designed to not only further provide that stability to drone makers but also to pull in “nontraditional” defense industry players—startups like Anduril or drone boat maker Saronic, the latter of which recently received $175 million in Series B funding to scale up its manufacturing capacity.

Replicator “provides the commercial sector with a demand signal that allows companies to make investments in building capacity, strengthening both the supply chain and the industrial base,” according to the Defense Innovation Unit, the Pentagon organ responsible for capitalizing on emerging commercial technologies. “Replicator investments incentivize traditional and non-traditional industry players to deliver record volumes of all domain attritable autonomous systems in line with the ambitious schedule set forth by the deputy secretary of defense.”

“It comes down to contracts,” Pettyjohn says. “Where Replicator is potentially most impactful is where the Pentagon buys something they keep for a few years before they get something new for a different mission set so the DOD isn’t keeping a system in their inventory for decades. Establishing those practices, getting those contracts out there, and getting enough money into it so there’s competition and resiliency within industry is really needed to fuel innovation and provide the capabilities that are needed.”

It’s unclear whether the United States will actually be ready to defend Taiwan when the moment arrives; as legendary Prussian military commander Helmuth von Moltke is famously quoted as saying, “no plan survives first contact with the enemy.” But with the right preparation, funding, and training (and a little luck), the Pentagon and its Taiwanese partners may end up successfully throwing a wrench in China’s suspected invasion plans by flooding the zone with lethal drones. War is hell, but when the next big conflict in the Indo-Pacific rolls around, the US wants to guarantee that it will be an absolute hellscape—for the Chinese military, at least.

Source

Wired