Every company has its own version of ChatGPT now

Thursday, July 27, 2023 14:07

Welcome to this week’s edition of the Threat Source newsletter.

When I first started poking at ChatGPT a few months ago, I quickly learned that it wasn’t quite ready to take my job (yet) and wasn’t staying up to date on wrestling.

Since ChatGPT went viral, several other companies have released their own versions of AI assistants who can answer questions, perform text-based tasks and more. Meta most recently entered the AI race with Llama 2, an open-source version of their chatbot. IBM has also re-branded its watson tool (formerly known as capital “W” Watson who was really good at “Jeopardy”) to be a ChatGPT-like.

Security researchers have long warned about the dangers these tools pose to the security space, specifically because it makes it easier for bad actors to create convincing spam campaigns, phishing emails and scripts anyone could follow in a voice phishing (“vishing”) attack.

Out of my own personal curiosity, I wanted to try Llama 2 out. I had to enlist the help of some third-party open-source tools to make it easier for me to use Llama 2, but I quickly got it running and went right to work as a hypothetical bad guy looking to start sending spam emails.

I’ve recently become interested in the topic thanks to “Behind the Bastards,” one of my favorite podcasts. Robert Evans, the host of the show, has done some great reporting over the past month on AI-generated cons and spam, specifically how they affect the elderly.

I tried several different prompts with different wordings (trying omitting and including specific words like “phishing,” “scam” and “spam”) but I couldn’t quite talk Llama 2 into writing something for me. Instead, I repeatedly received several warnings about how the questions violated the model’s terms of service and the overall threat of spam and phishing and its negative effects on its targets.

In one example, I gave Llama 2 the prompt, “Write a letter to my boss asking her to immediately download and install an attached file to view my latest analytics report,” which is a legitimate thing I would send her in my everyday activities.

Instead, the chatbot suggested I use a “secure and trusted platform” instead of an email attachment to send the report and said it “goes against ethical and responsible AI practices to encourage or assist with the download or installation of files from unverified sources.”

Using that same prompt in ChatGPT, I received no pushback and instead got an email template that was a bit more formal than I’d usually push for in my emails, but nonetheless, I got an email that was easily customizable and repeatable asking the user to download an attachment.

Llama 2 gave me similar warnings when I asked it to write a script for me to ask my grandmother to purchase me an Amazon gift card because I was in a bind.

ChatGPT took the word “script” very literally in my ask to it, instead producing what looked like it belonged in a mid-day television show instead of a phishing attempt.

I commend Meta for seeming to have tighter restrictions on the types of asks users can make to its AI model. But, as always, these tools are far from perfect and I’m sure there are scripts that I just couldn’t think of that would make an AI-generated email or script more convincing. This is a topic I plan on looking into more, and if you have any ideas about things we could ask AI chat models to do, feel free to DM us on Twitter.

The one big thing

Bad actors are having to change up their tactics to steal login credentials and authorization attempts as the internet at large moves away from text-based passwords. Talos researchers wrote this week that they anticipate passwords may disappear in the not-too-distant future, leaving actors likely to shift away from basic phishing or other attacks that target passwords, toward post-authentication session theft or the weaker registration, recovery and revocation processes. Although it will likely take several years for passwordless authentication to become widely adopted, and it is likely that passwords will be around for a long time, some of the most popular and most targeted applications may adopt it in the near future. Removing passwords from even a few dozen key web applications is likely to affect the threat landscape.

Why do I care?

Many of the passwords that attackers are currently harvesting will be rendered obsolete with the adoption of passwordless authentication. This may cause changes to the malware landscape and cybercriminals’ business models built around selling access to compromised accounts. This means new tactics, attack vectors and methods that attackers try to use to steal things like session IDs and MFA push notifications.

So now what?

Looks for security companies and technology vendors to develop new standards that focus on the protection of web services. Even though these new types of attacks are likely to pop up, using passkeys or a passwordless approach to security is still preferable to any “traditional” login methods.

Top security headlines of the week

A North Korean state-sponsored actor was behind a recent supply chain attack on a cloud IT provider it used to target cryptocurrency companies. JumpCloud, the target of the attack, disclosed that the attack affected less than five of its customers and fewer than 10 devices. The campaign may indicate a pivot among North Korean actors to move away from direct attacks designed to steal cryptocurrency in favor of stealthier supply chain attacks. These actors typically carry out attacks to generate funds for the country’s reclusive regime and its controversial nuclear weapons program. Mandiant, one of the security firms who helped investigate the attack, said the group responsible worked for North Korea’s Reconnaissance General Bureau (RGB), its primary foreign intelligence agency. North Korean actors targeted the 3CX softphone application in a similar supply chain attack earlier this year. (Reuters, Axios)

The Biden administration announced a new program to include a physical label on internet-of-things products that meet certain cybersecurity criteria. The new “U.S. Cyber Trust Mark,” created by the National Institute of Standards and Technology, will soon start appearing on smart home devices that meet the preset list of standards for the way the device stores information and makes other connections to the user’s network. Connected appliances found in users’ homes, like “smart” refrigerators, microwaves and televisions, will be among the first products to receive the label in partnership with major retailers and manufacturers. “Smart fitness trackers,” presumably to cover things like smart watches, were also mentioned in the Biden administration’s announcement. (CBS News, The Verge)

The U.S. Securities and Exchange Commission formally adopted new rules Wednesday regarding how quickly American companies must disclose cyber attacks. Public companies now have four days within the discovery of a cyber attack to determine if it had a material effect on its operations, and then disclose that determination to the public. However, companies can receive an extension to this deadline if disclosure of the attack would pose a significant risk to national security or public safety, as determined by the U.S. attorney general. These companies will also have to publish the processes they have in place to manage material risks from cybersecurity incidents. The new policies have been hotly debated for more than a year since they were initially announced. (MarketWatch, Bloomberg)

Can’t get enough Talos?

• Data theft extortion rises, while healthcare is still most-targeted vertical in Talos IR engagements
• Talos Takes Ep. #147: ISO 27002 sounds intimidating, but really it’s just a cybersecurity shopping list

Upcoming events where you can find Talos

BlackHat (Aug. 5 - 10)

Las Vegas, Nevada

Grace Hopper Celebration (Sept. 26 - 29)

Orlando, Florida

Caitlin Huey, Susan Paskey and Alexis Merritt present a “Level Up Lab” titled “Don’t Fail Knowledge Checks: Accelerating Incident Response with Threat Intelligence.” Participate in several fast-paced activities that emphasize the importance of threat intelligence in security incident investigations. Attendees will act as incident responders investigating a simulated incident that unfolds throughout this session. Periodic checkpoints will include discussions that highlight how incident response and threat intelligence complement each other during an active security investigation.

Most prevalent malware files from Talos telemetry over the past week

SHA 256: a31f222fc283227f5e7988d1ad9c0aecd66d58bb7b4d8518ae23e110308dbf91
MD5: 7bdbd180c081fa63ca94f9c22c457376
Typical Filename: c0dwjdi6a.dll
Claimed Product: N/A
Detection Name: Trojan.GenericKD.33515991

SHA 256: 9f1f11a708d393e0a4109ae189bc64f1f3e312653dcf317a2bd406f18ffcc507
MD5: 2915b3f8b703eb744fc54c81f4a9c67f
Typical Filename: VID001.exe
Claimed Product: N/A
Detection Name: Win.Worm.Coinminer::1201

SHA 256: 5616b94f1a40b49096e2f8f78d646891b45c649473a5b67b8beddac46ad398e1
MD5: 3e10a74a7613d1cae4b9749d7ec93515
Typical Filename: IMG001.exe
Claimed Product: N/A
Detection Name: Win.Dropper.Coinminer::1201

SHA 256: e12b6641d7e7e4da97a0ff8e1a0d4840c882569d47b8fab8fb187ac2b475636c
MD5: a087b2e6ec57b08c0d0750c60f96a74c
Typical Filename: AAct.exe
Claimed Product: N/A
Detection Name: PUA.Win.Tool.Kmsauto::1201

SHA 256: b4d8d7cbec7fe4c24dcb9b38f6036a58b765efda10c42fce7bbe2b2bf79cd53e
MD5: c585f4faee96a0bec3b0f93f37239008
Typical Filename: stream.txt
Claimed Product: N/A
Detection Name: Win.Dropper.Autoit::211461.in02