Urlscan.io API Inadvertently Leaked Sensitive Data and URLs

Berlin-based digital security firm Positive Security’s latest blog post revealed how Urlscan API has been leaking sensitive URLs and data accidentally. The company was alerted by an email from GitHub in February 2022.

It is suspected that GitHub Pages URLs were leaked accidentally through a third-party while conducting metadata assessment.

What is Urlscan.io?

Urlscan.io is a website scanning and analyzing engine that accepts URL submissions and creates a trove of data such as IPs, domains, DOM information, screenshots, and cookies.

According to Urlscan.io developers, the engine is like a sandbox for the web. It was created to let users “easily and confidently” assess potentially malicious websites. The engine supports countless enterprise customers and open-source projects. It comes with an API that integrates its scans into third-party products.

What’s the Issue?

Given the API’s integration type, such as a security tool that scans all incoming emails and conducts a Urlscan on all links, there could be a vast amount and variety of sensitive data that an anonymous user can easily search for and retrieve.

An investigation was launched after GitHub’s notification regarding users sharing their private repository names and usernames to Urlscan.io for metadata analysis automatically.

When pingbacks to leaked email IDs were checked, researchers detected that misconfigured security tools submitting links received in emails as public scans were the root cause of the problem. For instance, some API integrations use generic python-requests/2.X.Y. If user agents ignored account visibility settings, it allowed scans to be mistakenly submitted as public.

Extent of Impact

Further probe revealed that the issue could be affecting urlscan.io dorks, setup pages, password reset links, Telegram bots, meeting invitations, DocuSign signing requests, shared Google Drive links, PayPal, SharePoint, Discord, Zoom invoices, Cisco Webex meeting recordings, Dropbox file transfers package tracking links, and PayPal invoices, said Fabian Bräunlein, Positive Security co-founder in the report.

It is worth noting that the company also found juicy URLs belonging to Apple domains, and some contained publicly-shared links to iCloud files. However, these have now been removed.

For your information, when it comes to SEO, a “juicy” URL is one that is packed with keywords and phrases that help improve your ranking in search engine results pages.

Positive Security got in touch with most of the leaked email addresses and received a response from an unnamed organization. It traced the leak of a DocuSign work contract link to its misconfigured Security Orchestration, Automation, and Response (SOAR) solution that Urlscan.io integrated.

Threat Mitigation Efforts

After completing the issue assessment in July, Positive Security notified Urlscan.io about its findings and collaborated with its developers to address the issue. A new engine version was released later that month with an enhanced scan visibility interface and team-wide visibility settings.

The company also published its Scan Visibility Best Practices to explain the risks associated with the three visibility settings users selected when submitting a URL- Public, Unlisted, and Private. After reviewing third-party SOAR tool integrations, the developers added deletion rules and enhanced visibility settings in the user interface. They also included a report button for the deactivation of problematic search results.

1. Hackers can access trove of stolen credentials on VirusTotal
2. Leaked Amazon Prime Video Server Exposed Users Viewing Habits
3. Sensitive data of cyber security firm & other businesses leaked online
4. Personal details of 21M SuperVPN, GeckoVPN users leaked on Telegram
5. Man Accidentally Destroyed Production Database on First Day of His Job