Not All URL Scanners Are Created Equal – CheckPhish vs UrlScan.io vs ScamAdviser
Got a URL looking for Phishy?
Phishing, Spear Phishing, and Business Email Compromise (BEC) … however you identify the threat, the concern is always the same. As a security analyst, how do you know for sure that the link (s) in a suspected phishing email is indeed malicious?
There are several tactics you can employ. Conventional techniques include examining the MIME header of the email, decoding encoded URLs, or hovering over the link in the URL. All of these, however, require a certain level of specialist knowledge to be effective. Not only that, it takes time. Depending on the number of emails you scan, this can take hours each day.
There is a simpler alternative, where you copy the URL link from the email (without clicking on it!) And paste it into a URL scanner. In this blog, we’ll take a look at three popular URL analysis tools that will inspect a URL and determine if it’s safe:
We will review the pros and cons of these tools to assess their accuracy and usefulness. And along the way, we’ll look at two different approaches to inspecting a suspicious URL: 1) A simple scan to understand if it’s malicious or not, and 2) A review of related threat intelligence to understand the context of a verdict.
For the comparison, we’ll use a hypothetical scenario to walk through the two use cases. In the scenario, a user reported an email sent by Replit.com (an online browser-based IDE). The user reports that they don’t usually get emails like this and when they hover over the links things look like hooks. (see below).
Figure 1. Example of a suspicious URL delivered via Replit.
Real-time URL analysis
Although I agree this url looks odd and the user said it was unusual to receive an email from them hard to be sure. By copying the link (right click> copy link address) and pasting it into CheckPhish.ai, we get a verdict in seconds. It’s clean!
But how does CheckPhish.ai know it’s clean? Unlike other URL scanners, the tool outputs and analyzes the URL in real time. The backend of CheckPhish.ai is the same as the award-winning Bolster enterprise platform used by companies like Zoom, LinkedIn, and Dropbox. When scanning a URL, the tool launches a headless browser to display the site. From there, it uses computer vision to identify logos and brands, then combines it with natural language processing to figure out whether the intent of the site is malicious or not. What’s unique about CheckPhish is that it’s not just an aggregation of open source threat streams available for free. This is a real-time expert analysis of the site with a false positive rate of 1 / 100,000.
Figure 2. Check the phishing exit
CheckPhish is also able to handle scenarios that other scanners seem unable to handle. For example, here we can see a “source” URL and a “redirect” URL on the left side, near the top. The redirected URL is the official domain of replit.com and the path appears to lead to a careers page. In this case, the reported email is legitimate and poses no threat. CheckPhish.ai can access the redirected URL and complete the scan. Not all of the scanners I tested were able to do this.
Performing the same series of steps on ScamAdviser.com appears to lead to failure. It is apparently unable to handle redirected URLs, which is what we are working with in this case. For this reason, ScamAdviser.com is unable to assist you in this investigation.
Figure 3. ScamAdviser exit
Using urlscan.io we may run a successful scan on the URL but we are getting erroneous results as reported by Google Safe Browsing Test (Malicious). Urlscan.io aggregates a large amount of threat feeds, and it appears that Google Safe Browsing has at some point classified Repl.it as a malicious site. Urlscan seems not to have updated its data since the site is no longer classified as malicious by Google Safe Browsing.
Figure 4. Urlscan.io exit
Collecting Threat Intelligence
Due to the dynamic nature of URLs and domains, it’s important to gather as much contextual information as possible, which gives you (the analyst) better information to make decisions about the best method of remediation. CheckPhish.ai provides threat intelligence that examines historical data to identify trends or patterns. Using this threat information, we can see that the IP address currently hosting replit.com has at some point been involved in a phishing campaign.
You also have all the relevant information you need if you want to take action against the site, for example by initiating a takedown from the site. CheckPhish.ai provides you with the hosting provider, the IP address, and even the number of past phishing sites that have used that same IP address.
Figure 5. CheckPhish Threat Intelligence
Since ScamAdviser failed to convert the shortened URL in this demo, I entered the “redirected” URL (replit.com/site/careers) obtained from Checkphish.ai for this part of the survey .
As we can see here, ScamAdviser provides very little threat intelligence for an analyst to use in their decision making process. This makes a simple verdict which is correct, however, they had the problem of not being able to handle a redirected url. Beyond the simple verdict, ScamAdviser does not appear to be designed for serious threat researchers or SOC analysts trying to assess a threat and determine a method of remediation.
Figure 6. ScamAdviser threat intelligence
Figure 7. Urlscan.io threat intelligence
We might be biased here, but it looks like CheckPhish stands out in both the accuracy of verdicts and actionable threat intelligence. CheckPhish.ai is built on an enterprise platform used by some of the largest companies in the world. It relies on computer vision and AI to quickly determine a website’s intent in real time rather than relying on threat streams that can be days or even weeks old. This provides very accurate and actionable verdicts which, in turn, help incident response teams take swift action. Additionally, it provides important historical context to understand past phishing activity and help security teams better secure their businesses. Check it out for yourself!
CheckPhish real-time URL analysis: checkphish.ai
*** This is a syndicated Security Bloggers Network blog from Bolster Blog written by Jeff Baher. Read the original post on: https://bolster.ai/blog/not-all-url-scanners-are-created-equal-checkphish-vs-urlscanio-vs-scamadviser/