CyberRatings.org Announces Test Results for Cloud Network Firewall

PRESS RELEASE

Austin, TX – April 3, 2024 – CyberRatings.org (CyberRatings), the non-profit entity dedicated to providing confidence in cybersecurity products and services through its research and testing programs, has completed an independent test of eleven market leading Cloud Network Firewall vendors. Six products were Recommended, one product received a Neutral rating, and four received a Caution rating.

Cloud network firewalls are considered to be the first line of defense when deployed in public cloud providers such as Amazon Web Services, Google Cloud Platform and Microsoft Azure. But implementing security in the cloud can be complex, with multiple factors influencing effectiveness.

CyberRatings tested the cloud firewall products to determine how they handled TLS/SSL (authentication) 1.2 and 1.3 cipher suites (algorithms), how they defended against 984 exploits (attacks that take advantage of a software flaw or install malware), and whether any of 1,645 evasions could bypass protection. At all times the devices needed to remain stable under adverse conditions. To provide a more realistic rating based on modern network traffic, both clear text (HTTP) and encrypted traffic (HTTPS) were measured. Amazon Web Services (AWS) was the public cloud service chosen to run the test.

The combination of Security Effectiveness and Value dictated where products landed on the Security Value Map™ (SVM). Six out of the eleven products were Recommended for their Security Effectiveness with scores ranging from 99.70% to 100%. Recommended ratings are based on threat prevention (how many exploits and evasions were blocked?), TLS/SSL functionality, routing and policy enforcement, and stability and reliability to achieve a final Security Effectiveness score. These same products also demonstrated competitive pricing in the Total Cost per Protected Mbps (Value). The product rated Neutral received a 48.44% Security Effectiveness score. Four products rated Caution had Security Effectiveness scores ranging from 5.39% to 48.37%.

“We have been testing firewalls for years, and more recently cloud network firewalls,” said Vikram Phatak, CEO of CyberRatings.org. “All of the products chosen were market leaders and the range of scores clearly shows that building a product for the cloud is different than building a product on an appliance where you control the environment,” said Phatak. “We recommend that enterprises check with their service providers or IT teams to see which cloud firewall products are currently deployed in their networks.”

As part of the cloud firewall test, CyberRatings also checked to see if products were secure by default. It was discovered that some firewall evasion defenses are not on by default, potentially leaving customers at significant risk. In response, CyberRatings is providing a policy and configuration guide to help enterprises ensure that their firewalls are configured properly.

Encryption matters: roughly 80% of web traffic is encrypted. The top four cipher suites account for over 95% of HTTPS traffic. In some products, decryption was not on by default. Firewalls will not see attacks delivered via HTTPS unless configured to do so. Performance is significantly different when TLS/SSL is turned on. With the exception of one vendor that failed to handle TLS 1.3 despite claiming support, all other vendors supported encryption.

Enterprises should monitor security and performance capabilities, and update firewalls regularly. With the everchanging cloud platform and agile development, something can go wrong even when the security vendor does not make a change.

The following products were evaluated:

Cloud Network Firewall

Rating

Security

Effectiveness

Rated Throughput

(Mbps)

Price per Protected Mbps

Amazon Web Services (AWS) Network Firewall

Caution

5.39%

1,000

$601.34

Barracuda CloudGen Firewall

Caution

11.38%

441

$287.86

Check Point CloudGuard

Recommended

99.80%

1,180

$12.70

Cisco Secure Firewall Threat Defense Virtual

Caution

20.86%

373

$76.91

Forcepoint NGFW

Recommended

99.80%

698

$8.45

Fortinet FortiGate-VM

Recommended

100%

1,458

$10.56

Juniper Networks vSRX

Recommended

99.70%

1,228

$5.72

Palo Alto Networks VM-Series Next-Generation Firewall w/ Advanced Threat Prevention

Recommended

100%

1,036

$5.83

Sophos Firewall

Caution

48.37%

135

$83.16

Versa Networks NGFW

Recommended

99.90%

2,553

$7.58

WatchGuard Firebox Cloud

Neutral

48.44%

291

$27.06

Additional Resources:

Cloud Network Firewall Comparative Report and Test Reports

2024 Best Practices for Cloud Network Firewall Deployment

Exploring the Landscape of Cloud Network Firewalls Available on AWS

Why Firewalls Should be Secure by Default

About CyberRatings.org

CyberRatings.org is a 501©6 non-profit organization dedicated to providing confidence in cybersecurity products and services through our research and testing programs. We provide enterprises with independent, objective ratings of security product efficacy to make informed decisions. To become a member, visit www.cyberratings.org and follow us on LinkedIn.