West Coast Labs performed an anti-spam comparative test on various email solutions related to spam detection rate over a series of at least 100,000 emails per solution.
Anti-spam Comparison Report January 2009 Ant i-spam Comparison Report www.westcoastlabs.com 1
Anti-spam Comparison Report
Test Laboratory Details
US Headquarters and Test Facility
West Coast Labs, 16842 Von Karman Avenue, Suite 125
Irvine, CA 92606, U.S.A., Tel: +1 (949) 870 3250, Fax: +1 (949 251 1586)
European Headquarters and Test Facility
West Coast Labs, Unit 9 Oak Tree Court, Mulberry Drive, Cardiff Gate Business
Park, Cardiff, CF23 8RS, UK,
Tel: +44 (0) 29 2054 8400, Fax: +44 (0) 29 2054 8401
Test Facilities also in Delhi, Hong Kong and Sydney, Australia.
Authors: Richard Thomas, Paul Jones Tel : +44 (0)2920 548 400
Date: 23rd January 2009 Issue: 1.1
www.westcoastlabs.com 2
Anti-spam Comparison Report
Contents
Introduction 4
Test Configuration 5
Test Methodology 7
Test Results 9
Product Details 10
Conclusion 15
www.westcoastlabs.com 3
Anti-spam Comparison Report
Introduction
West Coast Labs performed an anti-spam comparative test on various email
solutions related to spam detection rate over a series of at least 100,000
emails per solution.
The vendors that were considered were the following(alphabetically, and by
deployment type):
· Appliances
· Barracuda
· Fortinet
· Ironport
· McAfee
· Hosted Services (SaaS)
· Trend Micro
· Software
· Microsoft
· Sophos
· Symantec
· Trend Micro
· Websense
www.westcoastlabs.com 4
Anti-spam Comparison Report
Test Configuration
Each solution in the test was configured in accordance with the solution
provider's instructions, and was allowed to download the latest updates prior
to the testing. During the course of the testing, signature updates were
allowed. It should also be noted that no training was conducted during the
testing.
All the software-based solutions were installed and configured to receive the
West Coast Labs mail feed. A series of test messages were then sent through
each solution in turn to determine whether they were correctly routing
through to the target collection point. Once confirmed, the products were
then updated to the latest releases and a forensic image was taken of each.
Simultaneously, the four appliances had the appropriate setup
processes/wizards run. As with the software-based solutions, engineers then
sent a series of test messages to check that the messages were correctly
routing to the target collection points. Where possible, backups of the
appliance configuration files were then saved.
The hosted service, InterScan Messaging Hosted Service, was then configured
by engineers at Trend Micro using the network and domain information
provided by West Coast Labs. The routing of traffic was then subsequently
checked.
www.westcoastlabs.com 5
Anti-spam Comparison Report
Test Configuration (Cont.)
With the setup complete, West Coast Labs reset and deleted all previous test
emails and then redirected the live corporate mail feed at each of the
solutions, using in-house multiplexing systems.
Throughout testing, West Coast Labs engineers took a daily backup of each
solution's mailbox. After six days, the mail feed was redirected away from the
solutions and the final mailbox saved.
www.westcoastlabs.com 6
