West Coast Labs performed a comparison test on a range of email solutions (appliances, software, and hosted service solutions) to measure their relative spam detection rates over
a series of at least 100,000 genuine spam emails using West Coast Labs real-time spam feed.
Independent Product Testing & Validation
Anti-Spam Solutions - January 2009Comparative Testing: Appliances, Software and Hosted Service Solutions
Anti-Spam Test Results Test MethodologyWest Coast Labs performed a comparison test on a range Each of the solutions was tested against the West Coast of email solutions (appliances, software, and hosted service Labs live enterprise spam feed, multiplexed across each of solutions) to measure their relative spam detection rates over the solutions, so that each solution received the same emails. a series of at least 100,000 genuine spam emails using West Testing was conducted over a total of 6 days. Coast Labs real-time spam feed. Each solution under test was configured in accordance with The following table highlights the spam detection rates of the vendor's instructions, and was allowed to download the each solution. The False Positive detection rates for each latest updates prior to the commencement of testing. During solution were also calculated, and were negligible across the course of the test period, signature updates were allowed. each of the solutions, except Microsoft, which recorded a It should also be noted that no performance training was significant False Positive rate of 0.71%. conducted on any of the solutions during the testing.
The overall test results are based on an analysis carried Vendor Spam Catch Rate out against a subset of 50,124 spam messages and 4,249 Trend Micro (hosted) 96.71% genuine messages sent through each of the devices. Trend Micro (software) 96.48%Symantec 95.73%IronPort 95.63%Sophos 94.63%Microsoft 93.89%Websense 88.25%Barracuda 88.07%Fortinet 61.96% The catch rates and false positives calculated were based upon the following formulae: Spam detection rate = (spam marked as spam / total spam) * 100 McAfee 59.66% False Positive rate = (genuine marked as spam / total genuine) * 100
West Coast Labs Anti-Spam Comparison Test: January 2009 Independent Product Testing & Validation
US Sales: T +1 (949) 870 3250Europe Sales: T +44 (0) 2920 548 400West Coast Labs Observations China Sales: T +86 1 343 921 7464 E info@westcoast.com W www.westcoastlabs.comThis test was performed with no ongoing training-a step Full details of the products, test methodology and results data Corporate Offices and Test Facilities:usually recommended by most anti-spam vendors. Instead can be found in the West Coast Labs Test Report which is this test was configured based on out-of-box configurations available for download at: US Headquarters and Test FacilityWest Coast Labs, using the vendor's recommended settings to mimic the www.westcoastlabs.com/producttestreports. 16842 Von Karman Avenue, Suite 125, customer experience after initial purchase. With a period of Irvine, California 92606, USAtraining and ongoing maintenance, these figures are likely to T +1 (949) 870 3250 be different and result in an improvement to detection rates or F +1 (949) 251 1586False Positive rates as appropriate. European Headquarters and Test FacilityOverall, both of the solutions provided by Trend Micro for this West Coast Labs, Unit 9, Oak Tree Court, test demonstrated better protection rates for end users using Mulberry Drive, Cardiff Gate Business Park, Cardiff CF23 8RS, UKthis live test set against the other solutions tested. With a T +44 (0) 2920 548400period of training, customers can expect to receive an even F +44 (0) 2920 548401higher effectiveness from these products. Asia Headquarters and Test FacilityA2/9 Lower Ground floor, Safdarjung Footnote: While West Coast Labs is dedicated to ensuring the highest standard of security product testing in the industry, it is not always possible within the scope of Enclave, Main Africa Avenue Road, any given test to completely and exhaustively validate every variation of the security capabilities and/or functionality of any particular product tested and/or guarantee New Delhi 110 029, India. that any particular product tested is fit for any given purpose. T +91 (0) 11 4602 0622Test results published within any given report should not be taken and accepted in isolation. Potential customers interested in deploying any particular product tested by West Coast Labs are recommended to seek further confirmation that the said product will meet their... [download for more]
