Antivirus Quality Checking Report
The Antivirus Quality Checking report is split into two parts. The top part of the report is the results against "active" viruses – those viruses that have been found on the Internet by us within the last week. The second part of the report is made up of the archived viruses in our entire virus collection.
Lightspeed Systems Virus Collection
To quality test our own antivirus software an extensive collection of viruses was systematically built up over the years. The collection's three primary sources are: other virus researchers, websites on the Internet, and virus-infected emails sent to Lightspeed and/or our customers. The virus collection is the heart of our quality testing.
Each virus is saved as a unique file. The file names are created using the MD5 hash of the contents of the file, plus whatever file extension the virus originally had, with an underscore added so that the file won't be accidentally executed. Each type of virus is kept in its own subdirectory that is named the same as the Lightspeed virus name. An example of our virus file naming convention is: q:\Virus Archive\Spyware\Adware.Win32.Wintol.o\f6d59d3614b82874ca281b9837d4ef2b.dll_.
If you are interested in virus research and would like access to our virus collection please contact [email protected].
Lightspeed Systems Virus Signature Testing
When we create a new virus signature we make sure that it blocks all the variants of the virus that we have in our virus collection. We then put the signature into the 'security.test' category and distribute it to all of our customers. This lets our customers get a jump start on blocking new virus threats – especially email-borne viruses. This is why our antispam software automatically blocks the 'security.test' category, but our desktop antivirus application does not block it by default.
The new 'security.test' virus signature is then checked against several hundred thousand known good programs to see if the signature over blocks a good program. If the signature does over block a good program, we move the signature into the 'errors' category and try to create a better signature. If the signature passes the testing, it is then moved into the appropriate virus category: 'security.virus', 'security.spyware', or 'security.nettool'. Each time we make any change to a signature, such as changing the category, the changes are automatically sent to all of our customers as part of our automatic database update process. We continue to test the new signatures against programs we have recently downloaded from the Internet to make sure that we haven't made a mistake. If a mistake is detected in a virus signature, it is changed in the master database to the 'errors' category, and within an hour the change is distributed to all of our customers.
There are very few days where new viruses or virus variants are not found. Some days over 1,000 new virus signatures are added to our database and distributed to our customers. Some of our virus signatures are redundant – that is a single virus might be detected by multiple signatures. This is because different types of virus signatures can be processed at different speeds, and so common viruses may be detected by a single 'fast' signature, while slower signatures will detect ALL the variants of the virus.
Even with the above testing procedures, it is possible for the Lightspeed Systems virus researchers to create a virus signature that accidentally identifies a perfectly fine program or document as virus infected. That is why we quality check our antivirus solution against other antivirus vendors' products. If Lightspeed Systems is the only antivirus vendor that identifies a given file as virus infected, then we check that file again and again to make sure we are not over blocking good programs or documents.
Occasionally other antivirus vendors will also over block good programs, but our quality testing will not normally detect this sort of mistake on their part.
Lightspeed Systems Virus Names
A virus is still a virus even if called a different name by different antivirus vendors. Lacking a unified theory on virus naming, Lightspeed Systems does not wish to add to the confusion by coming up with yet another system of virus naming. If Lightspeed is the only vendor to identify a virus, then we use the virus 'type' and the Internet location where we first found the virus as part of the name we assign to the virus. If or when other antivirus vendors identify the same virus, we adopt the most descriptive name of the other antivirus vendors. Some antivirus vendors virus names contain characters that are invalid within the Microsoft Windows directory naming convention, and in those cases we convert the invalid characters to comply with the Windows naming requirements. For example, another antivirus vendor could make up a name of "W32\BadVirus". This would be converted to a Lightspeed virus name of "W32.BadVirus".
You can enter a virus name at http://archive.lightspeedsystems.com and we'll give you a cross reference with the most common alternate names and important information about each virus.
Findings in Our Tests against Other Antivirus Products
When we started testing other antivirus products we found some interesting results. It turned out that some antivirus products were better at detecting viruses than others. We decided to publish the results of our testing and to try to make a good faith effort to keep our cross-vendor testing results updated each week.
The results show that Kaspersky Labs consistently does better than every other antivirus vendor on both active and archived viruses. In addition, some vendors do a reasonable job on the archived viruses, but do a much worse job on active viruses. And it should be noted, "Spyware" specific programs do not do very well at blocking much of anything.
There is one vendor specifically that this Antivirus Quality Checking report does not accurately represent – and that is Lightspeed Systems. Of course we are always going to score 100% when testing against our own virus collection. We would be negligent if we didn't block every dangerous virus that we knew about.
Also important to note is that it is very possible that there might exist huge numbers of viruses that we do not have samples of in our virus collection. You might call this the "Dark Matter Theory of the Antivirus Universe" - there may be these other viruses out there in the universe of the Internet, but since we haven't been able to detect them, as far as we are concerned they don't interact with ordinary matter.
As necessarily flawed as our testing of other antivirus products is, you can still use this report to add to other information you have about the quality of these products.