PC MAGAZINE PC LABS Photos

Simplified HOME | CONTACT

Comprehensive guide to the best sites on the net!

SECTIONS: Richard Brill’s Digital Photography | The Computer Forum | Events Calendar | EDITORS PICK and Product Guide | TV RADIO Network Live

The Brill Report Richard Brill Publisher and Editor in Chief

Monday, May 31, 2004

TCF.NET EDITORS PICK

Photo Essay

An inside look at PC Magazines PC Labs.

About PC MAGAZINE

PC Magazine www.pcmag.com, the most important technology publication in the world, delivers authoritative, labs-based comparative reviews of computing and Internet products to more than 6.6 million highly engaged technology buyers.

PC Magazine defines technology for e-business and is the only magazine with in-depth reviews and accurate, repeatable testing from PC Magazine Labs placed in the unique context of today's business technology landscape.

To meet its readers needs for buying information that is as current as it is comprehensive, PC Magazine publishes 22 times a year in print and continuously on the Web

About PC MAGAZINE PC Labs

PC Labs is responsible for developing and conducting accurate, appropriate, and reproducible test methods for all products reviewed in PC Magazine. PC Labs' goal is to provide PC Magazine readers with unbiased, detailed technical analysis of a product's features, performance, ease of use, design, competitive positioning, suitability to perform common tasks, and, to a lesser extent, its stability and compatibility. PC Labs generates test results that are instrumental in assisting PC Magazine readers with their product purchasing decisions.

PC Labs tests products only for editorial purposes, and does not test products commercially for a fee.

Accurate means that the product is evaluated using test methodologies based on technical research of the industry, our readers, and experts in the field.

Appropriate means that product evaluation measurements are based on metrics that are applicable for both individual and corporate real-world use.

Reproducible means that test scenarios used in an evaluation can be recreated at a later time, with results deviating not more than 5 percent from the original results.

Labs Facilities

PC Labs facilities are located at 28 East 28th Street in New York City and occupy approximately 8,000 square feet of operating space. It is situated on the same floor as the rest of PC Magazine's editorial staff.

PC Labs is staffed by more than 25 permanent, full-time employees and, as workload dictates, utilizes additional services from freelance testers, reviewers, and contributing editors.

Test Plans

Before testing begins, PC Labs project leaders develop test plans, commonly called "test scripts", that describe in detail the entire project test methodology for a comparative review project. Product testing may include computer software or hardware performance tests, compatibility tests, electrical or magnetic tests, subjective product or output quality tests, feature verification tests, comparative feature analyses, and suitability-to-task tests. Each of these kinds of tests has associated test or inspection methodologies defined by the project leader.

The test script is instrumental in ensuring that accurate, appropriate, and reproducible testing processes are in place. Test scripts help to remove any possible product bias, permitting each product to receive a fair and balanced technical review. Test scripts also act as reference points after a project is complete and the information it generates is distributed to our readers.

Test scripts may vary in complexity and size, depending on the product category and editorial focus. It is common for a test script to be comprised of 15 or more pages of detailed, step-by-step testing processes.

Test Procedures

All performance and benchmark testing is conducted in PC Labs facilities. We generally test all other aspects in the labs as well, although this is occasionally impractical.

Normally, performance tests are run a minimum of two times to reduce the chances of generating anomalous results. If the second test run generates a result that is less than or equal to plus or minus 5-percent variation from the first test run, we use the results from the first run. If the second test run generates a result greater than plus or minus 5-percent variation, but less than or equal to plus or minus 10-percent variation, we conduct a third run and average the results. Any second or third run that varies by more than plus or minus 10 percent from the original is discussed with the technical analyst or project leader and an appropriate resolution procedure determined.

In certain situations such as PC testing, in which all test parameters are well-defined and automated, the benchmark tests themselves are validated in advance, and test procedures are repeated on a regular basis from story to story, the need for a second run is subject to the project leader's discretion. If the test-results analysis shows unexpected or out-of-range results compared with similar products, products will be re-tested.

Product Handline

PC Labs does not typically purchase reviewed products. The rationale behind this decision is that the computer-distribution channel is not quick enough for us to guarantee that we will always have access to the latest version of a product that we intend to test. By contacting vendors directly, we allow a level playing field, and stay better informed on what the latest version of a product is and when it will be available. This product information is passed directly to our readers so that they, in turn, are better informed.

We employ several tactics to ensure that the product under testing is indeed the product that can ultimately be purchased by an end user. Since these products are loaned by vendors who are selected to participate in a comparative review, PC Labs is fiscally responsible for the products it receives, and must compensate vendors for damage or loss. After the evaluation period (typically 90 days) has passed, the equipment is returned to the vendor.

Vendors usually provide Not for Resale or Evaluation copies of their software products for testing, and when testing is completed, the software is held in inventory and eventually transferred to our software library.

Confidentiality

Prior to publication, PC Labs test results are disclosed only on a need-to-know basis to product reviewers, PC Magazine editorial employees, and other Ziff Davis Media employees directly associated with assisting product testing (such as staff of the eTesting Labs Benchmark Operations). Vendors, advertisers, and members of the Ziff Davis Media sales and marketing staff are prohibited from access to test results until publication.

PC Magazine and PC Labs honor non-disclosure agreements (NDAs), legally binding agreements that give editors and Labs staff access to detailed, proprietary product and strategic information on the condition that this information not be published before a specific date. We use this information to better prepare for product testing and to assist in the analysis and interpretation of results.