Note: We've updated our data set to include results from Vista with 2GB of RAM and also with Office 2003 instead of 2007. Check out our revised numbers here.
With the initial performance characteristics of Windows Vista leaving much to be desired (see our previous post on the subject), many IT organizations have put off deploying the new OS until the first service pack (SP1) is released by Microsoft early next year. The thinking goes that SP1 will address all of these early performance issues and somehow bring Windows Vista on par with - or at least closer to - Windows XP in terms of runtime performance.
Unfortunately, this is simply not the case. Extensive testing by the exo.performance.network (www.xpnet.com) research staff shows that SP1 provides no measurable relief to users saddled with sub-par performance under Vista.
How We Tested
The above conclusion is based on an analysis of the RC0 (v.658) build of Service Pack 1 for Windows Vista. Testing was conducted on a dual-core Dell notebook with 1GB of RAM. The staff ran a variety of test scenarios against both "before" (RTM w/no updates) and "after" (RTM w/SP1 installed) configurations, using the DMS Clarity Studio testing framework to capture scenario scoring and metrics data for upload to the exo.repository.
- During office productivity testing, the staff used the DMS Clarity Studio OfficeBench test script to drive Microsoft Office 2007 through a scripted set of productivity tasks, including creating a compound document and supporting workbooks and presentations materials.
- To test multitasking performance, the staff used the ADO, MAPI and WMP Stress modules - all part of DMS Clarity Studio - to generate a multi-process workload scenario involving client/server database, workflow and streaming media tasks.
Note: DMS Clarity Studio is available as a free download from the exo.performance.network (www.xpnet.com) site. Simply register for your free DMS Clarity Analysis Portal account to access these and other free tools from xpnet.
Test Results
During OfficeBench testing we noted a statistically insignificant delta (~2%) in favor of the SP1-patched configuration. CPU Saturation, Memory Pressure and I/O Contention factors were all comparable, as were process specific metrics - including the Thread Utilization and Thread Growth Potential Indices.
Figure 1 - OfficeBench Completion Times (Seconds)
The multitasking scenario was also comparable, with the ADO and MAPI Stress workloads showing a delta of less than 1% in favor of the SP1-patched configuration. As with the OfficeBench test scenario, system and process metrics for CPU, Memory and I/O were all nearly identical between the two configurations.
Figure 2 - ADO and MAPI Avg. Transaction Times (Seconds)
Note: For more information on the various system and process metrics employed in this article, please login to your private DMS Clarity Analysis Portal site and refer to the Glossary section of the Online Help
Not yet a member of the exo.performance.network? Sign up today! It's free, and you'll be helping us to build the world's first global repository of computer performance-related knowledge and data.
Conclusions
After extensive testing of both RTM and SP1-patched versions of Windows Vista, it seems clear that the hoped-for performance fixes that Microsoft has been hinting at never materialized. Vista + SP1 is no faster than Vista from the RTM image.
Bottom Line: If you've been disappointed with the performance of Windows Vista to date, get used to it. SP1 is simply not the panacea that many predicted. In the end, it's Vista's architecture - not a lack of tuning or bug fixes - that makes it perform so poorly on systems that were "barn-burners" under Windows XP.
Read more...
Posts
