Thursday, September 4, 2008

IE 8/Chrome Follow-Up

After posting our controversial findings about memory usage under IE 8 and Chrome (see previous post here and here), we received feedback from some our our readers criticizing our use of the Process\Working Set counter values to size the various browser processes. The complaint was that this counter does not factor code (OCX, DLLs, et al) shared across the various process instances.

Fortunately, the DMS Clarity Tracker agent also collects the Page File Bytes counter for each process (i.e. the “Private Bytes” counter as represented in Task Manager), which is a more accurate measure of the RAM footprint (as backed by reserved page file space) of each process instance. And indeed, PF Bytes paints a slightly different picture than Working Set.

image 
Figure 1 – Page File (i.e. Private) Bytes under Windows XP

For starters, IE 8’s average memory consumption under Windows XP, as tallied across all process instances, drops from 271MB to 242MB when viewed through PF Bytes. Likewise, Chrome’s total process working set (i.e. all 12 instances) drops from 267MB to 208MB, while Firefox’s single process footprint goes from 104MB to 94MB. Interesting, IE 7’s Working Set and PF Bytes values are almost identical – it appears to be sharing nothing with any other process.

image
Figure 2- % Shared Code Across All Browser Instances

In a way, we’re glad that we received the aforementioned critical feedback because it forced us to take a look at the level of code re-use present in each competing browser. And here is where Google’s clean design reaps real dividends: By comparing the total Working Set and PF Bytes values, we can conclude that Chrome is sharing roughly 28% of its code image across its various instances. Compare this to IE 8 (12%) and Firefox (11%), and you begin to see why its sometimes better to start with a clean slate.

Bottom Line: IE 8 is still “fat” by any measure, while Google Chrome’s “chubbiness” is offset somewhat by its more efficient use of shared code across process instances.

Read more...

Wednesday, September 3, 2008

Google Chrome the “Fattest” of them All

Earlier this week, we posted our findings from a 3-way browser comparison we had conducted featuring Internet Explorer 7, Internet Explorer 8 (Beta 2) and Firefox 3.01. As we were going to “print,” we learned that Google was readying its own foray into the browser market – Google Chrome – prompting us to quickly revisit our test scenario to add this high-profile newcomer to the mix.

Note: For more information on the exo.performance.network, or to register for your free Analysis Portal account, visit www.xpnet.com.

What we found was truly shocking: After re-executing our 10-site, multi-tab scenario across all 4 browsers, we discovered that it is Google Chrome, not Internet Explorer 8, that is the true memory consumption leader.

With a peak working set (324MB) just shy of IE 8’s updated 332MB mark, Chrome put a massive hit on available memory for our 2GB Windows XP (SP3) test system. And when we switched our focus to the average working set, Chrome actually topped IE 8 with an impressive 267MB footprint (compared to 211MB for IE 8).

image
Figure 1 – Average and Peak Working Sets

Of course, both browser look absolutely porcine when compared to the lean, mean Firefox 3.01 (151MB peak, 104MB average working set size). And lest we forget, IE 7 continues to hover somewhere between the fit & trim Firefox and the obesity that defines Chrome/IE 8 (209MB peak, 142MB average).

One area where the Mozilla.org byproduct shares common ground with Chrome is average CPU utilization. Both products chew-up CPU bandwidth much more aggressively than either of the IE variants, with Chrome actually outpacing Firefox by 2 percentage points (45% to 42% for FF). Contrast these numbers with IE 8 (24%) and IE 7 (13%) and you begin to see that, when it comes to gobbling up CPU cycles, Chrome and Firefox are in fact birds of a feather.

Interestingly, the one place where Chrome completely befuddled us was in its use of threading. Both Firefox and IE 7 spawn a relatively modest number of threads (25 and 43, respectively), a fact related to their reliance on a single process instance to handle all tabbed sessions. By contrast, IE 8 spawns potentially hundreds of threads (153 in our latest test round), and spreads them out across its various instances (in our case, 6 discrete copies of iexplore.exe).

image
Figure 2 – Average and Peak Thread Count 

Given its use of a similar, multi-process model, we would have expected Google Chrome to introduce a comparable thread workload. So we were understandably surprised to discover that Chrome had spun a much more manageable 48 execution threads at the peak of the test scenario (10 sites loaded into 10 separate tabs). Even more surprising was how these threads were distributed: 25 of them belonged to the initial Chrome process instance (i.e. the one that’s created when you first start Chrome); the rest were distributed almost evenly across the other process instances – in our case, 2 threads per instance over 12 discrete instances (with one instance grabbing an extra 3 threads for unknown reasons).

Our guess – and it really is just that until we get confirmation – is that those initial 25 threads are there primarily to handle the base UI functions (client window, address bar, menus), and that all subsequent tabs/processes are created with just two execution threads to handle all of the the core rendering functions for that instance. It’s an interesting design, one likely made possible by Chrome’s modular architecture. And while it may not deliver the kind of massive parallelism that Microsoft seems to be shooting for with IE 8, it should serve to offset Chrome’s voracious appetite for memory. We shudder to think what Google’s browser would look like if it incorporated Microsoft’s overwhelming threading model.

Bottom Line: Chrome, like IE 8, is a browser designed with tomorrow’s hardware in mind. It’s use of a multi-process tabbing model – which, according to Google, helps isolate failures and protect complex web applications (like GMail or Google Docs) – means that it will always use more memory than Firefox, IE 7 and similar, single-process browsers. How such model will hold up under heavy use, especially on today’s hardware, remains to be seen.

Users may decide that the added protection of isolated tab processes is worth the cost in terms of browser “memory bloat.” However, Microsoft is also incorporating isolated tabs in IE 8, albeit on a smaller scale (just 6 process instances to host our 10 tabs – clearly not total isolation). And with its more granular threading model, Microsoft’s browser may hold an advantage as CPU core counts climb and customers embrace the next wave of PC platforms.

One thing’s for certain: The once sedate browser landscape just got a whole lot more interesting!

Read more...

Monday, September 1, 2008

Internet Explorer 8: Over 2x “Fatter” than Firefox

Internet Explorer 8 Beta 2 arrived last week amid great expectations and some heated controversy. The latter was due to Microsoft’s decision to adopt a standards-compliant rendering model that breaks a great many IE-specific web sites.

However, while various bloggers and pundits waxed breathlessly about all of the new bells and whistles (including the much-reported “porn mode”), we were busy putting Microsoft’s new browser under the microscope of the exo.repository.

Note: For more information on the exo.performance.network, or to register for your free Analysis Portal account, visit www.xpnet.com.

Over the course of several days, we evaluated IE 8 under both Windows XP (SP3) and Vista x86 (SP1) to determine how this major update behaves and what, if any, new burdens it might place on today’s over-taxed (in the case of Vista) Windows PCs.

What we found was another example of unchecked Microsoft  code “bloat,” complete with “shirt-bursting, waistline-stretching” memory consumption and the kind of CPU-hogging thread growth normally reserved for massively parallel server farms.

For example, on our Dell OptiPlex 745 (Core 2 Duo @ 2.66GHz) Vista/XP test bed with 2GB of RAM, IE 8 consumed just under 380MB of memory during a 10-site, multi-tab browsing scenario of popular general media, technical media and humor-related Web destinations (see bottom of this article for a list of sites).

By contrast, IE 7 consumed just under 250MB rendering this same workload, while Firefox 3.01 put both IE versions to shame by completing the same browsing scenario in just 159MB of RAM.

image
Figure 1 – Peak Browser Working Set Under XP/Vista

The story gets worse for IE 8 when you examine the number of threads spawned to complete the scenarios. Under Firefox, the count never exceed 29 concurrent threads. IE 7 spawned a hefty 65 execution threads, while IE 8 tried to choke the life out of the CPU with a massive 171 concurrent threads.

By comparison, the instance of Microsoft SQL Server 2005 x64 Edition that hosts the exo.repository rarely spins more than 100 concurrent threads – this on an 8-way, 8GB server machine servicing over 3000 actively monitored PCs.

image
Figure 2 – Peak Browser Thread Count Under XP/Vista

One area where IE 8 came out ahead of Firefox is in overall CPU utilization. Firefox consumed an average 33% Processor Time under XP (48% under Vista), while IE 8 took-up 22% of the CPU (33% under Vista) and IE 7 was the least aggressive at 13% (24% under Vista).

Considering how much faster Firefox is than IE 7/8 at displaying most web sites, the additional CPU cycles are understandable and likely attributable to a more efficient rendering engine employing fewer threads in exchange for faster linear performance.

Some observations:

  • With a massive working set approaching 20% of our test bed’s 2GB RAM configuration, IE 8 may be the long awaited “killer app” that drives customers towards 4GB+ systems and the 64-bit flavors of Windows Vista/7.
  • By greatly increasing the number of concurrent execution threads, and then spreading them out across multiple, discrete processes (in our case, 6 separate instances of iexplore.exe), Microsoft seems to be positioning IE 8 to take advantage of the greatly expanded core counts of future Intel and AMD CPUs – at the cost of overwhelming today’s single and dual-core PCs.
  • By delivering superior performance while maintaining a nearly 2x smaller memory and CPU footprint across the board, Firefox 3.01 proves once again that the open source community can often trump even the most well-funded commercial projects when it comes to delivering efficient, streamlined code.

Bottom Line: No matter how you slice the data, IE 8 represents a massive expansion of the baseline runtime requirements for Microsoft’s Internet Explorer web browser. Meanwhile, the Firefox folks continue to embarrass Microsoft by “doing it better” (including delivering superior performance and overall standards compliance) while consuming fewer hardware resources. How these efficiencies will play in the aforementioned multi-core future remains to be seen.

It may be that Microsoft’s focus on parallelism will reap dividends once the hardware catches up with the software - much like Vista has looked better as the Windows PC ecosystem has evolved to support its epically porcine girth. Such massively-threaded products will likely feel more at home on the 4, 8 and 16-core systems of tomorrow, but for right now they represent the worst kind of code bloat.


List of web sites visited as part of our test scenario:

Note: To ensure an accurate, rich-media experience, we installed both the Adobe Flash 9 and Microsoft SilverLight 2.0 (Beta) plug-ins under each tested browser/OS combination.

Read more...