By Scott M. Fulton, III, Betanews
The developers at Mozilla have set next week as the tentative rollout window for the first public beta of Firefox 3.6 -- the first edition of the organization's big fixes for 3.5 where it's accepting analysis and advice from the general public. Betanews tests this week on a late version of the 3.6 beta preview, close to the organization's planned code freeze, indicate that users will be visibly pleased by what they see: Generally faster JavaScript execution and much faster page rendering will result in a browser that's almost one-fourth faster than its predecessor -- by our estimate, 22.78% faster on average.
Betanews tested the latest available development and stable builds of all five brands of Web browser, on all three modern Windows platforms -- XP SP3, Vista SP2, and Windows 7 RTM. Once again, we threw the kitchen sink at them: our new and stronger performance benchmark suite, consisting of experiments in all facets of rendering, mathematics, control, and even geometry. Like before, we use a slow browser (Microsoft Internet Explorer 7, the previous version, running in Windows Vista SP2) as our 1.00 baseline, and we produce performance indices representing browsers' relative speed compared to IE7.
Averaging all three Windows platforms, the latest Beta 1 preview of Firefox 3.6 posts a Betanews CRPI score of 9.02 -- just better than nine times the performance of IE7. Compare this against a 7.34 score for the latest stable version of Firefox 3.5.3, and 7.81 for the latest daily build of the Firefox 3.5.4 bug fix.
When looking at the results broken down by platform, we've noticed that in the CRPI scores for Firefox's development builds, moving up through the latest Alpha of version 3.7, the Windows 7 score has been pulling closer to the XP score. Typically, XP has been the fastest platform by a noticeable margin; that hasn't been the case in the latest builds. This trend has definitely permeated Opera 10, whose stable and development builds on XP and Win7 are almost dead-even.
Still, Firefox 3.6's score on XP crossed over the 10 mark on our new index, with a 10.44 on that platform versus 9.09 for the current stable version 3.5.3, and 7.38 for version 3.6 on Vista.
Click here for a complete introduction to the Betanews Comprehensive Performance Index.
For our long-time readers who watched browsers' performance scores before we added new tests to our comprehensive suite: You'll recall how during the development of Firefox 3.5, test builds flirted with a 10.00 score on the old index in Windows XP SP3, and only occasionally crossed that mark. Right now, using the old scoring system, Firefox 3.6 Beta 1 is scoring a 13.59 on XP -- moving into the territory that Google Chrome blazed when it first hit the scene.
For all its forward strides, the pace of Firefox's acceleration is finally approaching that of Chrome. That is, Firefox is getting faster at the rate Chrome is getting faster. Of course, Chrome's acceleration has been, and continues to be, meteoric, absorbing about one IE7's worth of performance in its development builds every two months. Having posted a CRPI score of 15.37 just one month ago, the latest stable build of Chrome 3.0.195.25 posted a score of 15.85 this week. (In strange but typical Google fashion, build 195.25 was posted to both the stable and beta channels simultaneously, for reasons known only to Google at the moment.)
While the Chrome 4 development builds had been lagging behind Chrome 3 somewhat, they're catching up nicely now with a CRPI score of 15.35. On Windows 7, Chrome 3 and Chrome 4 are almost dead even in performance (15.27 versus 15.23), and only two tenths of a point separate the stable and dev builds on Vista.
Next: Performance breakdowns by individual test...
Once again, our rendering test shows that Google Chrome 3 appears to have been intentionally developed with Windows XP in mind -- the version of Windows many believe best suits netbooks, which is where Google is still seeking to break through. But Chrome 4 does not follow suit; on average, it actually falls behind the latest stable Apple Safari 4 build.
The Celtic Kane test is the one that last year called out Opera as the "fastest browser," much faster than Firefox. If the CK test alone designated speed, then Firefox would actually be the champion over even Chrome. Firefox tends to be faster at simpler tests; when the loads get tougher, Firefox gets weaker and Chrome and Safari march right on.
Acid3 compliance scores are unchanged for the last several months. We may not see Mozilla attempt to close that last standards compliance gap until the first daily builds of Firefox 4, which may not come until next year.
Chrome loves heavier workloads; and here in the SunSpider battery, we see it surging ahead, better than doubling Firefox's scores. The dev build of Chrome 4 has closed the gap here somewhat against the stable/beta Chrome 3.
Click here for a complete introduction to the Betanews Comprehensive Performance Index.
Next: The four new tests in our CRPI battery...
We found the TestCube geometry test, which was created to address a bug that a Mozilla developer had discovered in Firefox. After we were told there were several areas we hadn't discovered where Opera is clearly the faster browser, we found our readers to be correct on that count, at least with respect to DHTML graphics rendering. Here not only is Opera the clear leader, but Firefox 3.6 Beta 1 is right on the heels of Chrome 3.
One of the more unique test batteries we've uncovered evaluates each browser's performance with handling 10 different permutations of JavaScript libraries -- specifically, how each browser performs when selecting a CSS element mathematically. This is Safari's strong point, though even Opera outperforms Firefox in this class as well.
When a Web page doesn't use CSS to divide its elements, it uses old-style <TABLE> elements. And just because developers...deprecate their use doesn't mean they're not everywhere. In concentrating on developing for newer standards, Firefox actually performs relatively slowly in the table handling category, though the standards-compliant Chrome handles old-style tables with ease. What's amazing here, though, is the gap between performance scores in this battery for both stable/beta and dev editions of Chrome, on Windows 7 (12.60) and Vista (6.69). Chrome is also faster on this test on Win7 than on XP, which is unusual given its usual preference for XP.
Chrome's performance on the general page load test is all over the map, and this appears to be due to Google changing its strategy for how it "fires" (enacts) certain JavaScript loading events, evidently across platforms. We understand that the JavaScript onLoad event is not a good indicator of the time it takes for pages to load, so we don't pretend that it is. However, from a performance standpoint, the fact that these events can fire earlier on some browsers (especially Safari) is a true performance boost for developers -- it means that scripts can start accessing page elements and filling their contents dynamically, even as other elements of the page are still loading. This isn't cheating; it's multitasking, and Safari does it well especially in XP. But you're reading this right: The availability of page elements on the everyday page loading test was much faster on Chrome 3 in Vista (!) than on any other platform. (Now you see why we run a variety of tests and find the average or mean of their scores.)
Click here for a complete introduction to the Betanews Comprehensive Performance Index.
Copyright Betanews, Inc. 2009
