Markham (Ontario) - As we reported yesterday, in the first test results compiled using Tom's Hardware Guide tests of relative graphics card performance, and average sale prices sampled Tuesday from PriceGrabber and Froogle, ATI-brand graphics cards led Nvidia across the board, with a few noteworthy exceptions. In our new projected price/performance curve for GPUs, ATI takes the lead in both the value and premium segments.
Using the same premise as our tests for dual-core CPU price performance in past weeks, we projected exponential price/performance best-fit curves for all GPUs whose cards Tom's Hardware Guide has tested over the past two years, for both Nvidia and ATI brands. TG Daily then averaged the prices for multiple models of graphics cards using these same GPUs, from manufacturers including Asus, BFG, XFX, PNY, and MSI. Click the miniature below for the complete chart.
CORRECTION: The chart above now contains a correction from yesterday concerning the Nvidia 7800 GT, which we received many letters about yesterday evening and this morning. Benchmark data we used for our initial projection did not correlate properly, especially with real-world observation. So we re-assessed the figure using a different set of Tom's Hardware Guide data. As a number of you had noted, it simply didn't make sense that the GeForce 7800 GT would outperform the GeForce 7800 GTX Extreme. We'll chalk that initial figure as an anomaly. Also, one of our readers did some research on his own, and wrote us to say he believed the PriceGrabber data for the 7800 GT's price was inaccurate, believing PriceGrabber erroneously included a dual-card package in its price estimates, therefore inflating the price. As a backup, we did what we've done in the past with CPU performance charts when PriceGrabber data was unavailable: We sampled vendors manually from Froogle, this time making sure they listed only single-card 256 MB configurations. Indeed, we noted a price difference of about $75 in the average. Hopefully, the 7800 GT now rests in a more comfortable position on the chart. But as you'll note, any change to our final conclusions about price/performance curves is almost undetectable.
We also received e-mails asking whether our performance data comparing the Nvidia GeForce 7900 GT with the GeForce 7900 GTX was also in error. As the Wednesday chart indicates, the GT was given a performance score of 15.58, while the GTX was scored at 13.17. We reviewed our data here as well, and determined the source of that conclusion was this 3DMark06 test by Tino Kreiss published last May, in which the GTX fared considerably worse than the GT (both cards were assembled by Gainward). As new numbers come in to us, we may learn whether these results were verified or simply the fault of a bad line of cards.
How we computed the GPU index
Last month, TG Daily premiered our dual-core CPU price/performance curves, comparing Intel and AMD models. There, we were able to show that after precipitous price declines last month, AMD recaptured the price/performance lead for "value" processors, although Intel now holds on to the high-performance category, and has the best performing dual-core CPU per dollar overall.
Our GPU index is based on the same logic, though our approach has to differ somewhat: We correlated multiple rounds of benchmark tests over the years, featured in Tom's Hardware Guide. In order to obtain a reliable index for relative performance, we needed to choose multiple tests with objectives that were distinguished from one another, but whose history extended back far enough in time for us to obtain a "1.0" card to base our comparisons on.
This turned out not to be an easy choice. With the features and capabilities of graphics cards having changed radically over just the last few years, it's more difficult than ever before to consider a recent model card in the same category as one made three years ago. As long-time Tom's readers already know, this means the test suites themselves have changed, so a direct answer with regard to how much a product line has evolved quantitatively is not forthcoming. It takes a bit of math, actually...which gives us a good reason to be doing something with these computers other than just playing games.
Our index system, therefore, is the 128 MB Nvidia GeForce 5900, which launched in 2003. For the purposes of these comparisons, the 5900 always scores a 1.0. All performance indices are therefore relative to this card. Does this materially affect our judgment with regard to either brand, Nvidia or ATI? No, because all three test suites we chose for our overall performance index judge both brands' performance relative to one another as well.
The three test suites for our performance index cover categories which pertain to three-year-old models as well as today's models, which does indeed limit what capabilities we cover. For example, high dynamic range (HDR), or the ability to extend the range between the brightest brights and darkest darks - which is a feature not only of modern cards but of newer games such as Serious Sam 2 - doesn't apply to the 5900, nor does Shader Model 3.0. Of course, we could have chosen a newer card as our baseline index, but like the case of a Doppler radar whose granularity becomes weaker the closer it is to the center of a moving storm, index values for other cards in the vicinity of the baseline index card would have been less informative. We needed a card that was old enough that we were no longer too interested in other cards of its same era, but new enough that we didn't place a gulf of capability between our baseline card and our test subjects.
The benchmarks we chose were as follows:
- F.E.A.R. at 1024 x 768 resolution, with 4x anti-aliasing and 8x anisotropic filtering, and which tests the card's abilities with Microsoft DirectX drivers;
- Quake 4 at 1280 x 1024 resolution, with 4x anti-aliasing and 8x anisotropic filtering, and which tests the card's rendering capability with OpenGL;
- 3DMark06, which is the leading synthetic benchmark.
For this latter test, the problem is as obvious as the big, giant year in its name. We don't have test results for many cards which premiered two years ago, for the 3DMark06 test; likewise, we don't have 3DMark05 results for some of the newest cards. But we do have a certain swatch of tests for which we have both sets of results. So we had to do a bit of mathematical modeling: Specifically, given the older cards' 3DMark05 scores, we estimated what their 3DMark06 score would be. (We attempted the converse approach to see whether it was more reliable, estimating the 3DMark05 score for one card for which we already had the score. We then estimated the 3DMark06 score for a card whose results we also knew, and concluded the latter estimate was more reliable.)
The relative capabilities of each card for which Tom's has obtained results were scored relative to our figures for the Nvidia GeForce 5900. (By the way, our estimated 3DMark06 score for the 5900 is 126.) So a 512 MB ATI Radeon X800 XL with a Quake 4 score of 6.92 in our tests performed 692% better than the 5900 at rendering OpenGL scenes at 12x10 resolution. Its overall performance index is 7.27, which is the average of the three test scores put together, each test counting 33.3% of the total.
Here's something else that readers yesterday noticed: We don't have scores yet for all the cards produced, including both old and new cards in both product lines. As Tom's Hardware Guide continues to compile more comprehensive data, we'll incorporate new numbers into our charts. Wouldn't these missing cards affect the projected price/performance curve one way or the other? If so, we feel only slightly, mainly because the existing data gives us more than enough information to actually project these curves. It's like a very high percentage polling sample for a precinct of fewer than a hundred voters; if one or the other candidate is in the lead by 20% or more, you can pretty much project the winner with 90% of the precincts tallied.
Analysis of the results
As we reported yesterday, an Nvidia GeForce 7950 GX2 with an index score of 17.97 performs almost 18 times better than the Nvidia GeForce 5900. Although the 7950 GX2 has the best overall performance score of all the non-SLI, non-Crossfire setups, ATI's Radeon X1900 XTX sells for $370.75, which is 26.9% less than what we project a hypothetical Nvidia card would sell for ($507.43) if Nvidia were to make one whose performance score was also 15.63 (note the slight correction). The 7950 GX2 sells for $575.14 on average.
Despite that, Nvidia's 256 MB GeForce 7900 GT may be the price/performance leader among all graphics cards currently available, with an index score of 15.58 and an average price of $287.69. A hypothetical ATI card with the same index score, we project, would sell for $404.50; and the nearest real-world competitor is ATI's top-of-the-line Radeon X1900XTX at $370.75. (ATI's release of the Radeon X1950XTX just yesterday could shake up that scenario a bit.)
Meanwhile, ATI's best price performer overall comes in the middle of its product line: The X1600 XT sells for an average of $109.47, according to PriceGrabber, with an index score of 5.62. As wild as this might seem, the older X700 Pro sells for only four cents less on average ($109.43), yet scores only a 3.99 on the index. If Nvidia were to manufacture a card with a 5.62 score, our curve would project its price at $184.98. Among the cards Tom's has tested, however, performance jumps dramatically between the GeForce 6800 at 4.51 and the GeForce 6800 GT at 6.19. There's also a big price gulf there as well, from $131.04 for the 6800 to $234.50 for the 6800 GT. Just above the GT in performance is the 6800 GS, with a score of 7.41. It proves, however, to be the much better deal at $194.55 on average.
One of the things we learn right away from analyzing both performance and price averages is that they are not as perfectly, linearly scalar as manufacturers would like you to believe. Among older models, for instance, ATI's Radeon X1300 Pro, with an index score of 3.19 and an average price of $115.31, doesn't compare to its allegedly older counterpart, the X700 Pro which scores a 3.99 and sells for $109.43. And with an index score of 10.63 and a price of $474.01 on average, according to PriceGrabber, the Nvidia GeForce 7800 GTX Extreme is nowhere near the bargain as the 256 MB 7900 GT, scoring 15.58 and priced at $287.69 on average.
On a price/performance chart, the price a consumer wants to see plot points heading is toward the lower right - lesser cost, greater performance. Right now, it's an Nvidia card that's the furthest in that direction: the 7900 GT. One reader wrote us this morning to question our original headline's conclusion, given the 7900 GT's position, of ATI being the leader across the board. To respond, our conclusion is based on the current state of the market as a whole, for which there will always be exceptions. In earlier years, when the automotive industry used to be held to standards of price/performance, one manufacturer could hold an across-the-board lead, while another usually had a breakaway hit that bucked the trend. This is the case today with Nvidia bucking the ATI trend at this one plot point.
But helping ATI's case is the fact that even its top-of-the-line systems (the new X7950 XTX nothwithstanding) fall at or below the company's projected curve, demonstrating that even its best cards provide respectable value without artificial premiums. Nvidia's GeForce 7950 GX2 also falls below its company's projected curve at the top of the line, but notice how those cards in the middle don't fare nearly as well.
Keep in mind: You're looking at a snapshot of the overall graphics market, taken yesterday, which is in a state of flux. Just because ATI has the lead today doesn't mean it will keep that lead toward the holidays. Also, as Tom's Hardware Guide continues to test even the models of cards it's tested once or twice or three or four or even five times before, the performance numbers may also change. When Tom's Hardware Guide releases a new set of performance benchmarks, we'll want to reassess every plot point you see here, both horizontally and vertically.