Indianapolis (IN) - If you visit AMD's website and look at Barcelona's performance numbers you'll see a very rosy picture.  Benchmarks there show performance numbers which most would desire.  And yet, if you turn to the real-world benchmarks published today, the situation is much different.

AMD's benchmarks

The first benchmark listed on AMD's website is called VMmark Performance.  It shows a 3.2 GHz Opteron 2224 SE compared to a non-existent Barcelona Opteron 2360 SE.  They also use a "VMware ESX 3.5 experimental build 55263" release which is not available to the public.  VMware's website shows VMware ESX Server 3i as their current product, which on build 52542.

The increase in performance for this benchmark is 124% over the much faster clocked 880 system.  That's more than double the performance and it shows just how advanced Barcelona is over their previous architecture.  And yet to obtain these high performance numbers, AMD is using a publicly unavailable processor on an build of VMware ESX, which is also not available.  These numbers are not something we can duplicate in the real world, or even take advantage of yet.  They show only what is possible with Barcelona, not what's available.

The next benchmarks actually show the available 2350 part that was launched today.  The performance increase through AMD's Rapid Virtualization Indexing (RVI) is shown in these benchmarks.  RVI is the same technology previously called Nested Paging.  There are no doubts that RVI speeds up virtualization performance.  And that fact is the reason AMD introduced it into this generation of architecture in the first place.  MS Terminal Services sees a 23% speedup and SQL DB sees 14%.  These speedups shown are very powerful and real-world benefits will be had.  Again, hardware improvements relative to a prior architecture show AMD's increases with Barcelona.

When we get to the benchmarks against Intel's Xeon, they use the 5345.  It's two steps below the highest-clocked Xeon part, but is comparable in power.  Here we find the FLUENT 6.4.3 (beta release) benchmark showing Barcelona exceeding the 5345 by 10%, 28%, 46% and 67% in four different tests.  AMD's Barcelona is also shown scaling at a ratio of 6.45 on this test.  The ideal increase would be 8.0 and Intel's 5345 is shown at only 3.54.

If we look at energy consumption, AMD sees a 45% advantage with four cores, and an 83% advantage with eight cores, over the 5345.  Again, significant increases across the board, though they are using a beta release of the benchmark software in all tests.  The beta release has not been fully validated and could be changed before the final version of this benchmark can be trusted.  They show relative increases in performance, as well as those against Intel.  And they show AMD in a very positive light (up to 67% better than the 5345).

Next we get to the test which prompted me to write this article.  It's also the one which will be the focus for the AMD-benchmark-to-real-world comparison.  AMD's STREAM memory comparison.  In this test they show MB/sec throughput around 17,000 (it's graphical, no hard numbers are shown).  These numbers are 261% to 289% faster than Intel's 5345.  That's nearly 3x as fast.

I should also point out that for the test they use two different memory sizes.  Barcelona used 8x 1GB DDR2 while Xeon use 8x 2GB FB-DIMMs.  They also used two different hard drive sizes.  Barcelona used 150 GB Seagate SATA while Xeon uses 80 GB Seagate SATA.  And the SuSE Linux versions that are listed are not identical.  It does not specifically say they are different, but they are not listed identically.  And to go even one step further, the graphic showing the benchmark results is labeled "Xeon 5345" while the "Configuration Information" link pointing to the details shows two 5355s being used.  So which is it?

How can we trust a benchmark that is so poorly labeled and directed?  With this different hardware and potentially software, are they not comparing apples to oranges?  Shouldn't tests like this, especially those published by AMD itself, use the same equipment and the same Linux install?

AMD has not done its work on these benchmarks on their own website.  They are not complete, yet they indicate numbers which look very promising.  Still, not all of those numbers are even found in the real-world.

Available benchmarks

It's worth noting again that not every reviewer was able to get a Barcelona CPU for review.  And the highest-end 2350 part was not even handed out to all of them.  Some were forced to use 2347 parts at 1.9 GHz.  And even more, some reviewer had to go through special channels outside of AMD, even to other countries.  And those that did receive parts from AMD received them so close to the release date that a whole host of proper benchmarks could not be run.  They wanted to do more, but they didn't have the time because of the way AMD has handled this launch.

By not giving reviewers products with enough lead time, AMD has cast all of us into a position of having to comment on that fact.  We are also required to release lesser results than we might've been able to do otherwise.  Or to paint part of the picture to day, only to find the truer picture in a few days.  And that very fact has been prominent in the reviews so far.  And, whether AMD realizes it or not it does raise suspicion by consumers.  Why did they wait so long to send out Barcelona samples?

Read on the next page: Real world benchmarking and results published by review sites