Posts tagged with Benchmark
Is a Cloud a Cluster or is a Cluster a Cloud?
Posted by Guest Blogger in 10:55 pm
(3 votes, average: 4.67 out of 5)
Loading ...With the recent publication of several articles about using cloud computing concepts for High Performance Computing applications (here and here), I’ve been wondering about the applications for high-performance, low-power processors. There’s been quite a bit written here already about cloud computing and energy efficiency, but what about applications where high performance and energy efficiency are both important?
At this time last year, the most energy efficient AMD OpteronTM processor-based server (based on the SPECpowerTM_ssj benchmark) could achieve only a score of 203 overall ssj_ops/watt (95,853 ssj_ops & 276W @ 100% target load) and consumed 164W at Active Idle. A server using the newest Six-Core AMD Opteron HE processor achieved a score of 1228 overall ssj_ops/watt (419,277 ssj_ops & 221W @ 100% target load) and consumed only 120W at Active Idle1. That’s more than 6x the performance-per-watt AND more than a 25% drop in Active Idle power.
AMD technology-based servers help increase performance-per-watt and decrease power consumption at the same time by using a suite of features we call AMD-P. AMD-P is supported by the Six-Core AMD Opteron 2400 and 8400 Series processors as well as the Quad-Core AMD Opteron 2300 and 8300 Series processors. This suite of features and the large number or processors that support them enable customers to build energy efficient two-socket, four-socket, and eight-socket servers which can efficiently meet the needs of almost any server application.
When we compare servers using the newest Six-Core AMD Opteron 2400 Series HE processors to servers using existing AMD Opteron processors, we find that a server based on the AMD Opteron 2400 Series HE processor is able to achieve 18% higher performance-per-watt than a server using Quad-Core AMD Opteron 2300 Series HE processors2 and also consumes 18% lower platform-level power than a server using Six-Core AMD Opteron 2400 Series processors3.
That’s a pretty big improvement over a processor that was released just six months ago!
In addition to lowering server Active Idle power and boosting server performance-per-watt, these new Six-Core AMD Opteron HE processors are designed to provide significantly more processing performance than prior low-power AMD Opteron processors. Servers using these processors are able to achieve up to 50% higher performance than servers using Quad-Core AMD Opteron 2300 Series HE and 8300 Series HE processors in the same power and thermal envelope. That’s like getting the performance of V6-powered Ford Mustang and the fuel efficiency of a four-cylinder Ford Fusion in the same car.
Whether they’re being used in a cloud cluster or a High Performance Computing cluster, the newest AMD Opteron HE processors provide plenty of performance for only a few watts.
What do you think – is a cloud a cluster or is a cluster a cloud?
Andy Parma is a Product Marketing Manager for Server/Workstation products at AMD. His postings are his own opinions and may not represent AMD’s positions, strategies or opinions. Links to third party sites are provided for convenience and unless explicitly stated, AMD is not responsible for the contents of such linked sites and no endorsement is implied.
1Configuration Information: 2 x Six-Core AMD Opteron™ processors (“Istanbul”) Model 2425 HE in Supermicro 1021M-UR+ server, 16GB (4×4GB DDR2-800) memory, 500GB SATA disk drive, Coldwatt CWA2-0650-10-SM01 power supply, Microsoft® Windows Server® 2008 Enterprise Edition SP1 64-bit
2Six-Core AMD Opteron™ processor Model 2425 HE [SPECpower_ssj™2008 1228 overall ssj_ops/watt, 419,277 ssj_ops, 221W @ 100% target load] compared to Quad-Core AMD Opteron™ processor Model 2376 HE [SPECpower_ssj™2008 1044 overall ssj_ops/watt, 346,326 ssj_ops, 210W @ 100% target load]
3Six-Core AMD Opteron™ processor Model 2425 HE [SPECpower_ssj™2008 1228 overall ssj_ops/watt, 419,277 ssj_ops, 221W @ 100% target load] compared to Six-Core AMD Opteron™ processor Model 2435 [SPECpower_ssj™2008 1228 overall ssj_ops/watt, 487, 764 ssj_ops, 270W @ 100% target load]
SPEC and the benchmark name SPECpower_ssj are trademarks of the Standard Performance Evaluation Corporation. For the latest SPECpower_ssj2008 benchmark results, visit http://www.spec.org/power_ssj2008.
That Five-letter Dirty Word
Posted by John Fruehe in 9:24 am
Value. There, I said it. Someone once threatened to wash my PowerPoint slides out with soap for using it.
But customers care about it. Today, more than ever; but even in the crazy dot-com 90’s, people still gravitated to value. Who doesn’t want to get the best value for their IT investments? In the business world, people just don’t burn money; it never makes sense.
On the consumer side, people sometimes associate “value” with “cheap.” But that might be because consumers tend to buy one computer at a time. They research the purchase, they obsess, they compare. Nobody wants to go “cheap” because it is their one shot for the next few years.
However, in the enterprise world, server purchases are a constant fact of life, not a single point in time. Most companies, even in today’s environment, continue to deploy servers and look for the best value that they can get, usually with a price/performance or performance/watt metric. Rarely is the decision made just on raw performance.
I have been a critic of performance benchmarks as much as I have been a fan of them. One of the problems that we see in so many benchmarks is that they fail to comprehend the value of the solution – they only measure one vector. Typically that is raw performance.
Take VMmark for instance. If you take the results at face value, it shows you the approximate performance of different systems. But, it is showing the performance for a specific configuration. Perhaps a configuration that you may never actually deploy in real life.
We recently introduced our new Six-Core AMD Opteron™ processors, and along with those processors there were several new benchmark results introduced, including a VMmark benchmark.
Just looking at the raw benchmarks, one might conclude that a 2P Nehalem-based system is going to be a better choice because of the higher performance. However, Collin MacMillan points out in his Solution Oriented Blog that if you look at only one vector, raw performance, then you might miss the big picture. The reason is that, depending on configuration, the Six-Core AMD Opteron processor-based HP DL385 can be priced almost 2/3 less than the Nehalem-based HP DL380 server.
Just over one third of the cost. That is just stunning. Especially when you consider that the typical customer may be loading 5-10 virtual machines on a single 2P server.
Maybe I am going out on a limb here, but if I was trying to justify a virtualization project, I think that telling the CIO to replace 5-10 physical servers with a single server that costs nearly 3X as much as a competing product is simply a losing proposition.
My colleague, Margaret Lewis, takes a closer look at configurations and pricing of Six-Core AMD Opteron processor-based systems (“Istanbul”) and Quad-Core Intel Xeon processor-based systems (“Gainestown”) that have posted top VMmark scores.
In the enterprise world, value might not have the same stigma that it has in the consumer world, and that is why enterprise customers don’t seem to be afraid of it.
When you are lining expensive data centers with row after row of computing devices, someone is bound to ask you what all this is costing and where is the benefit. If you are buying servers based on our new Six-Core AMD Opteron processors you can rest assured that you are filling your data centers with a superior value – and that speaks volumes.
John Fruehe is the Director of Business Development for Server/Workstation products at AMD. His postings are his own opinions and may not represent AMD’s positions, strategies or opinions. Links to third party sites are provided for convenience and unless explicitly stated, AMD is not responsible for the contents of such linked sites and no endorsement is implied.
The Great Benchmark Debate
Posted by John Fruehe in 5:39 pm
I read an interesting take from Forrester the other day about TPC benchmarks. Noel Yuhanna makes the case that “Transaction Processing Performance Council (TPC) benchmarks, once widely accepted as the standard DBMS benchmark, are becoming obsolete.” Why? He says:
“ First, all top-tier DBMS vendors such as […] are delivering high performance and scalability to support most large workloads. Second, TPC benchmarks no longer reflect the complex workloads of today’s real-world deployments. Third, customers that need high-end performance often prefer internal benchmarks to TPC benchmarks. Finally, virtualization, cloud computing, and database-as-a-service are changing the way customers deploy databases, and TPC does not address these architectures.” (from “TPC Benchmarks Don’t Matter Anymore: Features and Cost are Key Factors When Choosing a DBMS” by Noel Yuhanna with Mike Gilpin and David D’Silva, March 6, 2009, copyright Forrester Research)
This touches on a larger issue that we have blogged about before – we are seeing here at AMD that data center demands on the whole are changing. There is less stock being put into how a hardware or software product performs in a lab, and more emphasis on total cost of ownership and how they perform in real-life scenarios.
I want to be clear, there is a place for benchmarks, and we absolutely value how our products perform against the competition in a standardized setting. For instance, our 45nm Quad-Core AMD Opteron™ 8384 processor just set another world record virtualization score on the VMmark test (29.11@19 tiles on a Sun Fire X4600 M2 server running VMware ESX Server 3.5.0 hypervisor.) We also just set the record on the SPECweb2005 benchmark, measuring web server performance, with 45nm Quad-Core AMD Opteron processors in the four-socket 2RU x64 Sun Fire X4440 server.
However, both AMD and our OEM partners recognize that they are merely a component of the overall value equation. The days of saying that purchasing decisions can be simplified based on a single benchmark are long gone.
Benchmarks don’t always reflect real-world implementations but they have been accepted by and large by the industry as a good initial performance indicator. In fact, benchmark progression can even sometimes be interesting to track as multicore processing is crunching data at unprecedented speeds, resulting in some eye-popping results. AMD recognizes that there are always going to be applications that require the best raw performance. However, recent processor unit data from Mercury Research shows that high-end, performance processors are less than 10 percent of the market today. This tells us that customers ultimately buy on more than just raw performance. They look at how a system is optimized for software, what level of energy is it going to consume, how scalable is the hardware, and in today’s economic environment, price-performance and TCO are more critical than ever.
Whether it is a DBMS or not, Forrester raises a really good point here, and one that – at a minimum – is worth discussing, regardless of if you agree or disagree. There are many factors beyond TPC performance - reliability, manageabilty, security, performance to name a few – that impact a decision to buy and are spanning all enterprise computing environments. More specifically, how these factors impact the ongoing demands of your business (something we like to call the “workloads that matter”) is only going to become increasingly important.
John Fruehe is the Director of Business Development for Server/Workstation products at AMD. His postings are his own opinions and may not represent AMD’s positions, strategies or opinions. Links to third party sites are provided for convenience and unless explicitly stated, AMD is not responsible for the contents of such linked sites and no endorsement is implied.
A Socket Full of Growth
Posted by John Fruehe in 12:36 pm
Recently there have been some who have speculated that new server technology allows you to rip out older inefficient servers and replace them with sparkling new servers, making you "a hero at work."
Here’s a little secret for you: the real hero is the person that figures out how to get more performance out of their servers without having to tear those servers out of the racks.
If you bought AMD-based servers you have an opportunity to do just that. With our Direct Connect architecture, we engineered the platform to handle multiple generations of AMD Opteron™ processors. In August, 2006 we introduced the Socket F1207 to the world, initially with dual-core AMD Opteron processors. Later "Barcelona", our first quad-core processor was also able to plug into those sockets. Today’s "Shanghai" processors also fit in those sockets as well.
Later this year, when we plan to introduce the 6-core "Istanbul" processors, they will fit into many of these existing sockets, giving customers longevity in their platforms.
If you want to be the real hero to your organization, AMD-based servers are really the way to go. Changing a processor is a pretty fast and easy, as many of you probably found out when one of our internal training videos ended up on YouTube a few years ago. Typically, upgrading a processor is a pretty straightforward process; because of my job I have access to a number of pre-production parts and I swap them in and out of systems all the time. In 15 minutes you can go from dual core Rev F processors to the latest "Shanghai" quad-cores, giving you up to 200% more performance:
Best of all, because you still get to utilize the energy efficient DDR-2 memory, the upgrade is much more cost effective. Transitioning to a server based on DDR-3 memory is not necessarily a smart move as a 2GB DDR-2 DIMM is currently priced at ~$71 on www.crucial.com vs. a 2GB DDR-3 DIMM currently priced at ~$171. (Registered ECC DIMMs – what you would expect to use in a server.) In many cases the memory costs can be some of the highest component costs in the server.
Of course if you wanted to pull out your server and replace it with a new one, you do have to think about all of the cabling, brackets, slide rails and loose parts that you will have to deal with. If you do head down that path, it makes sense to choose an AMD Opteron-based platform instead, giving you some longevity with its common socket infrastructure.
So if you really want to be a hero in the data center, spend 15 minutes increasing (in some cases doubling) the performance in some of your servers. They’ll think you worked all night, but you’ll know the secret of being a data center hero. And your secret is safe with me.
Two-Socket SPECint®_rate2006:
http://www.spec.org/cpu2006/results/res2008q1/cpu2006-20071220-02913.html http://www.spec.org/cpu2006/results/res2008q4/cpu2006-20081024-05683.html
Two-Socket SPECfp®_rate2006:
http://www.spec.org/cpu2006/results/res2008q1/cpu2006-20071220-02910.html http://www.spec.org/cpu2006/results/res2008q4/cpu2006-20081024-05684.html
Two-Socket SPECjbb®2005:
http://www.spec.org/osg/jbb2005/results/res2008q1/jbb2005-20080130-00442.html http://www.spec.org/osg/jbb2005/results/res2008q4/jbb2005-20081024-00551.html
Two-Socket SPECweb®2005:
http://www.spec.org/osg/web2005/results/res2007q3/web2005-20070828-00079.html
http://www.spec.org/osg/web2005/results/res2008q4/web2005-20081203-00121.html
SPECint®, SPECfp®, SPECjbb® and SPECwe® are registered trademarks of the Standard Performance Evaluation Corporation.
John Fruehe is the Director of Business Development for Server/Workstation products at AMD. His postings are his own opinions and may not represent AMD’s positions, strategies or opinions. Links to third party sites are provided for convenience and unless explicitly stated, AMD is not responsible for the contents of such linked sites and no endorsement is implied.
You Can’t Make a Cake with Just Flour… Why Performance Benchmarks Alone Don’t Work
Posted by Kevin Knox in 3:54 am
Seems like everywhere I turn people are talking about IT best practices. But what about worst practices? After all, most of us were taught that you need to learn from your mistakes. So let me take a second to share some real-life worst practices with respect to server selection and inappropriate use of performance benchmarks to get you thinking:
One HPC end customer once told me that he knew his application would not perform as well on one particular configuration as another, but achieving a specific benchmark score and having a higher ranking on the Top 500 list than their chief competition was what was driving their decision
A large server cluster was purchased based on performance benchmarks and the theoretical performance it could achieve. After spending months in product and vendor selection, procurement, acquisition, assembly, the user found that the datacenter didn’t have enough power to run the servers and the system had to be replaced by lower power servers with an alternate chip architecture (can you guess what the alternate was?)
A Fortune 500 company reported that they were seeing almost a 10x performance difference between 2 systems they were testing using home grown benchmarks. Sensing something didn’t seem right; we took a look at the benchmark code and discovered that the dataset being used fit entirely in the L2 cache of one of the systems. After minimally increasing the dataset size, the results changed very quickly and very dramatically in favor of the other system.
Pretty ugly stuff if you ask me. Now this is not to suggest that benchmarks are unimportant to consider when selecting hardware. They are, but as a sole selection criteria, they may very well lead you astray. The terms benchmarks” and “performance” are not synonyms and are not interchangeable. With the advent of things like server virtualization, the growing importance of power and energy, and the quest for ever better application scalability, the “one size fits all” benchmark approach to server selection no longer works when looking for optimal performance. Consider the following perspective on performance:
- Virtualization = the amount of performance I can get out of a single physical piece of hardware via virtualization software. Better utilization. Let me elaborate a bit more here as I know this is the hot topic for everyone today. End-users must not only look at how fast their virtual sessions can run, but also how many virtual sessions can run efficiently on a single server.
- Power and Energy = The amount of performance I can get out of a piece of hardware per a given amount of power. Performance Per Watt.
- Scalability = The amount of performance I can get out of an application by adding additional processors or cores to the available compute resources.
- Benchmarks = The amount of performance generated for a specific set of routines and/or application modules. Benchmarking your own applications is always best.
- Price = The amount of performance for a fixed amount of money. Or sometimes people use a fixed amount of performance for the best price. Price/Performance.
So what does all of this mean? Simply put, performance is not what it used to be.
By looking differently at performance across a set of relevant vectors, IT decision-makers can significantly improve server selection and better map to specific application and business requirements. My hope is that this comes across as somewhat of a wakeup call and light bulbs are going off as you read this. In the coming weeks, AMD plans to release a number of documents on this topic and to provide metrics and frameworks for helping create server selection models which look beyond just performance as a benchmark result.
I know some of you might suggest that I’m just trying to detract attention from our benchmark scores. Fact of the matter is AMD wins more benchmarks today than we have in years, so this is far from a self-serving, but hopefully it’s self motivating for you and keeps you from being a worst practices example.
 |
Kevin Knox is Vice President of Worldwide Commercial Business at AMD. His postings are his own opinions and may not represent AMD’s positions, strategies or opinions. Links to third party sites are provided for convenience and unless explicitly stated, AMD is not responsible for the contents of such linked sites and no endorsement is implied. |