Posts tagged with Price

Seems like everywhere I turn people are talking about IT best practices. But what about worst practices? After all, most of us were taught that you need to learn from your mistakes. So let me take a second to share some real-life worst practices with respect to server selection and inappropriate use of performance benchmarks to get you thinking:

One HPC end customer once told me that he knew his application would not perform as well on one particular configuration as another, but achieving a specific benchmark score and having a higher ranking on the Top 500 list than their chief competition was what was driving their decision

A large server cluster was purchased based on performance benchmarks and the theoretical performance it could achieve. After spending months in product and vendor selection, procurement, acquisition, assembly, the user found that the datacenter didn’t have enough power to run the servers and the system had to be replaced by lower power servers with an alternate chip architecture (can you guess what the alternate was?)

A Fortune 500 company reported that they were seeing almost a 10x performance difference between 2 systems they were testing using home grown benchmarks. Sensing something didn’t seem right; we took a look at the benchmark code and discovered that the dataset being used fit entirely in the L2 cache of one of the systems. After minimally increasing the dataset size, the results changed very quickly and very dramatically in favor of the other system. 

Pretty ugly stuff if you ask me. Now this is not to suggest that benchmarks are unimportant to consider when selecting hardware. They are, but as a sole selection criteria, they may very well lead you astray. The terms benchmarks” and “performance” are not synonyms and are not interchangeable. With the advent of things like server virtualization, the growing importance of power and energy, and the quest for ever better application scalability, the “one size fits all” benchmark approach to server selection no longer works when looking for optimal performance. Consider the following perspective on performance:

• Virtualization = the amount of performance I can get out of a single physical piece of hardware via virtualization software. Better utilization. Let me elaborate a bit more here as I know this is the hot topic for everyone today. End-users must not only look at how fast their virtual sessions can run, but also how many virtual sessions can run efficiently on a single server.

• Power and Energy = The amount of performance I can get out of a piece of hardware per a given amount of power. Performance Per Watt.

• Scalability = The amount of performance I can get out of an application by adding additional processors or cores to the available compute resources.

• Benchmarks = The amount of performance generated for a specific set of routines and/or application modules. Benchmarking your own applications is always best.

• Price = The amount of performance for a fixed amount of money. Or sometimes people use a fixed amount of performance for the best price. Price/Performance.

So what does all of this mean? Simply put, performance is not what it used to be.

By looking differently at performance across a set of relevant vectors, IT decision-makers can significantly improve server selection and better map to specific application and business requirements. My hope is that this comes across as somewhat of a wakeup call and light bulbs are going off as you read this. In the coming weeks, AMD plans to release a number of documents on this topic and to provide metrics and frameworks for helping create server selection models which look beyond just performance as a benchmark result.

I know some of you might suggest that I’m just trying to detract attention from our benchmark scores. Fact of the matter is AMD wins more benchmarks today than we have in years, so this is far from a self-serving, but hopefully it’s self motivating for you and keeps you from being a worst practices example.

Kevin Knox is Vice President of Worldwide Commercial Business at AMD.  His postings are his own opinions and may not represent AMD’s positions, strategies or opinions. Links to third party sites are provided for convenience and unless explicitly stated, AMD is not responsible for the contents of such linked sites and no endorsement is implied.