Storage Design for Single Threaded Applications
If there was one thing that an enterprise architect could do to improve their odds of success it would be this: understand the mission critical applications your business runs…
Now this may seem like a strange statement, you may be thinking: of course I know the applications my business runs! We run Oracle 12, with JBoss, and Apache, with our in-house application. This is not understanding your application, those are the platforms your application runs on. Why do I say this?
When you look at an application like Oracle there’s a LOT of parameters that can be adjusted, for example: Are your Oracle DBAs complaining that their batch jobs are running slowly? Of course they do that’s what DBAs do.
When you look at storage and application performance it’s ABSOLUTELY critical to understand if there are ANY single threaded processes in the application. When an application is single threaded, it has limited paths down to storage. If there are multiple threads the application has multiple paths. If you only have one path to storage, latency can become a very serious issue…
I will say up front this formula we’re about to use isn’t 100% spot on, because storage arrays can do some nice things like caching which returns some data faster than its average. However, this is a VERY good formula to base your performance assumptions on. It’s a conservative way to calculate your expected IOPS, and when we do sizing we should ALWAYS be conservative!
Let’s look at the math…
There are 1000ms in 1 second, and IOPS are a function of: (number of threads * 1000ms) divided by latency. The formula would look like this:
(n * 1000ms) / latency
For a single threaded app. with 1.5ms latency the formula would be…
(1 * 1000) / 1.5 = 666 IOPS
For an application with 4 threads with 1.5ms latency the formula would be…
(4 * 1000) / 1.5 = 2,666 IOPS
Don’t believe me? Go download IOMeter: http://www.iometer.org/doc/downloads.html
Move to a system that’s attached to your storage array, run an IO test with a single thread, and then run that same test with four threads. If I’m wrong, I’ll buy you a cup of coffee.
This is the real reason some application vendors don’t want you to virtualize their application. It has nothing to do with the performance hit of VMware. It has almost everything do with the fact that if you’re running VMware you’re most likely on a storage array. If you’re running a physical machine your most likely running local disk, and local disk has a MUCH lower latency than a storage array since the disks are connected directly to the local SAS or PCI bus. Let’s do the math on a single thread with a local disk with a quarter ms response time:
(1 * 1000) / .25 = 4000 IOPS
All things being equal and assuming you have enough spindles in your server, your response time can be drastically faster. Sadly I had to learn this lesion with a customer several years ago. They had a critical application that was responsible for storing medical images, and the database than ran their application was very old (like 1970’s old) and only used a single thread. When the customer had given me the design requirements they didn’t know that application was single threaded. The application vendor would only sign off on supporting the application if the database server could achieve 5000 IOPS from its storage platform.
It took about half a day’s testing to figure out what was going on under the covers. When we confronted the vendor about their application only making use of a single thread they stated we were correct. We showed them it was an application limitation and that the storage array could achieve 30,000 IOPS on a single VM with several threads, but that didn’t matter to the application vendor…
The sad truth is that Enterprise Architects will have to design for bad applications, but sometimes those bad applications run the business. We had to do a lot of creative engineering to get around the issue, and ultimately the customer was happy but it was a “learning opportunity” for me. One I don’t care to repeat, hopefully you can learn from my experience.