Poorly performing java application vs poorly performing server

This is a very generic question, but it has been coming up more and more often for me. I would like to try and determine an appropriate course of action for this generic case, as I think it could benefit me and other sysadmins in the future. The commonality in all cases though is java. The reason I am asking this question here rather than stack overflow is that this is a sysadmin question. I think it is very appropriate here. A user comes up to me and says "My application is very slow. The server you gave me has poor performance." This server could be bare metal, OpenVZ, or KVM. In all cases, OS is CentOS 6. When the user tells me this, they are referring to page load time (JIRA) or benchmark numbers. (Selenium or proprietary app) Now, I have a sneaking suspicion that the server itself is fine. I ssh onto the server, and notice that the load is less than 1. Memory usage is fine as well, ranging anywhere from 2-20 gigs of ram free. Finally, I check iowait with vmstat, and notice that it is 0. cpu idle is 99. Looking at Java with top, I can see that CPU percentage is around 200% or so for that process. This is on systems with anywhere from 4 to 24 cores. Of course, openvz does processor allocation different from KVM or bare metal.) In the cases that this occurs on an openvz or kvm guest, I hop onto the host as well, and see similar things. cpu idle is 99%. iowait is 0. memory is sometimes hundreds of gigs free. I have looked at these java processes and threads with strace, and strace -f. There are tons of futex, clock, and epoll. If I reverse grep those, I'm left with mostly ioctl, stat, sendto. Applications where this has happened: Selenium, JIRA, and misc proprietary java applications. Always java though. I have considered tuning the niceness of the java processes, but AFAIK niceness will only affect how processes are scheduled. If the load is low, it won't help to speed things up. Sometimes the application uses mysql, but looking at mysql in top shows almost no cpu usage there. I'm guessing that if that was the bottleneck, we would see higher cpu usage. Bottom line: How do I prove to the user that the server works just fine, but their application sucks? Theoretically, the application could be slow because it encountered a while loop that will never exit. Which leads me to believe that this task is impossible, analogous to solving the halting problem. o_O But perhaps I can tune java to behave a bit better, or perhaps there is a way to profile the running application, point to a chunk of code and say "There's your problem".