slow down of mpi by Torque

I'm running Torque with Open MPI on a cluster with 30 nodes and 360 cores. I have found that the wall time of mpirun -np N ~./myjob and qsub -l nodes=1:ppn=N mpirun -np N ~./myjob differs many times. For small jobs it grows from 1.2 s to 20 s, from 2 s to 37 s and so on. For larger jobs the difference becomes important. How to overcome it?