Why are my benchmark times not repeatable, even for a CPU-bound task?

I'm running some benchmarks on my Linux desktop/laptop computer, but I'm not getting reliable results. I'm running a CPU-intensive task that does negligible I/O and doesn't use much RAM. My computer is multicore and not doing much. So I expect only slight variations between runs. But I see huge variations.

$ bash -c 'x=0; time while ((x < 999999)); do ((++x)); done'

real    0m2.281s
user    0m2.279s
sys     0m0.001s
$ bash -c 'x=0; time while ((x < 999999)); do ((++x)); done'

real    0m0.906s
user    0m0.906s
sys     0m0.000s
$ bash -c 'x=0; time while ((x < 999999)); do ((++x)); done'

real    0m1.030s
user    0m1.030s
sys     0m0.000s

There seems to be a cluster of fast times and a cluster of slow times, with the occasional thing in between. The variability between fast times is small enough for my purposes, and the variability between slow times is small enough for my purposes. But I can't work reliably when I don't know if I'm getting a slow time or a fast time or an in-between time. The variability is in user time, and the real time is almost equal to the user time (it's a single-threaded task). So the problem isn't that some other process is sharing the CPU. What's going on, and how can I get reliable benchmarks for a CPU-bound task on my PC?