Does a process filling all logical cores have a negative impact on performance?

Because of hyper-threading, my CPU has 2 logical processors per core. If I understand the premise of hyper-threading correctly, it allows each core to have a separate cache and instruction pointer for 2 separate threads simultaneously, but does *not* allow for simultaneous execution of 2 threads by a single core. As such, it basically just mitigates the high overhead of thread swapping, meaning that the negative performance impact that occurs from having more threads running than cores to run them is reduced. I would, however, still expect that there would be *some* overhead involved, and *some* negative performance impact when running, for example, 16 threads on a system with only 8 physical cores. It appears that the software running in my system believes I have 16 CPU cores, due to my hyper-threading with 8 physical cores. There is some software, such as certain build systems, that default to using all available cores in order to maximize parallelization. I know that I can specify the number of threads through arguments for the software I am thinking of. Am I correct in thinking that going beyond 8 threads will have no benefit to performance? Am I correct in thinking that going beyond 8 threads will actually impede performance? Should I, therefore, instruct the programs to use no more than 8 threads?