How is a signal "delivered" in Linux?

I am confused by the terminology used to describe Linux signal delivery. Most texts say things like "the signal is delivered to the process" or "the signal is delivered to the thread". It is my understanding that a signal is "delivered" to a signal handler, which resides in a process, when the kernel calls that handler. The process itself is running asynchronously, and this "delivery" process is akin to a CPU calling an interrupt handler. The interrupt handler (signal handler) is not the process thread, nor any thread running under that process, correct? It is a separate thread of its own started by the kernel. So the signal is not delivered to a thread or a process, but is delivered to a signal handler residing in the process and not necessarily associated with any specific thread. If this is not correct, please tell me, for example, the association between the signal handler and a pthread that justifies the terminology of "signal delivered to a pthread".