How can I recompile only a specific part of the Linux kernel on Debian Buster?

I'm currently developing a Linux Security Module which is stored in the security directory of the kernel source tree. When I compile and install the kernel using the following commands, the module is loaded and everything is working fine:

fakeroot make -j9 -f debian/rules.gen binary-arch_amd64_none_amd64
apt remove linux-image-4.19.0-9-amd64-unsigned 
dpkg -i linux-image-4.19.0-9-amd64-unsigned_4.19.118-2_amd64.deb

If I make the changes to the module and rebuild the kernel using the commands above however, they won't be included in the new image, unless I delete all build output and recompile the whole kernel. Is there a way to only rebuild a specific part of the kernel i.e. only the security directory?