Build Linux kernel with prebuilt object files

I am trying to build the files in the kernel mm/ directory with a personalized LLVM compiler, then link them together with the rest of the kernel to get a bootable kernel. I extracted the exact command lines that compile the files in the mm/ directory when I compile the entire kernel, I used those command lines but with my personalised LLVM compiler and I generated the entire set of object files for the mm/ directory. I moved that directory into build/mm/ directory and I started a top down compile on the kernel - but it looks like it recompiled everything. Any ideas how could I prevent the kernel build system from rebuilding everything ?