Change the GID of an existing process

Is it possible - for a privileged process - to change the GID of another process. I know that there is the setgroups() system call that allows a process to change it's own GID. However, I want to do this from 'outside' the process (and after the PAM stack was processed). I already saw this post, but there was no real answer to the problem. Are there any reasons for not changing the GID after PAM traversal other than there might be problems with opened resources like file descriptors? I understand that opened files would still use the old GID after the change.