To what extent are the "factory settings" immutable?

**Context**: I have a Samsung phone and I am doing a "reset to factory settings" **Question**: could an app that was installed before have altered the factory settings? I am not very acquainted with Android (as opposed to Linux) but my understanding is that, for a given Android installation, specific code is loaded during boot. This code ("Android OS") is initially present on a market phone and then updated with the vendor patches as they are published. They modify this original code. You end up with an "up-to-date Android OS". I guess that the updates are signed so the OS you have is always "vendor-certified". This "patched up-to-date Android OS" is what you get by restoring your phone to factory settings. With the above in mind - **is there a way for a non-vendor application to modify this "up-to-date Android OS"?** Note: I am not addressing here the situation with alternative OS reflashes.