How to identify and remove packages no longer in use?

For some reason I have installed a lot of packages on my Linux distros (Fedora/CentOS/Debian/Ubuntu/Arch/...). Some of them are only for one-time use. But they will be included for upgrade whenever I run a full system upgrade with the package manager. In this case, those one-time packages simply waste time and bandwidth because they are actually not needed any more. Therefore I want to figure out what they are exactly and remove them. I know some package managers allow you to install packages as dependencies so that they can be automatically removed when their downstream packages are removed. But sometimes I forget to add this option when installing packages, and some of those packages are installed by scripts which didn't use the *as-dependency* option. So the current situation is, I don't know which packages are no longer in use. I think one way to identify these packages is to track the last access time of package files. But I'm not aware of a method to do so using any of the popular package managers (dnf/yum/apt/pacman/...). Does any of them have such a functionality or similar functionality which allows me to identify and remove those one-time packages? A cumbersome way of doing so is to get a full list of files for all packages and sort by their access time. But before I do this I'd like to make sure I'm not reinventing the wheels (i.e. making something a package manager already has).