xmodmap lost after sleep

In Ubuntu 18.04 I use the following autostart script:

[Desktop Entry]
Type=Application
Exec=/home/user/.xinitrc
Version=1.0
X-GNOME-Autostart-enabled=true
Name=xmodmap
Comment=xmodmap script

which just xmodmap /path/.Xmodmap &. When the system boots up, it works. When the system awakes from sleep, the remapping no longer works. How can I fix this? EDIT: (reply to a comment) This also does not fix the issue:

$ cat /etc/systemd/system/xmodmapbindings.service
[Unit]
Description=xmodmap bindings
Before=sleep.target
StopWhenUnneeded=yes

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStop=-/home/norake/.xinitrc

[Install]
WantedBy=sleep.target

$ cat ~/.xinitrc 
#!/bin/bash

if [ "$USER" != norake ]; then
    su norake -c 'sleep 5; /usr/bin/xmodmap /home/norake/.Xmodmap' &
    # without su, without sleep, without fork (&): doesn't work either
else
    (sleep 5; /usr/bin/xmodmap /home/norake/.Xmodmap) &
fi

sleep 30 doesn't work either. Of course the script run manually works.