Even after setting static IP address on my Ubuntu server, I am not able to connect to the server without restarting the Network Manager

I have been trying to turn my old laptop into a server so that I can try hosting few projects to showcase, so I have installed Ubuntu server and experimenting with it and I have connected to the server using the ssh connection using the ssh-key pairs and its was successful, and I have also set the IP address to be static on the Wi-Fi interface to which internet is connected. The problem is if I try connecting to the same server after being away from the server (terminated ssh client) say for 15 mins or so I am not able to connect to the server again until I restart the Network Manager on the server directly. I have tried everything like turning power saving mode on wifi in case its disabling the interface while idle and also assigned static IP address in the file 00-installer-wifi-config.yaml file in netplan and the static IP address is assigned as well. One interesting thing to note is that when I check the server connection using the ifconfig command if everything is running properly while my client is not able to connect to server everything seems fine on server perfect pinging google.com works on server, static IP available, ssh service running properly but still I need to restart Network Manager every time to connect and once I restart the Network Manager the client connects flawlessly. Here is the relevant configuration for problem this is present in /etc/netplan/00-installer-wifi-config.yaml : network: version: 2 renderer: NetworkManager wifis: wlp1so: dhcp4: false addresses: - 192.168.1.***/24 nameservers: addresses: [8.8.8.8, 8.8.4.4] routes: - to: default via: 192.***.*.* access-points: "Parker": password: "**********" Note: the above configuration is only for the static address, and I was the facing the same issue even before using the above config where I had dhcp4 to be true. I changed to static just in hope to resolve the issue, but I had no luck. Network Manager logs:

Dec 13 23:51:59 my-ubuntu-server NetworkManager:   [1734133919.7582] device (lo): state change: config -> ip-config (reason 'none', sys-iface-state: 'external')
Dec 13 23:51:59 my-ubuntu-server NetworkManager:   [1734133919.7598] device (lo): state change: ip-config -> ip-check (reason 'none', sys-iface-state: 'external')
Dec 13 23:51:59 my-ubuntu-server NetworkManager:   [1734133919.7632] manager: (FC:D4:36:07:EF:87): new Bluetooth device (/org/freedesktop/NetworkManager/Devices/4)
Dec 13 23:51:59 my-ubuntu-server NetworkManager:   [1734133919.7637] device (FC:D4:36:07:EF:87): state change: unmanaged -> unavailable (reason 'managed', sys-iface-state: 'external')
Dec 13 23:51:59 my-ubuntu-server NetworkManager:   [1734133919.7657] manager: (FC:DE:90:24:40:1A): new Bluetooth device (/org/freedesktop/NetworkManager/Devices/5)
Dec 13 23:51:59 my-ubuntu-server NetworkManager:   [1734133919.7666] device (FC:DE:90:24:40:1A): state change: unmanaged -> unavailable (reason 'managed', sys-iface-state: 'external')
Dec 13 23:51:59 my-ubuntu-server NetworkManager:   [1734133919.7677] device (lo): state change: ip-check -> secondaries (reason 'none', sys-iface-state: 'external')
Dec 13 23:51:59 my-ubuntu-server NetworkManager:   [1734133919.7680] device (FC:D4:36:07:EF:87): state change: unavailable -> disconnected (reason 'none', sys-iface-state: 'managed')
Dec 13 23:51:59 my-ubuntu-server NetworkManager:   [1734133919.7683] device (FC:DE:90:24:40:1A): state change: unavailable -> disconnected (reason 'none', sys-iface-state: 'managed')
Dec 13 23:51:59 my-ubuntu-server NetworkManager:   [1734133919.7687] device (lo): state change: secondaries -> activated (reason 'none', sys-iface-state: 'external')
Dec 13 23:51:59 my-ubuntu-server NetworkManager:   [1734133919.7695] device (lo): Activation: successful, device activated.
Dec 13 23:51:59 my-ubuntu-server NetworkManager:   [1734133919.7707] modem-manager: ModemManager available
Dec 13 23:51:59 my-ubuntu-server NetworkManager:   [1734133919.8773] device (wlp1s0): supplicant interface state: internal-starting -> disconnected
Dec 13 23:51:59 my-ubuntu-server NetworkManager:   [1734133919.8779] Wi-Fi P2P device controlled by interface wlp1s0 created
Dec 13 23:51:59 my-ubuntu-server NetworkManager:   [1734133919.8797] manager: (p2p-dev-wlp1s0): new 802.11 Wi-Fi P2P device (/org/freedesktop/NetworkManager/Devices/6)
Dec 13 23:51:59 my-ubuntu-server NetworkManager:   [1734133919.8812] device (p2p-dev-wlp1s0): state change: unmanaged -> unavailable (reason 'managed', sys-iface-state: 'external')
Dec 13 23:51:59 my-ubuntu-server NetworkManager:   [1734133919.8838] device (wlp1s0): state change: unavailable -> disconnected (reason 'supplicant-available', sys-iface-state: 'managed')
Dec 13 23:51:59 my-ubuntu-server NetworkManager:   [1734133919.8869] device (p2p-dev-wlp1s0): state change: unavailable -> disconnected (reason 'none', sys-iface-state: 'managed')
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9342] policy: auto-activating connection 'CSS Parker' (7eabc9ff-3833-48b6-aca0-243230b32503)
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9349] device (wlp1s0): Activation: starting connection 'CSS Parker' (7eabc9ff-3833-48b6-aca0-243230b32503)
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9350] device (wlp1s0): state change: disconnected -> prepare (reason 'none', sys-iface-state: 'managed')
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9354] manager: NetworkManager state is now CONNECTING
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9361] device (wlp1s0): state change: prepare -> config (reason 'none', sys-iface-state: 'managed')
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9366] device (wlp1s0): Activation: (wifi) access point 'CSS Parker' has security, but secrets are required.
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9366] device (wlp1s0): state change: config -> need-auth (reason 'none', sys-iface-state: 'managed')
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9379] device (wlp1s0): state change: need-auth -> prepare (reason 'none', sys-iface-state: 'managed')
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9382] device (wlp1s0): state change: prepare -> config (reason 'none', sys-iface-state: 'managed')
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9386] device (wlp1s0): Activation: (wifi) connection 'CSS Parker' has security, and secrets exist.  No new secrets needed.
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9386] Config: added 'ssid' value 'CSS Parker'
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9386] Config: added 'scan_ssid' value '1'
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9387] Config: added 'bgscan' value 'simple:30:-65:300'
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9387] Config: added 'key_mgmt' value 'WPA-PSK WPA-PSK-SHA256 FT-PSK SAE FT-SAE'
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9387] Config: added 'auth_alg' value 'OPEN'
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9387] Config: added 'psk' value ''
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9588] device (wlp1s0): supplicant interface state: disconnected -> authenticating
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9588] device (p2p-dev-wlp1s0): supplicant management interface state: disconnected -> authenticating
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9624] device (wlp1s0): supplicant interface state: authenticating -> associating
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9624] device (p2p-dev-wlp1s0): supplicant management interface state: authenticating -> associating
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9709] device (wlp1s0): supplicant interface state: associating -> associated
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9709] device (p2p-dev-wlp1s0): supplicant management interface state: associating -> associated
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9967] device (wlp1s0): supplicant interface state: associated -> completed
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9967] device (wlp1s0): Activation: (wifi) Stage 2 of 5 (Device Configure) successful. Connected to wireless network "CSS Parker"
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9967] device (p2p-dev-wlp1s0): supplicant management interface state: associated -> completed
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9969] device (wlp1s0): state change: config -> ip-config (reason 'none', sys-iface-state: 'managed')
Dec 13 23:52:02 my-ubuntu-server NetworkManager:   [1734133922.9976] dhcp4 (wlp1s0): activation: beginning transaction (timeout in 45 seconds)
Dec 13 23:52:03 my-ubuntu-server NetworkManager:   [1734133923.0046] dhcp4 (wlp1s0): state changed new lease, address=192.168.1.106, acd pending
Dec 13 23:52:03 my-ubuntu-server NetworkManager:   [1734133923.1472] dhcp4 (wlp1s0): state changed new lease, address=192.168.1.106
Dec 13 23:52:03 my-ubuntu-server NetworkManager:   [1734133923.1494] policy: set 'CSS Parker' (wlp1s0) as default for IPv4 routing and DNS
Dec 13 23:52:03 my-ubuntu-server NetworkManager:   [1734133923.2524] device (wlp1s0): state change: ip-config -> ip-check (reason 'none', sys-iface-state: 'managed')
Dec 13 23:52:03 my-ubuntu-server NetworkManager:   [1734133923.2572] device (wlp1s0): state change: ip-check -> secondaries (reason 'none', sys-iface-state: 'managed')
Dec 13 23:52:03 my-ubuntu-server NetworkManager:   [1734133923.2577] device (wlp1s0): state change: secondaries -> activated (reason 'none', sys-iface-state: 'managed')
Dec 13 23:52:03 my-ubuntu-server NetworkManager:   [1734133923.2586] manager: NetworkManager state is now CONNECTED_SITE
Dec 13 23:52:03 my-ubuntu-server NetworkManager:   [1734133923.2600] device (wlp1s0): Activation: successful, device activated.
Dec 13 23:52:03 my-ubuntu-server NetworkManager:   [1734133923.2613] manager: NetworkManager state is now CONNECTED_GLOBAL
Dec 13 23:52:05 my-ubuntu-server NetworkManager:   [1734133925.5534] manager: startup complete