Sample Header Ad - 728x90

Why are my network connections being rejected and the ping command between server does not work?

0 votes
2 answers
90 views
kubernetes alma-linux
Cluster information: 
kubectl version
Client Version: v1.29.14
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.29.14
Cloud being used: bare-metal
Installation method:
Host OS: AlmaLinux 8
CNI and version: Flannel ver: 0.26.4
CRI and version: cri-dockerd ver: 0.3.16
I have a master node and create my first worker node, before executing the command kubeadm join in the worker I could ping from the worker to the master and viceversa without troubles, now that I have executed the kubeadm join ... command I cannot ping between them anymore and I get this error: 
[root@worker-1 ~]# kubectl get nodes -o wide
E0308 19:38:31.027307   59324 memcache.go:265] couldn't get current server API group list: Get "https://198.58.126.88:6443/api?timeout=32s ": dial tcp 198.58.126.88:6443: connect: connection refused
E0308 19:38:32.051145   59324 memcache.go:265] couldn't get current server API group list: Get "https://198.58.126.88:6443/api?timeout=32s ": dial tcp 198.58.126.88:6443: connect: connection refused
E0308 19:38:33.075350   59324 memcache.go:265] couldn't get current server API group list: Get "https://198.58.126.88:6443/api?timeout=32s ": dial tcp 198.58.126.88:6443: connect: connection refused
E0308 19:38:34.099160   59324 memcache.go:265] couldn't get current server API group list: Get "https://198.58.126.88:6443/api?timeout=32s ": dial tcp 198.58.126.88:6443: connect: connection refused
E0308 19:38:35.123011   59324 memcache.go:265] couldn't get current server API group list: Get "https://198.58.126.88:6443/api?timeout=32s ": dial tcp 198.58.126.88:6443: connect: connection refused
The connection to the server 198.58.126.88:6443 was refused - did you specify the right host or port?
Ping from the worker node to the master node: 
[root@worker-1 ~]# ping 198.58.126.88
PING 198.58.126.88 (198.58.126.88) 56(84) bytes of data.
From 198.58.126.88 icmp_seq=1 Destination Port Unreachable
From 198.58.126.88 icmp_seq=2 Destination Port Unreachable
From 198.58.126.88 icmp_seq=3 Destination Port Unreachable
If I run this: 
[root@worker-1 ~]# iptables -F && sudo iptables -t nat -F && sudo iptables -t mangle -F && sudo iptables -X
The ping command starts to work: 
[root@worker-1 ~]# ping 198.58.126.88
PING 198.58.126.88 (198.58.126.88) 56(84) bytes of data.
64 bytes from 198.58.126.88: icmp_seq=1 ttl=64 time=0.030 ms
64 bytes from 198.58.126.88: icmp_seq=2 ttl=64 time=0.025 ms
(The ping command works with the IPv6 address, it just fails with the IPv4 address) But after about one minute it gets blocked again: 
[root@worker-1 ~]# ping 198.58.126.88
PING 198.58.126.88 (198.58.126.88) 56(84) bytes of data.
From 198.58.126.88 icmp_seq=1 Destination Port Unreachable
From 198.58.126.88 icmp_seq=2 Destination Port Unreachable
[root@worker-1 ~]# cat /etc/sysctl.conf
# sysctl settings are defined through files in
# /usr/lib/sysctl.d/, /run/sysctl.d/, and /etc/sysctl.d/.
#
# Vendors settings live in /usr/lib/sysctl.d/.
# To override a whole file, create a new file with the same in
# /etc/sysctl.d/ and put new settings there. To override
# only specific settings, add a file with a lexically later
# name in /etc/sysctl.d/ and put new settings there.
#
# For more information, see sysctl.conf(5) and sysctl.d(5).
net.ipv6.conf.default.forwarding=1
net.ipv6.conf.all.forwarding=1
[root@worker-1 ~]# cd /etc/systctl.d/
-bash: cd: /etc/systctl.d/: No such file or directory
The port 6443/TCP is closed in the worker node and I have tried to open it without success: 
nmap 172.235.135.144 -p 6443                                                                                            ✔  2.7.4   06:19:47
Starting Nmap 7.95 ( https://nmap.org  ) at 2025-03-11 16:22 -05
Nmap scan report for 172-235-135-144.ip.linodeusercontent.com (172.235.135.144)
Host is up (0.072s latency).

PORT     STATE  SERVICE
6443/tcp closed sun-sr-https

Nmap done: 1 IP address (1 host up) scanned in 0.26 seconds
master node: 
[root@master ~]# iptables -nvL
Chain INPUT (policy ACCEPT 1312K packets, 202M bytes)
 pkts bytes target     prot opt in     out     source               destination
1301K  201M KUBE-FIREWALL  all  --  *      *       0.0.0.0/0            0.0.0.0/0
1311K  202M KUBE-IPVS-FILTER  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes ipvs access filter */
1311K  202M KUBE-PROXY-FIREWALL  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-proxy firewall rules */
1311K  202M KUBE-NODE-PORT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes health check rules */
   40  3520 ACCEPT     icmp --  *      *       198.58.126.88        0.0.0.0/0
    0     0 ACCEPT     icmp --  *      *       172.233.172.101      0.0.0.0/0

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
  950  181K KUBE-PROXY-FIREWALL  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-proxy firewall rules */
  950  181K KUBE-FORWARD  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes forwarding rules */
  212 12626 DOCKER-ISOLATION-STAGE-1  all  --  *      *       0.0.0.0/0            0.0.0.0/0
  212 12626 DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  *      br-09363fc9af47  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
   20  1068 DOCKER     all  --  *      br-09363fc9af47  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  br-09363fc9af47 !br-09363fc9af47  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  br-09363fc9af47 br-09363fc9af47  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  *      br-05a2ea8c281b  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    4   184 DOCKER     all  --  *      br-05a2ea8c281b  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  br-05a2ea8c281b !br-05a2ea8c281b  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  br-05a2ea8c281b br-05a2ea8c281b  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  *      br-032fd1b78367  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      br-032fd1b78367  0.0.0.0/0            0.0.0.0/0
    9   504 ACCEPT     all  --  br-032fd1b78367 !br-032fd1b78367  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  br-032fd1b78367 br-032fd1b78367  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  *      br-ae1997e801f3  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      br-ae1997e801f3  0.0.0.0/0            0.0.0.0/0
  132  7920 ACCEPT     all  --  br-ae1997e801f3 !br-ae1997e801f3  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  br-ae1997e801f3 br-ae1997e801f3  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  *      br-9f6d34f7e48a  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
   14   824 DOCKER     all  --  *      br-9f6d34f7e48a  0.0.0.0/0            0.0.0.0/0
    4   240 ACCEPT     all  --  br-9f6d34f7e48a !br-9f6d34f7e48a  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  br-9f6d34f7e48a br-9f6d34f7e48a  0.0.0.0/0            0.0.0.0/0
   29  1886 FLANNEL-FWD  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* flanneld forward */

Chain OUTPUT (policy ACCEPT 1309K packets, 288M bytes)
 pkts bytes target     prot opt in     out     source               destination
1298K  286M KUBE-FIREWALL  all  --  *      *       0.0.0.0/0            0.0.0.0/0
1308K  288M KUBE-IPVS-OUT-FILTER  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes ipvs access filter */

Chain DOCKER (6 references)
 pkts bytes target     prot opt in     out     source               destination
   14   824 ACCEPT     tcp  --  !br-9f6d34f7e48a br-9f6d34f7e48a  0.0.0.0/0            172.24.0.2           tcp dpt:3001
    0     0 ACCEPT     tcp  --  !br-ae1997e801f3 br-ae1997e801f3  0.0.0.0/0            172.21.0.2           tcp dpt:3000
    4   184 ACCEPT     tcp  --  !br-05a2ea8c281b br-05a2ea8c281b  0.0.0.0/0            172.22.0.2           tcp dpt:4443
   12   700 ACCEPT     tcp  --  !br-09363fc9af47 br-09363fc9af47  0.0.0.0/0            172.19.0.2           tcp dpt:4443
    8   368 ACCEPT     tcp  --  !br-09363fc9af47 br-09363fc9af47  0.0.0.0/0            172.19.0.3           tcp dpt:443

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination
  212 12626 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER-ISOLATION-STAGE-2 (0 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain FLANNEL-FWD (1 references)
 pkts bytes target     prot opt in     out     source               destination
   29  1886 ACCEPT     all  --  *      *       10.244.0.0/16        0.0.0.0/0            /* flanneld forward */
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            10.244.0.0/16        /* flanneld forward */

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination
  212 12626 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain KUBE-FORWARD (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes forwarding rules */
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes forwarding conntrack rule */ ctstate RELATED,ESTABLISHED

Chain KUBE-NODE-PORT (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* Kubernetes health check node port */ match-set KUBE-HEALTH-CHECK-NODE-PORT dst

Chain KUBE-PROXY-FIREWALL (2 references)
 pkts bytes target     prot opt in     out     source               destination

Chain KUBE-SOURCE-RANGES-FIREWALL (0 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain KUBE-IPVS-FILTER (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set KUBE-LOAD-BALANCER dst,dst
    2   104 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set KUBE-CLUSTER-IP dst,dst
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set KUBE-EXTERNAL-IP dst,dst
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set KUBE-EXTERNAL-IP-LOCAL dst,dst
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set KUBE-HEALTH-CHECK-NODE-PORT dst
    0     0 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate NEW match-set KUBE-IPVS-IPS dst reject-with icmp-port-unreachable

Chain KUBE-IPVS-OUT-FILTER (1 references)
 pkts bytes target     prot opt in     out     source               destination

Chain KUBE-FIREWALL (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all  --  *      *      !127.0.0.0/8          127.0.0.0/8          /* block incoming localnet connections */ ! ctstate RELATED,ESTABLISHED,DNAT

Chain KUBE-KUBELET-CANARY (0 references)
 pkts bytes target     prot opt in     out     source               destination
worker node: 
[root@worker-1 ~]# iptables -nvL
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
18469 1430K KUBE-IPVS-FILTER  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes ipvs access filter */
10534  954K KUBE-PROXY-FIREWALL  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-proxy firewall rules */
10534  954K KUBE-NODE-PORT  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes health check rules */
10767 1115K KUBE-FIREWALL  all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 KUBE-PROXY-FIREWALL  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kube-proxy firewall rules */
    0     0 KUBE-FORWARD  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes forwarding rules */
    0     0 DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0
    0     0 DOCKER-ISOLATION-STAGE-1  all  --  *      *       0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    0     0 DOCKER     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0
    0     0 ACCEPT     all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination
18359 1696K KUBE-IPVS-OUT-FILTER  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes ipvs access filter */
18605 1739K KUBE-FIREWALL  all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER (1 references)
 pkts bytes target     prot opt in     out     source               destination

Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DOCKER-ISOLATION-STAGE-2  all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER-ISOLATION-STAGE-2 (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all  --  *      docker0  0.0.0.0/0            0.0.0.0/0
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain KUBE-FIREWALL (2 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all  --  *      *      !127.0.0.0/8          127.0.0.0/8          /* block incoming localnet connections */ ! ctstate RELATED,ESTABLISHED,DNAT

Chain KUBE-KUBELET-CANARY (0 references)
 pkts bytes target     prot opt in     out     source               destination

Chain KUBE-FORWARD (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes forwarding rules */
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes forwarding conntrack rule */ ctstate RELATED,ESTABLISHED

Chain KUBE-NODE-PORT (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* Kubernetes health check node port */ match-set KUBE-HEALTH-CHECK-NODE-PORT dst

Chain KUBE-PROXY-FIREWALL (2 references)
 pkts bytes target     prot opt in     out     source               destination

Chain KUBE-SOURCE-RANGES-FIREWALL (0 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain KUBE-IPVS-FILTER (1 references)
 pkts bytes target     prot opt in     out     source               destination
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set KUBE-LOAD-BALANCER dst,dst
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set KUBE-CLUSTER-IP dst,dst
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set KUBE-EXTERNAL-IP dst,dst
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set KUBE-EXTERNAL-IP-LOCAL dst,dst
    0     0 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0            match-set KUBE-HEALTH-CHECK-NODE-PORT dst
   45  2700 REJECT     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ctstate NEW match-set KUBE-IPVS-IPS dst reject-with icmp-port-unreachable

Chain KUBE-IPVS-OUT-FILTER (1 references)
 pkts bytes target     prot opt in     out     source               destination
If I run iptables -F INPUT in the worker the ping command starts to work back again: 
[root@worker-1 ~]# iptables -F INPUT
[root@worker-1 ~]# ping 198.58.126.88
PING 198.58.126.88 (198.58.126.88) 56(84) bytes of data.
64 bytes from 198.58.126.88: icmp_seq=1 ttl=64 time=0.054 ms
64 bytes from 198.58.126.88: icmp_seq=2 ttl=64 time=0.043 ms
64 bytes from 198.58.126.88: icmp_seq=3 ttl=64 time=0.037 ms
64 bytes from 198.58.126.88: icmp_seq=4 ttl=64 time=0.039 ms
64 bytes from 198.58.126.88: icmp_seq=5 ttl=64 time=0.023 ms
64 bytes from 198.58.126.88: icmp_seq=6 ttl=64 time=0.022 ms
64 bytes from 198.58.126.88: icmp_seq=7 ttl=64 time=0.070 ms
64 bytes from 198.58.126.88: icmp_seq=8 ttl=64 time=0.072 ms
^C
--- 198.58.126.88 ping statistics ---
8 packets transmitted, 8 received, 0% packet loss, time 7197ms
rtt min/avg/max/mdev = 0.022/0.045/0.072/0.017 ms
strace command from worker: 
[root@worker-1 ~]# iptables -F INPUT
[root@worker-1 ~]# strace -eopenat kubectl version
openat(AT_FDCWD, "/sys/kernel/mm/transparent_hugepage/hpage_pmd_size", O_RDONLY) = 3
--- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=20723, si_uid=0} ---
openat(AT_FDCWD, "/usr/bin/kubectl", O_RDONLY|O_CLOEXEC) = 3
--- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=20723, si_uid=0} ---
--- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=20723, si_uid=0} ---
--- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=20723, si_uid=0} ---
--- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=20723, si_uid=0} ---
--- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=20723, si_uid=0} ---
--- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=20723, si_uid=0} ---
--- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=20723, si_uid=0} ---
--- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=20723, si_uid=0} ---
--- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=20723, si_uid=0} ---
--- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=20723, si_uid=0} ---
--- SIGURG {si_signo=SIGURG, si_code=SI_TKILL, si_pid=20723, si_uid=0} ---
openat(AT_FDCWD, "/usr/local/sbin", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/local/bin", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/sbin", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/usr/bin", O_RDONLY|O_CLOEXEC) = 3
openat(AT_FDCWD, "/root/bin", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/root/.kube/config", O_RDONLY|O_CLOEXEC) = 3
Client Version: v1.29.14
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
The connection to the server 198.58.126.88:6443 was refused - did you specify the right host or port?
+++ exited with 1 +++
nftables before and after executing the kubeadm join command in the worker enter image description here 
Chain KUBE-IPVS-FILTER (0 references)
target     prot opt source               destination
RETURN     all  --  anywhere             anywhere             match-set KUBE-LOAD-BALANCER dst,dst
RETURN     all  --  anywhere             anywhere             match-set KUBE-CLUSTER-IP dst,dst
RETURN     all  --  anywhere             anywhere             match-set KUBE-EXTERNAL-IP dst,dst
RETURN     all  --  anywhere             anywhere             match-set KUBE-EXTERNAL-IP-LOCAL dst,dst
RETURN     all  --  anywhere             anywhere             match-set KUBE-HEALTH-CHECK-NODE-PORT dst
REJECT     all  --  anywhere             anywhere             ctstate NEW match-set KUBE-IPVS-IPS dst reject-with icmp-port-unreachable
[root@worker-1 ~]# sudo iptables -S
-P INPUT ACCEPT
-P FORWARD ACCEPT
-P OUTPUT ACCEPT
-N KUBE-FIREWALL
-N KUBE-KUBELET-CANARY
-N KUBE-FORWARD
-N KUBE-NODE-PORT
-N KUBE-PROXY-FIREWALL
-N KUBE-SOURCE-RANGES-FIREWALL
-N KUBE-IPVS-FILTER
-N KUBE-IPVS-OUT-FILTER
-A INPUT -m comment --comment "kubernetes ipvs access filter" -j KUBE-IPVS-FILTER
-A INPUT -m comment --comment "kube-proxy firewall rules" -j KUBE-PROXY-FIREWALL
-A INPUT -m comment --comment "kubernetes health check rules" -j KUBE-NODE-PORT
-A FORWARD -m comment --comment "kube-proxy firewall rules" -j KUBE-PROXY-FIREWALL
-A FORWARD -m comment --comment "kubernetes forwarding rules" -j KUBE-FORWARD
-A OUTPUT -m comment --comment "kubernetes ipvs access filter" -j KUBE-IPVS-OUT-FILTER
-A OUTPUT -j KUBE-FIREWALL
-A KUBE-FIREWALL ! -s 127.0.0.0/8 -d 127.0.0.0/8 -m comment --comment "block incoming localnet connections" -m conntrack ! --ctstate RELATED,ESTABLISHED,DNAT -j DROP
-A KUBE-FORWARD -m comment --comment "kubernetes forwarding rules" -j ACCEPT
-A KUBE-FORWARD -m comment --comment "kubernetes forwarding conntrack rule" -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A KUBE-NODE-PORT -m comment --comment "Kubernetes health check node port" -m set --match-set KUBE-HEALTH-CHECK-NODE-PORT dst -j ACCEPT
-A KUBE-SOURCE-RANGES-FIREWALL -j DROP
-A KUBE-IPVS-FILTER -m set --match-set KUBE-LOAD-BALANCER dst,dst -j RETURN
-A KUBE-IPVS-FILTER -m set --match-set KUBE-CLUSTER-IP dst,dst -j RETURN
-A KUBE-IPVS-FILTER -m set --match-set KUBE-EXTERNAL-IP dst,dst -j RETURN
-A KUBE-IPVS-FILTER -m set --match-set KUBE-EXTERNAL-IP-LOCAL dst,dst -j RETURN
-A KUBE-IPVS-FILTER -m set --match-set KUBE-HEALTH-CHECK-NODE-PORT dst -j RETURN
-A KUBE-IPVS-FILTER -m conntrack --ctstate NEW -m set --match-set KUBE-IPVS-IPS dst -j REJECT --reject-with icmp-port-unreachable
The blocked connection from the worker to the master starts to happen as soon as the kubelet service is running; if the kubelet service is stopped then I can ping back the master from the worker. What might be causing this blocking on the worker node? Thanks.
Asked by Rafael Mora (101 rep)
Mar 10, 2025, 02:17 AM
Last activity: Mar 15, 2025, 03:20 AM
Sidebar Ad - 300x250
Related Questions

Browse more questions from Unix & Linux Stack Exchange

More questions tagged "kubernetes"
More questions tagged "alma-linux"
Footer Ad - 728x90