问题

rke加入新节点报错

ERRO[0150] Failed to upgrade hosts: k8s-node03 with error [host k8s-node03 not ready]

FATA[0150] [workerPlane] Failed to upgrade Worker Plane: [host k8s-node03 not ready]

解决思路

使用kubectl describe查看节点信息:

kubectl describe no k8s-node03

进入节点查看kubelet日志:

time="2023-08-04T05:45:02Z" level=info msg="Failed to read pod IP from plugin/docker: networkPlugin cni failed on the status hook for pod \"rook-discover-m5swz_rook-ceph\": CNI failed to retrieve network namespace path: cannot find network namespace for the terminated container \"9b622051ae21a401d7c3539cd59d13866bfd4d7e5392fa519da80fdd730796bd\""
I0804 05:45:02.806017    5450 kubelet.go:2134] "SyncLoop (PLEG): event for pod" pod="rook-ceph/rook-discover-m5swz" event=&{ID:bf39c7d0-693c-4587-8341-15ca0246728b Type:ContainerDied Data:9b622051ae21a401d7c3539cd59d13866bfd4d7e5392fa519da80fdd730796bd}
I0804 05:45:02.806096    5450 pod_container_deletor.go:79] "Container not found in pod's containers" containerID="9b622051ae21a401d7c3539cd59d13866bfd4d7e5392fa519da80fdd730796bd"
I0804 05:45:02.806646    5450 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="rook-ceph/rook-discover-m5swz"

通过日志发现,有个pod创建失败,停止创建任务,重新执行rke up