kubernetes 删除namespace 后一直处于Terminating状态
出现无法删除的情况,是因为kubelet 阻塞,有其他的资源在使用该namespace,比如CRD等,尝试重启kubelet,再删除该namespace 也不好使。
在尝试以下命令强制删除也不好使:
[root@k8s-master1 ~]# kubectl delete ns <terminating-namespace> --force --grace-period=0
解决:
1. 运行以下命令以查看处于“Terminating”状态的namespace:
[root@k8s-master1 ~]# kubectl get namespaces
2. 选择一个Terminating namespace,并查看namespace 中的finalizer。运行以下命令:
[root@k8s-master1 ~]# kubectl get namespace <terminating-namespace> -o yaml
得到类似信息如下:
apiVersion: v1
kind: Namespace
metadata:
creationTimestamp: "2019-11-20T15:18:06Z"
deletionTimestamp: "2020-01-16T02:50:02Z"
name: <terminating-namespace>
resourceVersion: "3249493"
selfLink: /api/v1/namespaces/knative-eventing
uid: f300ea38-c8c2-4653-b432-b66103e412db
spec:
finalizers:
- kubernetes
status:
phase: Terminating
3. 导出json格式到tmp.json:
[root@k8s-master1 ~]# kubectl get namespace <terminating-namespace> -o json >tmp.json
4.编辑tmp.josn,删除finalizers 字段的值:
{
"apiVersion": "v1",
"kind": "Namespace",
"metadata": {
"creationTimestamp": "2019-11-20T15:18:06Z",
"deletionTimestamp": "2020-01-16T02:50:02Z",
"name": "<terminating-namespace>",
"resourceVersion": "3249493",
"selfLink": "/api/v1/namespaces/knative-eventing",
"uid": "f300ea38-c8c2-4653-b432-b66103e412db"
},
"spec": {
"finalizers": []
},
"status": {
"phase": "Terminating"
}
}
5. 开启 proxy :
[root@k8s-master1 ~]# kubectl proxy
6. 打开新的terminal 窗口,运行以下命令:
[root@k8s-master1 ~]# curl -k -H "Content-Type: application/json" -X PUT --data-binary @tmp.json http://127.0.0.1:8001/api/v1/namespaces/<terminating-namespace>/finalize
输出以下信息:
{
"kind": "Namespace",
"apiVersion": "v1",
"metadata": {
"name": "istio-system",
"selfLink": "/api/v1/namespaces/istio-system/finalize",
"uid": "2e274537-727f-4a8f-ae8c-397473ed619a",
"resourceVersion": "3249492",
"creationTimestamp": "2019-11-20T15:18:06Z",
"deletionTimestamp": "2020-01-16T02:50:02Z"
},
"spec": {
},
"status": {
"phase": "Terminating"
}
}
7. 检查该namespace 是否被删除:
[root@k8s-master1 ~]# kubectl get namespaces
继续以上步骤删除其他Terminating namespace。