-
删除ns，一直处于Terminating状态中 强制删除也是出现报错
-
原因：因为ingress controller的镜像 pull 失败，一直在 retry ，所以我就把 ingress-controller delete 掉，但是一直卡住在删除 namespace 阶段 Ctrl + c
[root@master1 ingress]# kubectl delete -f mandatory.yaml
namespace "ingress-nginx" deleted
configmap "nginx-configuration" deleted
configmap "tcp-services" deleted
configmap "udp-services" deleted
serviceaccount "nginx-ingress-serviceaccount" deleted
clusterrole.rbac.authorization.k8s.io "nginx-ingress-clusterrole" deleted
role.rbac.authorization.k8s.io "nginx-ingress-role" deleted
rolebinding.rbac.authorization.k8s.io "nginx-ingress-role-nisa-binding" deleted
clusterrolebinding.rbac.authorization.k8s.io "nginx-ingress-clusterrole-nisa-binding" deleted
daemonset.apps "nginx-ingress-controller" deleted
^C
- 强制结束以后发现 pod 已经被删掉，导入容器以后重新 create 报错,报错内容就是 ingress-nginx 名称空间 处于 Terminating 状态
[root@master1 ingress]# kubectl create -f mandatory.yaml
clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created
clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created
Error from server (AlreadyExists): error when creating "mandatory.yaml": object is being deleted: namespaces "ingress-ngin x" already exists
Error from server (Forbidden): error when creating "mandatory.yaml": configmaps "nginx-configuration" is forbidden: unable to create new content in namespace ingress-nginx because it is being terminated
Error from server (Forbidden): error when creating "mandatory.yaml": configmaps "tcp-services" is forbidden: unable to cre ate new content in namespace ingress-nginx because it is being terminated
Error from server (Forbidden): error when creating "mandatory.yaml": configmaps "udp-services" is forbidden: unable to cre ate new content in namespace ingress-nginx because it is being terminated
Error from server (Forbidden): error when creating "mandatory.yaml": serviceaccounts "nginx-ingress-serviceaccount" is for bidden: unable to create new content in namespace ingress-nginx because it is being terminated
Error from server (Forbidden): error when creating "mandatory.yaml": roles.rbac.authorization.k8s.io "nginx-ingress-role" is forbidden: unable to create new content in namespace ingress-nginx because it is being terminated
Error from server (Forbidden): error when creating "mandatory.yaml": rolebindings.rbac.authorization.k8s.io "nginx-ingress -role-nisa-binding" is forbidden: unable to create new content in namespace ingress-nginx because it is being terminated
Error from server (Forbidden): error when creating "mandatory.yaml": daemonsets.apps "nginx-ingress-controller" is forbidd en: unable to create new content in namespace ingress-nginx because it is being terminated
- 强制删除名称空间报错
[root@master1 ingress]# kubectl delete ns ingress-nginx --force --grace-period=0
warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
Error from server (Conflict): Operation cannot be fulfilled on namespaces "ingress-nginx": The system is ensuring all cont ent is removed from this namespace. Upon completion, this namespace will automatically be purged by the system.
解决步骤
- 导出运行的名称空间至json文件，删掉其中的spec字段内容，因为k8s集群是携带认证的
[root@master1 ingress]# kubectl get ns ingress-nginx -o json > tmp.json
[root@master1 ingress]# vim tmp.json
[root@master1 ingress]# cat tmp.json
{
"apiVersion": "v1",
"kind": "Namespace",
"metadata": {
"annotations": {
"kubectl.kubernetes.io/last-applied-configuration": "{\"apiVersion\":\"v1\",\"kind\":\"Namespace\",\"metadata\":{\"annotations\":{},\"name\":\"ingress-nginx\"}}\n"
},
"creationTimestamp": "2020-09-11T02:16:47Z",
"deletionTimestamp": "2020-09-11T02:33:30Z",
"name": "ingress-nginx",
"resourceVersion": "175097",
"selfLink": "/api/v1/namespaces/ingress-nginx",
"uid": "9f748136-88c5-4627-a531-81547e191073"
},
"status": {
"conditions": [
{
"lastTransitionTime": "2020-09-11T02:33:35Z",
"message": "Discovery failed for some groups, 1 failing: unable to retrieve the complete list of server APIs: metrics.k8s.io/v1beta1: the server is currently unable to handle the request",
"reason": "DiscoveryFailed",
"status": "True",
"type": "NamespaceDeletionDiscoveryFailure"
},
{
"lastTransitionTime": "2020-09-11T02:33:35Z",
"message": "All legacy kube types successfully parsed",
"reason": "ParsedGroupVersions",
"status": "False",
"type": "NamespaceDeletionGroupVersionParsingFailure"
},
{
"lastTransitionTime": "2020-09-11T02:33:35Z",
"message": "All content successfully deleted",
"reason": "ContentDeleted",
"status": "False",
"type": "NamespaceDeletionContentFailure"
}
],
"phase": "Terminating"
}
}
- 执行以下curl命令，使用kube-apiserver的8080端口，执行删除操作
curl -k -H "Content-Type: application/json" -X PUT --data-binary @tmp.json http://127.0.0.1:8080/api/v1/namespaces/ingress-nginx/finalize
- 如果kube-apiserver未开非安全端口，可以手动创建一个代理会话
[root@master1 ingress]# kubectl proxy --port=8081
# 新开一个shell终端执行curl命令
[root@master1 ~]# curl -k -H "Content-Type: application/json" -X PUT --data-binary @tmp.json http://127.0.0.1:8081/api/v1/namespaces/ingress-nginx/finalize
- 然后就会删掉此ingress-nginx名称空间
[root@master1 ingress]# kubectl get ns
NAME STATUS AGE
default Active 7d20h
kube-node-lease Active 7d20h
kube-public Active 7d20h
kube-system Active 7d20h
kubernetes-dashboard Terminating 7d14h
