k8s中namespace有两种常见的状态,即Active和Terminating状态,其中Terminating状态一般会比较少见,当对应的命名空间下还存在运行的资源,但该命名空间被删除时才会出现所谓的Terminating状态,这种情况下只要等待k8s本身将命名空间下的资源回收后,该命名空间将会被系统自动删除。

但是在某些情况下,即使命名空间下没有运行的资源,但依然无法删除Terminating状态的命名空间的情况,它会一直卡在Terminating状态下,即使用--force --grace-period=0 也删除不了。

接下来的步骤将描述如何手动删除卡住的命名空间:

  1. Run the following command to view the namespaces that are stuck in the Terminating state:
kubectl get namespaces
  1. Select a terminating namespace and view the contents of the namespace to find out the finalizer. Run the following command:
kubectl get namespace <terminating-namespace> -o yaml

Your YAML contents might resemble the following output:

apiVersion: v1
 kind: Namespace
 metadata:
   creationTimestamp: 2018-11-19T18:48:30Z
   deletionTimestamp: 2018-11-19T18:59:36Z
   name: <terminating-namespace>
   resourceVersion: "1385077"
   selfLink: /api/v1/namespaces/<terminating-namespace>
   uid: b50c9ea4-ec2b-11e8-a0be-fa163eeb47a5
 spec:
   finalizers:
   - kubernetes
 status:
   phase: Terminating
  1. Run the following command to create a temporary JSON file:
kubectl get namespace <terminating-namespace> -o json >tmp.json
  1. Edit your tmp.json file. Remove the kubernetes value from the finalizers field and save the file.
    Your tmp.json file might resemble the following output:
{
      "apiVersion": "v1",
      "kind": "Namespace",
      "metadata": {
          "creationTimestamp": "2018-11-19T18:48:30Z",
          "deletionTimestamp": "2018-11-19T18:59:36Z",
          "name": "<terminating-namespace>",
          "resourceVersion": "1385077",
          "selfLink": "/api/v1/namespaces/<terminating-namespace>",
          "uid": "b50c9ea4-ec2b-11e8-a0be-fa163eeb47a5"
      },
      "spec": {
         "finalizers": 
      },
      "status": {
          "phase": "Terminating"
      }
  }
  1. To set a temporary proxy IP and port, run the following command. Be sure to keep your terminal window open until you delete the stuck namespace:
kubectl proxy

Your proxy IP and port might resemble the following output:

Starting to serve on 127.0.0.1:8001
  1. From a new terminal window, make an API call with your temporary proxy IP and port:
curl -k -H "Content-Type: application/json" -X PUT --data-binary @tmp.json http://127.0.0.1:8001/api/v1/namespaces/<terminating-namespace>/finalize

Your output might resemble the following content:

{
   "kind": "Namespace",
   "apiVersion": "v1",
   "metadata": {
     "name": "<terminating-namespace>",
     "selfLink": "/api/v1/namespaces/<terminating-namespace>/finalize",
     "uid": "b50c9ea4-ec2b-11e8-a0be-fa163eeb47a5",
     "resourceVersion": "1602981",
     "creationTimestamp": "2018-11-19T18:48:30Z",
     "deletionTimestamp": "2018-11-19T18:59:36Z"
   },
   "spec": {

   },
   "status": {
     "phase": "Terminating"
   }
}

Note: The finalizer parameter is removed.

  1. Verify that the terminating namespace is removed, run the following command:
kubectl get namespaces

Continue to follow the steps for other namespaces that are stuck in the Terminating state.

参考:https://www.ibm.com/support/knowledgecenter/en/SSBS6K_3.1.1/troubleshoot/ns_terminating.html

参考:https://support.huaweicloud.com/cce_faq/cce_faq_00277.html