Advanced DaemonSet
这个控制器基于原生 DaemonSet 上增强了发布能力，比如 灰度分批、按 Node label 选择、暂停、热升级等。
如果你对原生 DaemonSet 不是很了解，我们强烈建议你先阅读它的文档（在学习 Advanced DaemonSet 之前）：
- Concept of Kubernetes DaemonSet
- Perform a Rolling Update on a DaemonSet
- Perform a Rollback on a DaemonSet
注意 Advanced DaemonSet 是一个 CRD，kind 名字也是 DaemonSet，但是 apiVersion 是 apps.kruise.io/v1alpha1。 这个 CRD 的所有默认字段、默认行为与原生 DaemonSet 完全一致，除此之外还提供了一些 optional 字段来扩展增强的策略。
因此，用户从原生 DaemonSet 迁移到 Advanced DaemonSet，只需要把 apiVersion 修改后提交即可：
- apiVersion: apps/v1
+ apiVersion: apps.kruise.io/v1alpha1
kind: DaemonSet
metadata:
name: sample-ds
spec:
#...
增强策略
在 RollingUpdateDaemonSet 中我们新增了以下字段：
const (
+ // StandardRollingUpdateType replace the old daemons by new ones using rolling update i.e replace them on each node one after the other.
+ // this is the default type for RollingUpdate.
+ StandardRollingUpdateType RollingUpdateType = "Standard"
+ // InplaceRollingUpdateType update container image without killing the pod if possible.
+ InplaceRollingUpdateType RollingUpdateType = "InPlaceIfPossible"
)
// Spec to control the desired behavior of daemon set rolling update.
type RollingUpdateDaemonSet struct {
+ // Type is to specify which kind of rollingUpdate.
+ Type RollingUpdateType `json:"rollingUpdateType,omitempty" protobuf:"bytes,1,opt,name=rollingUpdateType"`
// ...
MaxUnavailable *intstr.IntOrString `json:"maxUnavailable,omitempty" protobuf:"bytes,2,opt,name=maxUnavailable"`
// ...
MaxSurge *intstr.IntOrString `json:"maxSurge,omitempty" protobuf:"bytes,7,opt,name=maxSurge"`
+ // A label query over nodes that are managed by the daemon set RollingUpdate.
+ // Must match in order to be controlled.
+ // It must match the node's labels.
+ Selector *metav1.LabelSelector `json:"selector,omitempty" protobuf:"bytes,3,opt,name=selector"`
+ // The number of DaemonSet pods remained to be old version.
+ // Default value is 0.
+ // Maximum value is status.DesiredNumberScheduled, which means no pod will be updated.
+ // +optional
+ Partition *int32 `json:"partition,omitempty" protobuf:"varint,4,opt,name=partition"`
+ // Indicates that the daemon set is paused and will not be processed by the
+ // daemon set controller.
+ // +optional
+ Paused *bool `json:"paused,omitempty" protobuf:"varint,5,opt,name=paused"`
}
升级方式
Advanced DaemonSet 在 spec.updateStrategy.rollingUpdate 中有一个 rollingUpdateType 字段，标识了如何进行滚动升级：
Standard: 对于每个 node，控制器会先删除旧的 daemon Pod，再创建一个新 Pod，和原生 DaemonSet 行为一致。Surging: 对于每个 node，控制器会先创建一个新 Pod，等它 ready 之后再删除老 Pod。Standard(默认): 控制器会重建升级 Pod，与原生 DaemonSet 行为一致。你可以通过maxUnavailable或maxSurge来控制重建新旧 Pod 的顺序。InPlaceIfPossible: 控制器会尽量采用原地升级的方式，如果不行则重建升级。请阅读该文档了解更多原地升级的细节。 注意，在这个类型下，只能使用maxUnavailable而不能用maxSurge。
adv-daemonset-updateStrategy-rollingUpdate-rollingUpdateType-Standard.yaml
apiVersion: apps.kruise.io/v1alpha1
kind: DaemonSet
metadata:
labels:
app: nginx
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:alpine
updateStrategy:
type: RollingUpdate
rollingUpdate:
rollingUpdateType: Standard
kubectl patch daemonsets.apps.kruise.io nginx --type merge -p '{"spec":{"template":{"metadata":{"annotations":{"test":"test"}}}}}' -n kruise
Selector 标签选择升级
这个策略支持用户通过配置 node 标签的 selector，来指定灰度升级某些特定类型 node 上的 Pod。
adv-daemonset-updateStrategy-rollingUpdate-selector.yaml
apiVersion: apps.kruise.io/v1alpha1
kind: DaemonSet
metadata:
labels:
app: nginx
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:alpine
updateStrategy:
type: RollingUpdate
rollingUpdate:
selector:
matchLabels:
nodeType: canary
kubectl label node 192.168.229.130 nodeType=canary
kubectl patch daemonsets.apps.kruise.io nginx --type merge -p '{"spec":{"template":{"metadata":{"annotations":{"test":"test"}}}}}' -n kruise
分批灰度升级或扩容
Partition 的语义是 保留旧版本 Pod 的数量，默认为 0。 如果在发布过程中设置了 partition，则控制器只会将 (status.DesiredNumberScheduled - partition) 数量的 Pod 更新到最新版本。
adv-daemonset-updateStrategy-rollingUpdate-partition.yaml
apiVersion: apps.kruise.io/v1alpha1
kind: DaemonSet
metadata:
labels:
app: nginx
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:alpine
updateStrategy:
type: RollingUpdate
rollingUpdate:
partition: 1
kubectl patch daemonsets.apps.kruise.io nginx --type merge -p '{"spec":{"template":{"metadata":{"annotations":{"test":"test"}}}}}' -n kruise
另外如果你在 Advanced DaemonSet 中定义了 daemonset.kruise.io/progressive-create-pod: "true" annotation， partition 同样会在扩容的时候控制创建出来 Pod 的数量。
暂停升级
用户可以通过设置 paused 为 true 暂停发布，不过控制器还是会做 replicas 数量管理：
adv-daemonset-updateStrategy-rollingUpdate-paused.yaml
apiVersion: apps.kruise.io/v1alpha1
kind: DaemonSet
metadata:
labels:
app: nginx
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:alpine
updateStrategy:
rollingUpdate:
paused: true
kubectl patch daemonsets.apps.kruise.io nginx --type merge -p '{"spec":{"template":{"metadata":{"annotations":{"test":"test"}}}}}' -n kruise
kubectl patch daemonsets.apps.kruise.io nginx --type merge -p '{"spec":{"updateStrategy":{"rollingUpdate":{"paused":false}}}}' -n kruise
升级镜像自动预热
FEATURE STATE: Kruise v1.3.0
如果你在安装或升级 Kruise 的时候启用了 PreDownloadImageForDaemonSetUpdate feature-gate， DaemonSet 控制器会自动在所有旧版本 pod 所在 node 节点上预热你正在灰度发布的新版本镜像。 这对于应用发布加速很有帮助。
默认情况下 DaemonSet 每个新镜像预热时的并发度都是 1，也就是一个个节点拉镜像。 如果需要调整，你可以通过 apps.kruise.io/image-predownload-parallelism annotation 来设置并发度。
不需要inplace
apps.kruise.io/image-predownload-min-updated-ready-pods: "3"这个注解还没实现
adv-daemonset-predownload.yaml
apiVersion: apps.kruise.io/v1alpha1
kind: DaemonSet
metadata:
annotations:
apps.kruise.io/image-predownload-parallelism: "10"
labels:
app: nginx
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:alpine
生命周期钩子
FEATURE STATE: Kruise v1.1.0
与 CloneSet 提供的生命周期钩子 能力相似。
目前 Advanced DaemonSet 只支持 PreDelete hook，它允许用户在 daemon Pod 被删除前执行一些自定义的逻辑。
type LifecycleStateType string
// Lifecycle contains the hooks for Pod lifecycle.
type Lifecycle struct {
// PreDelete is the hook before Pod to be deleted.
PreDelete *LifecycleHook `json:"preDelete,omitempty"`
}
type LifecycleHook struct {
LabelsHandler map[string]string `json:"labelsHandler,omitempty"`
FinalizersHandler []string `json:"finalizersHandler,omitempty"`
/********************** FEATURE STATE: 1.2.0 ************************/
// MarkPodNotReady = true means:
// - Pod will be set to 'NotReady' at preparingDelete/preparingUpdate state.
// - Pod will be restored to 'Ready' at Updated state if it was set to 'NotReady' at preparingUpdate state.
// Default to false.
MarkPodNotReady bool `json:"markPodNotReady,omitempty"`
/*********************************************************************/
}
例如:
apiVersion: apps.kruise.io/v1alpha1
kind: DaemonSet
spec:
# define with label
lifecycle:
preDelete:
labelsHandler:
example.io/block-deleting: "true"
当 DaemonSet 删除一个 Pod 时（包括缩容和重建升级）：
- 如果没有定义 lifecycle hook 或者 Pod 不符合 preDelete 条件，则直接删除
- 否则，会先将 Pod 更新为
PreparingDelete状态，并等待用户的 controller 将 Pod 中关联的 label/finalizer 去除，再执行 Pod 删除
apiVersion: v1
kind: Pod
metadata:
labels:
example.io/block-deleting: "true" # the pod is hooked by PreDelete hook label
lifecycle.apps.kruise.io/state: PreparingDelete # so we update it to `PreparingDelete` state and wait for user controller to do something and remove the label
删除 Pod 前将其置为 NotReady
FEATURE STATE: Kruise v1.2.0
lifecycle:
preDelete:
markPodNotReady: true
finalizersHandler:
- example.io/unready-blocker
如果设置 preDelete.markPodNotReady=true, Kruise 将会在 Pod 进入 PreparingDelete 状态时，将 KruisePodReady 这个 Pod Condition 设置为 False, Pod 将变为 NotReady。
用户可以利用这一特性，在容器真正被停止之前将 Pod 上的流量先行排除，防止流量损失。
注意: 该特性仅在 Pod 被注入 KruisePodReady 这个 ReadinessGate 时生效。
用户 controller 逻辑示例
与上述 yaml 例子类似，我们需要先将 example.io/block-deleting label 定义在 Advanced DaemonSet 的 template 和 lifecycle 中。
apiVersion: apps.kruise.io/v1alpha1
kind: DaemonSet
spec:
template:
metadata:
labels:
example.io/block-deleting: "true"
# ...
lifecycle:
preDelete:
labelsHandler:
example.io/block-deleting: "true"
用户自定义 controller 的执行逻辑:
- 当发现 Pod 进入
PreparingDelete状态，检查它的节点是否存在，并执行一些处理逻辑（例如资源预留等），最后将 Pod 中的example.io/block-deletinglabel 去掉。
实战
burstReplicas
不知道怎么用
ads-burstReplicas.yaml
apiVersion: apps.kruise.io/v1alpha1
kind: DaemonSet
metadata:
labels:
app: nginx
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:alpine
burstReplicas: 2
updateStrategy:
type: RollingUpdate
rollingUpdate:
rollingUpdateType: Standard
kubectl patch ads nginx --type='json' -p='[{"op": "replace", "path": "/spec/template/spec/containers/0/image", "value":"nginx:latest"}]' -n kruise
lifecycle
inPlaceUpdate
finalizersHandler
labelsHandler
markPodNotReady
preDelete
finalizersHandler
labelsHandler
markPodNotReady
minReadySeconds
这个字段只影响 DaemonSet Status 的 NumberAvailable 字段
ads-minReadySeconds.yaml
apiVersion: apps.kruise.io/v1alpha1
kind: DaemonSet
metadata:
labels:
app: nginx
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:alpine
minReadySeconds: 60
updateStrategy:
type: RollingUpdate
rollingUpdate:
rollingUpdateType: Standard
kubectl patch ads nginx --type='json' -p='[{"op": "replace", "path": "/spec/template/spec/containers/0/image", "value":"nginx:latest"}]' -n kruise
[root@node01 adv-daemonset]# kubectl get pod -n kruise
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-69557bfdb8-kkm7z 1/1 Running 0 4m43s
nginx-2gph5 1/1 Running 0 86s
nginx-lljxg 1/1 Running 0 86s
nginx-qqmqv 1/1 Running 0 86s
status:
currentNumberScheduled: 3
daemonSetHash: 9d7db5555
desiredNumberScheduled: 3
numberAvailable: 2
numberMisscheduled: 0
numberReady: 3
numberUnavailable: 1
observedGeneration: 1
updatedNumberScheduled: 3
revisionHistoryLimit
ads-revisionHistoryLimit.yaml
apiVersion: apps.kruise.io/v1alpha1
kind: DaemonSet
metadata:
labels:
app: nginx
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:alpine
revisionHistoryLimit: 1
updateStrategy:
type: RollingUpdate
rollingUpdate:
rollingUpdateType: Standard
kubectl patch ads nginx --type='json' -p='[{"op": "replace", "path": "/spec/template/spec/containers/0/image", "value":"nginx:latest"}]' -n kruise
kubectl patch ads nginx --type='json' -p='[{"op": "replace", "path": "/spec/template/spec/containers/0/image", "value":"nginx:1.14"}]' -n kruise
[root@node01 ~]# kubectl get controllerrevisions.apps -n kruise
NAME CONTROLLER REVISION AGE
nginx-7949b7b7cd daemonset.apps.kruise.io/nginx 3 18s
nginx-9d7db5555 daemonset.apps.kruise.io/nginx 1 100s
selector
略
template
略
updateStrategy
rollingUpdate
maxSurge和maxUnavailable不能同时使用
maxSurge
数字
ads-updateStrategy-rollingUpdate-maxSurge-digital.yaml
apiVersion: apps.kruise.io/v1alpha1
kind: DaemonSet
metadata:
labels:
app: nginx
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:alpine
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 2
rollingUpdateType: Standard
kubectl patch ads nginx --type='json' -p='[{"op": "replace", "path": "/spec/template/spec/containers/0/image", "value":"nginx:latest"}]' -n kruise
[root@node01 ~]# kubectl get pod -n kruise -w
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-69557bfdb8-kkm7z 1/1 Running 0 23m
nginx-8lw7p 1/1 Running 0 39s
nginx-gjdwn 1/1 Running 0 39s
nginx-tsbh5 1/1 Running 0 39s
nginx-msdv2 0/1 Pending 0 0s
nginx-msdv2 0/1 Pending 0 0s
nginx-5b6kg 0/1 Pending 0 0s
nginx-msdv2 0/1 Pending 0 0s
nginx-5b6kg 0/1 Pending 0 1s
nginx-5b6kg 0/1 Pending 0 1s
nginx-msdv2 0/1 ContainerCreating 0 1s
nginx-5b6kg 0/1 ContainerCreating 0 1s
nginx-msdv2 1/1 Running 0 18s
nginx-5b6kg 1/1 Running 0 18s
nginx-gjdwn 1/1 Terminating 0 62s
nginx-8lw7p 1/1 Terminating 0 62s
nginx-kvf2f 0/1 Pending 0 0s
nginx-kvf2f 0/1 Pending 0 0s
nginx-kvf2f 0/1 ContainerCreating 0 0s
nginx-5b6kg 1/1 Running 0 19s
nginx-msdv2 1/1 Running 0 19s
nginx-kvf2f 0/1 ContainerCreating 0 0s
nginx-8lw7p 0/1 Terminating 0 64s
nginx-gjdwn 0/1 Terminating 0 64s
nginx-8lw7p 0/1 Terminating 0 67s
nginx-8lw7p 0/1 Terminating 0 67s
nginx-kvf2f 1/1 Running 0 5s
nginx-tsbh5 1/1 Terminating 0 68s
nginx-kvf2f 1/1 Running 0 6s
nginx-gjdwn 0/1 Terminating 0 70s
nginx-gjdwn 0/1 Terminating 0 70s
百分比
ads-updateStrategy-rollingUpdate-maxSurge-percent.yaml
apiVersion: apps.kruise.io/v1alpha1
kind: DaemonSet
metadata:
labels:
app: nginx
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:alpine
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 40%
rollingUpdateType: Surging
kubectl patch ads nginx --type='json' -p='[{"op": "replace", "path": "/spec/template/spec/containers/0/image", "value":"nginx:latest"}]' -n kruise
[root@node01 ~]# kubectl get pod -n kruise -w
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-69557bfdb8-kkm7z 1/1 Running 0 33m
nginx-lbx4q 1/1 Running 0 6m18s
nginx-qpzxg 1/1 Running 0 6m18s
nginx-xwx5p 1/1 Running 0 6m18s
nginx-zwcm8 0/1 Pending 0 0s
nginx-zwcm8 0/1 Pending 0 0s
nginx-6tz6h 0/1 Pending 0 0s
nginx-zwcm8 0/1 Pending 0 0s
nginx-6tz6h 0/1 Pending 0 0s
nginx-6tz6h 0/1 Pending 0 0s
nginx-6tz6h 0/1 ContainerCreating 0 0s
nginx-zwcm8 0/1 ContainerCreating 0 0s
nginx-6tz6h 1/1 Running 0 17s
nginx-xwx5p 1/1 Terminating 0 6m39s
nginx-9z2hp 0/1 Pending 0 0s
nginx-9z2hp 0/1 Pending 0 0s
nginx-9z2hp 0/1 Pending 0 0s
nginx-9z2hp 0/1 ContainerCreating 0 0s
nginx-xwx5p 0/1 Terminating 0 6m40s
nginx-6tz6h 1/1 Running 0 18s
nginx-9z2hp 1/1 Running 0 2s
nginx-qpzxg 1/1 Terminating 0 6m41s
nginx-zwcm8 1/1 Running 0 20s
nginx-lbx4q 1/1 Terminating 0 6m42s
nginx-qpzxg 0/1 Terminating 0 6m42s
nginx-9z2hp 1/1 Running 0 3s
nginx-zwcm8 1/1 Running 0 21s
nginx-qpzxg 0/1 Terminating 0 6m43s
nginx-qpzxg 0/1 Terminating 0 6m43s
maxUnavailable
数字
ads-updateStrategy-rollingUpdate-maxUnavailable-digital.yaml
apiVersion: apps.kruise.io/v1alpha1
kind: DaemonSet
metadata:
labels:
app: nginx
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:alpine
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
rollingUpdateType: Standard
kubectl patch ads nginx --type='json' -p='[{"op": "replace", "path": "/spec/template/spec/containers/0/image", "value":"nginx:latest"}]' -n kruise
[root@node01 ~]# kubectl get pod -n kruise -w
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-69557bfdb8-hzplg 1/1 Running 0 102s
nginx-fzkl4 1/1 Running 0 10s
nginx-gnsbv 1/1 Running 0 10s
nginx-x66nb 1/1 Running 0 10s
nginx-fzkl4 1/1 Terminating 0 49s
nginx-fzkl4 0/1 Terminating 0 50s
nginx-fzkl4 0/1 Terminating 0 51s
nginx-fzkl4 0/1 Terminating 0 58s
nginx-fzkl4 0/1 Terminating 0 58s
nginx-cmvd5 0/1 Pending 0 0s
nginx-cmvd5 0/1 Pending 0 0s
nginx-cmvd5 0/1 Pending 0 0s
nginx-cmvd5 0/1 ContainerCreating 0 0s
nginx-cmvd5 1/1 Running 0 17s
nginx-x66nb 1/1 Terminating 0 75s
nginx-cmvd5 1/1 Running 0 18s
nginx-x66nb 0/1 Terminating 0 76s
nginx-x66nb 0/1 Terminating 0 83s
nginx-x66nb 0/1 Terminating 0 83s
nginx-wrtts 0/1 Pending 0 0s
nginx-wrtts 0/1 Pending 0 0s
nginx-wrtts 0/1 Pending 0 0s
nginx-wrtts 0/1 ContainerCreating 0 0s
nginx-wrtts 1/1 Running 0 1s
nginx-gnsbv 1/1 Terminating 0 84s
nginx-wrtts 1/1 Running 0 2s
nginx-gnsbv 0/1 Terminating 0 85s
nginx-gnsbv 0/1 Terminating 0 93s
nginx-gnsbv 0/1 Terminating 0 93s
nginx-ckj8z 0/1 Pending 0 0s
nginx-ckj8z 0/1 Pending 0 0s
nginx-ckj8z 0/1 Pending 0 0s
nginx-ckj8z 0/1 ContainerCreating 0 0s
ads-updateStrategy-rollingUpdate-maxUnavailable-digital-inplaceifpossible.yaml
apiVersion: apps.kruise.io/v1alpha1
kind: DaemonSet
metadata:
labels:
app: nginx
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:alpine
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 1
rollingUpdateType: InPlaceIfPossible
kubectl patch ads nginx --type='json' -p='[{"op": "replace", "path": "/spec/template/spec/containers/0/image", "value":"nginx:latest"}]' -n kruise
[root@node01 ~]# kubectl get pod -n kruise -w
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-69557bfdb8-hzplg 1/1 Running 0 4m25s
nginx-5pcjp 1/1 Running 0 12s
nginx-92r8n 1/1 Running 0 12s
nginx-vh98x 1/1 Running 0 12s
nginx-5pcjp 1/1 Running 0 42s
nginx-5pcjp 1/1 Running 0 42s
nginx-5pcjp 1/1 Running 0 42s
nginx-5pcjp 1/1 Running 1 44s
nginx-5pcjp 1/1 Running 1 44s
nginx-5pcjp 1/1 Running 1 45s
nginx-5pcjp 1/1 Running 1 46s
nginx-92r8n 1/1 Running 0 46s
nginx-92r8n 1/1 Running 0 46s
nginx-92r8n 1/1 Running 0 46s
nginx-92r8n 1/1 Running 1 47s
nginx-92r8n 1/1 Running 1 47s
nginx-92r8n 1/1 Running 1 48s
nginx-92r8n 1/1 Running 1 49s
nginx-vh98x 1/1 Running 0 49s
nginx-vh98x 1/1 Running 0 50s
nginx-vh98x 1/1 Running 0 50s
nginx-vh98x 1/1 Running 1 51s
nginx-vh98x 1/1 Running 1 51s
nginx-vh98x 1/1 Running 1 52s
nginx-vh98x 1/1 Running 1 53s
百分比
rollingUpdateType: Surging时不能设置maxUnavailable
ads-updateStrategy-rollingUpdate-maxUnavailable-percent.yaml
apiVersion: apps.kruise.io/v1alpha1
kind: DaemonSet
metadata:
labels:
app: nginx
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:alpine
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 20%
rollingUpdateType: Surging
Error from server: error when creating "ads-updateStrategy-rollingUpdate-maxUnavailable-percent.yaml": admission webhook "vdaemonset.kb.io" denied the request: spec.updateStrategy.rollingUpdate.maxSurge: Invalid value: intstr.IntOrString{Type:0, IntVal:1, StrVal:""}: may not be set when maxUnavailable is non-zero
ads-updateStrategy-rollingUpdate-maxUnavailable-percent-standard.yaml
apiVersion: apps.kruise.io/v1alpha1
kind: DaemonSet
metadata:
labels:
app: nginx
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:alpine
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 20%
rollingUpdateType: Standard
kubectl patch ads nginx --type='json' -p='[{"op": "replace", "path": "/spec/template/spec/containers/0/image", "value":"nginx:latest"}]' -n kruise
[root@node01 ~]# kubectl get pod -n kruise -w
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-69557bfdb8-hzplg 1/1 Running 0 12m
nginx-dmstk 1/1 Running 0 28s
nginx-ks95s 1/1 Running 0 28s
nginx-r5rwm 1/1 Running 0 28s
nginx-ks95s 1/1 Terminating 0 32s
nginx-ks95s 0/1 Terminating 0 33s
nginx-ks95s 0/1 Terminating 0 34s
nginx-ks95s 0/1 Terminating 0 34s
nginx-4s7r4 0/1 Pending 0 0s
nginx-4s7r4 0/1 Pending 0 0s
nginx-4s7r4 0/1 Pending 0 0s
nginx-4s7r4 0/1 ContainerCreating 0 0s
nginx-4s7r4 1/1 Running 0 16s
nginx-dmstk 1/1 Terminating 0 50s
nginx-4s7r4 1/1 Running 0 17s
nginx-dmstk 0/1 Terminating 0 51s
nginx-dmstk 0/1 Terminating 0 52s
nginx-dmstk 0/1 Terminating 0 52s
nginx-mnhnv 0/1 Pending 0 0s
nginx-mnhnv 0/1 Pending 0 0s
nginx-mnhnv 0/1 ContainerCreating 0 0s
nginx-mnhnv 0/1 ContainerCreating 0 0s
nginx-mnhnv 1/1 Running 0 2s
nginx-r5rwm 1/1 Terminating 0 54s
nginx-mnhnv 1/1 Running 0 3s
nginx-r5rwm 0/1 Terminating 0 55s
nginx-r5rwm 0/1 Terminating 0 67s
nginx-r5rwm 0/1 Terminating 0 67s
nginx-nwrs4 0/1 Pending 0 0s
nginx-nwrs4 0/1 Pending 0 0s
nginx-nwrs4 0/1 Pending 0 0s
nginx-nwrs4 0/1 ContainerCreating 0 0s
nginx-nwrs4 1/1 Running 0 16s
nginx-nwrs4 1/1 Running 0 17s
partition
cloneset partition可以是数字或百分比，adv statefulset和adv daemonset只能是数字
ads-updateStrategy-rollingUpdate-partition.yaml
apiVersion: apps.kruise.io/v1alpha1
kind: DaemonSet
metadata:
labels:
app: nginx
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
test: test
spec:
containers:
- name: nginx
image: nginx:alpine
env:
- name: test
valueFrom:
fieldRef:
fieldPath: metadata.labels['test']
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 20%
partition: 2
rollingUpdateType: InPlaceIfPossible
kubectl patch ads nginx --type='json' -p='[{"op": "replace", "path": "/spec/template/metadata/labels/test", "value":"test2"}]' -n kruise
[root@node01 ~]# kubectl get pod -n kruise -w
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-69557bfdb8-hzplg 1/1 Running 0 44m
nginx-87kvs 1/1 Running 0 14s
nginx-jqzjd 1/1 Running 0 14s
nginx-s228t 1/1 Running 0 14s
nginx-jqzjd 1/1 Running 0 50s
nginx-jqzjd 1/1 Running 0 50s
nginx-jqzjd 1/1 Running 0 50s
nginx-jqzjd 0/1 Completed 0 54s
nginx-jqzjd 1/1 Running 1 55s
nginx-jqzjd 1/1 Running 1 55s
nginx-jqzjd 1/1 Running 1 55s
nginx-jqzjd 1/1 Running 1 57s
paused
ads-updateStrategy-rollingUpdate-paused.yaml
apiVersion: apps.kruise.io/v1alpha1
kind: DaemonSet
metadata:
labels:
app: nginx
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
test: test
spec:
containers:
- name: nginx
image: nginx:alpine
env:
- name: test
valueFrom:
fieldRef:
fieldPath: metadata.labels['test']
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 20%
partition: 2
paused: true
rollingUpdateType: InPlaceIfPossible
kubectl patch ads nginx --type='json' -p='[{"op": "replace", "path": "/spec/template/metadata/labels/test", "value":"test2"}]' -n kruise
kubectl patch ads nginx --type='json' -p='[{"op": "replace", "path": "/spec/updateStrategy/rollingUpdate/paused", "value":false}]' -n kruise
[root@node01 ~]# kubectl get pod -n kruise -w
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-69557bfdb8-hzplg 1/1 Running 0 48m
nginx-kk6x4 1/1 Running 0 19s
nginx-t2275 1/1 Running 0 19s
nginx-tj54p 1/1 Running 0 19s
nginx-t2275 1/1 Running 0 53s
nginx-t2275 1/1 Running 0 53s
nginx-t2275 1/1 Running 0 53s
nginx-t2275 0/1 Completed 0 57s
nginx-t2275 1/1 Running 1 58s
nginx-t2275 1/1 Running 1 58s
nginx-t2275 1/1 Running 1 58s
nginx-t2275 1/1 Running 1 60s
rollingUpdateType
Standard
ads-updateStrategy-rollingUpdate-rollingUpdateType.yaml
apiVersion: apps.kruise.io/v1alpha1
kind: DaemonSet
metadata:
labels:
app: nginx
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:alpine
updateStrategy:
type: RollingUpdate
rollingUpdate:
selector:
matchLabels:
nodeType: canary
rollingUpdateType: Standard
kubectl label node 192.168.229.128 nodeType=canary
kubectl label node 192.168.229.129 nodeType=canary
kubectl patch ads nginx --type='json' -p='[{"op": "replace", "path": "/spec/template/spec/containers/0/image", "value":"nginx:latest"}]' -n kruise
[root@node01 ~]# kubectl get pod -n kruise -w -owide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nfs-client-provisioner-69557bfdb8-hzplg 1/1 Running 0 60m 172.20.0.133 192.168.229.128 <none> <none>
nginx-2hq88 1/1 Running 0 3s 172.20.1.247 192.168.229.129 <none> 1/1
nginx-9fz76 1/1 Running 0 3s 172.20.0.143 192.168.229.128 <none> 1/1
nginx-xr945 1/1 Running 0 3s 172.20.2.103 192.168.229.130 <none> 1/1
nginx-2hq88 1/1 Terminating 0 10s 172.20.1.247 192.168.229.129 <none> 1/1
nginx-2hq88 0/1 Terminating 0 11s 172.20.1.247 192.168.229.129 <none> 1/1
nginx-2hq88 0/1 Terminating 0 20s 172.20.1.247 192.168.229.129 <none> 1/1
nginx-2hq88 0/1 Terminating 0 20s 172.20.1.247 192.168.229.129 <none> 1/1
nginx-r6nbx 0/1 Pending 0 0s <none> <none> <none> 0/1
nginx-r6nbx 0/1 Pending 0 0s <none> <none> <none> 1/1
nginx-r6nbx 0/1 Pending 0 0s <none> 192.168.229.129 <none> 1/1
nginx-r6nbx 0/1 ContainerCreating 0 0s <none> 192.168.229.129 <none> 1/1
nginx-r6nbx 1/1 Running 0 16s 172.20.1.248 192.168.229.129 <none> 1/1
nginx-9fz76 1/1 Terminating 0 36s 172.20.0.143 192.168.229.128 <none> 1/1
nginx-r6nbx 1/1 Running 0 17s 172.20.1.248 192.168.229.129 <none> 1/1
nginx-9fz76 0/1 Terminating 0 37s 172.20.0.143 192.168.229.128 <none> 1/1
nginx-9fz76 0/1 Terminating 0 38s 172.20.0.143 192.168.229.128 <none> 1/1
nginx-9fz76 0/1 Terminating 0 38s 172.20.0.143 192.168.229.128 <none> 1/1
nginx-n584w 0/1 Pending 0 0s <none> <none> <none> 0/1
nginx-n584w 0/1 Pending 0 0s <none> 192.168.229.128 <none> 0/1
nginx-n584w 0/1 ContainerCreating 0 0s <none> 192.168.229.128 <none> 0/1
nginx-n584w 0/1 ContainerCreating 0 0s <none> 192.168.229.128 <none> 1/1
nginx-n584w 1/1 Running 0 16s 172.20.0.144 192.168.229.128 <none> 1/1
nginx-n584w 1/1 Running 0 17s 172.20.0.144 192.168.229.128 <none> 1/1
Surging
ads-updateStrategy-rollingUpdate-rollingUpdateType-Surging.yaml
apiVersion: apps.kruise.io/v1alpha1
kind: DaemonSet
metadata:
labels:
app: nginx
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:alpine
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 3
partition: 2
rollingUpdateType: Surging
kubectl patch ads nginx --type='json' -p='[{"op": "replace", "path": "/spec/template/spec/containers/0/image", "value":"nginx:latest"}]' -n kruise
[root@node01 ~]# kubectl get pod -n kruise -w
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-69557bfdb8-hzplg 1/1 Running 0 64m
nginx-c58rv 1/1 Running 0 6s
nginx-gdxhh 1/1 Running 0 6s
nginx-ngwl2 1/1 Running 0 6s
nginx-bt6j6 0/1 Pending 0 0s
nginx-bt6j6 0/1 Pending 0 0s
nginx-bt6j6 0/1 Pending 0 0s
nginx-bt6j6 0/1 ContainerCreating 0 0s
nginx-bt6j6 1/1 Running 0 2s
nginx-gdxhh 1/1 Terminating 0 28s
nginx-bt6j6 1/1 Running 0 3s
nginx-gdxhh 0/1 Terminating 0 30s
nginx-gdxhh 0/1 Terminating 0 36s
nginx-gdxhh 0/1 Terminating 0 36s
InPlaceIfPossible
ads-updateStrategy-rollingUpdate-rollingUpdateType-InPlaceIfPossible.yaml
apiVersion: apps.kruise.io/v1alpha1
kind: DaemonSet
metadata:
labels:
app: nginx
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:alpine
updateStrategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 3
partition: 1
rollingUpdateType: InPlaceIfPossible
kubectl patch ads nginx --type='json' -p='[{"op": "replace", "path": "/spec/template/spec/containers/0/image", "value":"nginx:latest"}]' -n kruise
[root@node01 ~]# kubectl get pod -n kruise -w
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-69557bfdb8-hzplg 1/1 Running 0 68m
nginx-4zl4n 1/1 Running 0 5s
nginx-qjl5s 1/1 Running 0 5s
nginx-qzc49 1/1 Running 0 5s
nginx-qjl5s 1/1 Running 0 15s
nginx-qzc49 1/1 Running 0 15s
nginx-qjl5s 1/1 Running 0 15s
nginx-qzc49 1/1 Running 0 15s
nginx-qzc49 1/1 Running 0 15s
nginx-qjl5s 1/1 Running 0 15s
nginx-qzc49 1/1 Running 1 16s
nginx-qzc49 1/1 Running 1 16s
nginx-qjl5s 1/1 Running 1 17s
nginx-qjl5s 1/1 Running 1 17s
nginx-qzc49 1/1 Running 1 17s
nginx-qjl5s 1/1 Running 1 18s
nginx-qzc49 1/1 Running 1 18s
nginx-qjl5s 1/1 Running 1 19s
selector
ads-updateStrategy-rollingUpdate-selector.yaml
apiVersion: apps.kruise.io/v1alpha1
kind: DaemonSet
metadata:
labels:
app: nginx
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
test: test
spec:
containers:
- name: nginx
image: nginx:alpine
env:
- name: test
valueFrom:
fieldRef:
fieldPath: metadata.labels['test']
updateStrategy:
type: RollingUpdate
rollingUpdate:
selector:
matchLabels:
nodeType: canary
partition: 2
maxUnavailable: 3
rollingUpdateType: InPlaceIfPossible
kubectl label node 192.168.229.128 nodeType=canary --overwrite
kubectl label node 192.168.229.129 nodeType=canary --overwrite
kubectl patch ads nginx --type='json' -p='[{"op": "replace", "path": "/spec/template/metadata/labels/test", "value":"test2"}]' -n kruise
[root@node01 ~]# kubectl get pod -n kruise -w
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-69557bfdb8-hzplg 1/1 Running 0 73m
nginx-bxkvs 1/1 Running 0 4s
nginx-l48jb 1/1 Running 0 4s
nginx-s4259 1/1 Running 0 4s
nginx-s4259 1/1 Running 0 8s
nginx-s4259 1/1 Running 0 8s
nginx-s4259 1/1 Running 0 8s
nginx-s4259 0/1 Completed 0 13s
nginx-s4259 1/1 Running 1 14s
nginx-s4259 1/1 Running 1 14s
nginx-s4259 1/1 Running 1 14s
nginx-s4259 1/1 Running 1 16s
type
RollingUpdate
略
OnDelete
ads-updateStrategy-type-OnDelete.yaml
apiVersion: apps.kruise.io/v1alpha1
kind: DaemonSet
metadata:
labels:
app: nginx
name: nginx
spec:
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
test: test
spec:
containers:
- name: nginx
image: nginx:alpine
env:
- name: test
valueFrom:
fieldRef:
fieldPath: metadata.labels['test']
updateStrategy:
type: OnDelete
kubectl patch ads nginx --type='json' -p='[{"op": "replace", "path": "/spec/template/spec/containers/0/image", "value":"nginx:latest"}]' -n kruise
kubectl delete pod -n kruise nginx-bj4x5
[root@node01 ~]# kubectl get pod -n kruise -w
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-69557bfdb8-hzplg 1/1 Running 0 77m
nginx-89hqj 1/1 Running 0 60s
nginx-bj4x5 1/1 Running 0 60s
nginx-r8p7n 1/1 Running 0 60s
nginx-bj4x5 1/1 Terminating 0 63s
nginx-bj4x5 0/1 Terminating 0 64s
nginx-bj4x5 0/1 Terminating 0 65s
nginx-bj4x5 0/1 Terminating 0 65s
nginx-q4rmt 0/1 Pending 0 0s
nginx-q4rmt 0/1 Pending 0 0s
nginx-q4rmt 0/1 Pending 0 0s
nginx-q4rmt 0/1 ContainerCreating 0 0s
nginx-q4rmt 1/1 Running 0 16s
nginx-q4rmt 1/1 Running 0 17s
