kubectl 查看 ingress 参数配置 kubernetes ingress controller

转载

level 2024-04-14 22:14:14

kubernetes

1. service
2. Service 是由 kube-proxy 组件，加上 iptables 来共同实现
3. 集群内部访问
4. IPVS模式的service，可以使K8s集群支持更多量级的Pod
5. Flannel vxlan模式跨主机通信原理
6. 外部访问
7. Kubernetes 提供了一个 DNS 插件 Service,通过解析访问
8.headless service（无头服务）
9. Pod滚动更新后，依然可以解析
10. LoadBalancer 类型的 Service
11. 从外部访问的第三种方式叫做ExternalName(pod访问集群外部资源)
12. service允许为其分配一个公有IP
13. ingress控制器
14. 让ingress-nginx访问内部服务

1. service

Service可以看作是一组提供相同服务的Pod对外的访问接口。借助Service，应用可以方便地实现服务发现和负载均衡。

service默认只支持4层负载均衡能力，没有7层功能。（可以通过Ingress实现）

service的类型：
ClusterIP：默认值，k8s系统给service自动分配的虚拟IP，只能在集群内部访问。
NodePort：将Service通过指定的Node上的端口暴露给外部，访问任意一个NodeIP:nodePort都将路由到ClusterIP。
LoadBalancer：在 NodePort 的基础上，借助 cloud provider 创建一个外部的负载均衡器，并将请求转发到 :NodePort，此模式只能在云服务器上使用。
ExternalName：将服务通过 DNS CNAME 记录方式转发到指定的域名（通过 spec.externlName 设定）。

2. Service 是由 kube-proxy 组件，加上 iptables 来共同实现

kube-proxy 通过 iptables 处理 Service 的过程，需要在宿主机上设置相当多的 iptables 规则，如果宿主机有大量的Pod，不断刷新iptables规则，会消耗大量的CPU资源。

IPVS模式的service，可以使K8s集群支持更多量级的Pod。

开启kube-proxy的ipvs模式：

# yum install -y ipvsadm 			//所有节点安装

$ kubectl edit cm kube-proxy -n kube-system	//修改IPVS模式
mode: "ipvs"

$ kubectl get pod -n kube-system |grep kube-proxy | awk '{system("kubectl delete pod "$1" -n kube-system")}'		//更新kube-proxy pod

3. 集群内部访问

[kubeadm@server2 manifest]$ cat deployment.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: myapp:v1
        ports:
        - containerPort: 80

[kubeadm@server2 manifest]$ cat service.yaml 
kind: Service
apiVersion: v1
metadata:
  name: myservice
spec:
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  selector:
    app: nginx

[kubeadm@server2 manifest]$ kubectl get pod -o wide
NAME                                READY   STATUS    RESTARTS   AGE   IP            NODE      NOMINATED NODE   READINESS GATES
nginx-deployment-5c58fb7c46-fm54f   1/1     Running   0          15m   10.244.1.20   server3   <none>           <none>
nginx-deployment-5c58fb7c46-qxqbr   1/1     Running   0          15m   10.244.2.22   server4   <none>           <none>

[kubeadm@server2 manifest]$ kubectl get svc 
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP   2d20h
myservice    ClusterIP   10.106.59.243   <none>        80/TCP    3m18s

[kubeadm@server2 manifest]$ kubectl describe svc myservice 
Name:              myservice
Namespace:         default
Labels:            <none>
Annotations:       Selector:  app=nginx
Type:              ClusterIP
IP:                10.106.59.243
Port:              <unset>  80/TCP
TargetPort:        80/TCP
Endpoints:         10.244.1.20:80,10.244.2.22:80
Session Affinity:  None
Events:            <none>

[kubeadm@server2 manifest]$ kubectl run test -it --image=busyboxplus
再次进入：[kubeadm@server2 manifest]$ kubectl attach -it test 
/ # curl 10.106.59.243
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>

[root@server3 ~]# iptables -t nat -nL|grep 10.106.59.243
KUBE-MARK-MASQ  tcp  -- !10.244.0.0/16        10.106.59.243        /* default/myservice: cluster IP */ tcp dpt:80
KUBE-SVC-DN4K6DJYBW27OJYO  tcp  --  0.0.0.0/0            10.106.59.243        /* default/myservice: cluster IP */ tcp dpt:80

4. IPVS模式的service，可以使K8s集群支持更多量级的Pod

在server2、3、4节点上安装ipvsadm

yum install -y ipvsadm

[kubeadm@server2 ~]$ kubectl -n kube-system get cm
NAME                                 DATA   AGE
coredns                              1      2d20h
extension-apiserver-authentication   6      2d20h
kube-flannel-cfg                     2      2d19h
kube-proxy                           2      2d20h
kubeadm-config                       2      2d20h
kubelet-config-1.18                  1      2d20h

[kubeadm@server2 ~]$ kubectl -n kube-system edit cm kube-proxy 
 43     mode: "ipvs"

更新kube-proxy pod

[kubeadm@server2 ~]$ kubectl get pod -n kube-system |grep kube-proxy | awk '{system("kubectl delete pod "$1" -n kube-system")}'

[root@server3 ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.96.0.1:443 rr
  -> 172.25.60.2:6443             Masq    1      0          0         
TCP  10.96.0.10:53 rr
  -> 10.244.0.6:53                Masq    1      0          0         
  -> 10.244.0.7:53                Masq    1      0          0         
TCP  10.96.0.10:9153 rr
  -> 10.244.0.6:9153              Masq    1      0          0         
  -> 10.244.0.7:9153              Masq    1      0          0         
TCP  10.106.59.243:80 rr
  -> 10.244.1.20:80               Masq    1      0          0         
  -> 10.244.2.22:80               Masq    1      0          0         
UDP  10.96.0.10:53 rr
  -> 10.244.0.6:53                Masq    1      0          0         
  -> 10.244.0.7:53                Masq    1      0          0

5. Flannel vxlan模式跨主机通信原理

kubectl 查看 ingress 参数配置 kubernetes ingress controller_nginx

6. 外部访问

方式一：

[kubeadm@server2 ~]$ kubectl edit svc myservice
53   type: NodePort

[kubeadm@server2 ~]$ kubectl get svc
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP        2d21h
myservice    NodePort    10.106.59.243   <none>        80:31701/TCP   93m  # 开启一个对外端口

[kubeadm@server2 ~]$ kubectl get pod -o wide
NAME                                READY   STATUS    RESTARTS   AGE    IP            NODE      NOMINATED NODE   READINESS GATES
nginx-deployment-5c58fb7c46-fm54f   1/1     Running   0          108m   10.244.1.20   server3   <none>           <none>
nginx-deployment-5c58fb7c46-qxqbr   1/1     Running   0          108m   10.244.2.22   server4   <none>           <none>
test                                1/1     Running   4          107m   10.244.1.21   server3   <none>           <none>

[root@server3 ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  172.25.60.3:31701 rr
  -> 10.244.1.20:80               Masq    1      0          0         
  -> 10.244.2.22:80               Masq    1      0          0

kubectl 查看 ingress 参数配置 kubernetes ingress controller_IP_02

方式二：直接在yaml文件中指定type：NodePort

[kubeadm@server2 manifest]$ cat service.yaml 
kind: Service
apiVersion: v1
metadata:
  name: myservice
spec:
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  selector:
    app: nginx
  type: NodePort

[kubeadm@server2 manifest]$ kubectl apply -f service.yaml 
service/myservice created

[kubeadm@server2 manifest]$ kubectl get svc
NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
kubernetes   ClusterIP   10.96.0.1        <none>        443/TCP        2d22h
myservice    NodePort    10.109.224.100   <none>        80:32480/TCP   7s

[kubeadm@server2 manifest]$ kubectl describe svc myservice 
Name:                     myservice
Namespace:                default
Labels:                   <none>
Annotations:              Selector:  app=nginx
Type:                     NodePort
IP:                       10.109.224.100
Port:                     <unset>  80/TCP
TargetPort:               80/TCP
NodePort:                 <unset>  32480/TCP
Endpoints:                10.244.1.20:80,10.244.2.22:80
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

[root@server4 ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  172.25.60.4:32480 rr
  -> 10.244.1.20:80               Masq    1      0          0         
  -> 10.244.2.22:80               Masq    1      0          1        

[kiosk@foundation60 ~]$ curl 172.25.60.4:32480
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>

7. Kubernetes 提供了一个 DNS 插件 Service,通过解析访问

[kubeadm@server2 manifest]$ kubectl get services kube-dns --namespace=kube-system
NAME       TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)                  AGE
kube-dns   ClusterIP   10.96.0.10(dns的ip地址)   <none>        53/UDP,53/TCP,9153/TCP   3d23h

[kubeadm@server2 manifest]$ cat service.yaml 
kind: Service
apiVersion: v1
metadata:
  name: myservice
spec:
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  selector:
    app: nginx
  type: ClusterIP 

[kubeadm@server2 manifest]$ kubectl get svc
NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1        <none>        443/TCP   3d23h
myservice    ClusterIP   10.106.210.179   <none>        80/TCP    15s

[kubeadm@server2 manifest]$ kubectl attach test -it
Defaulting container name to test.
Use 'kubectl describe pod/test -n default' to see all of the containers in this pod.
If you don't see a command prompt, try pressing enter.
/ # nslookup myservice
Server:    10.96.0.10
Address 1: 10.96.0.10
nslookup: can't resolve 'myservice'（kube-dns有问题）
/ # cat /etc/resolv.conf 
nameserver 10.96.0.10   # 使用集群kube-dns的地址
search default.svc.cluster.local svc.cluster.local cluster.local
options ndots:5

正确形式为：

kubectl 查看 ingress 参数配置 kubernetes ingress controller_IP_03

kubectl 查看 ingress 参数配置 kubernetes ingress controller_TCP_04

8.headless service（无头服务）

Headless Service不需要分配一个VIP，而是直接以DNS记录的方式解析出被代理Pod的IP地址。
域名格式： $kubectl 查看 ingress 参数配置 kubernetes ingress controller_TCP_05$ (namespace).svc.cluster.local

[kubeadm@server2 manifest]$ cat service.yaml 
kind: Service
apiVersion: v1
metadata:
  name: myservice
spec:
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  selector:
    app: nginx
  clusterIP: None

[kubeadm@server2 manifest]$ kubectl apply -f service.yaml 
service/myservice created

没有ip地址

[kubeadm@server2 manifest]$ kubectl get svc
NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   4d1h
myservice    ClusterIP   None         <none>        80/TCP    11s

[kubeadm@server2 manifest]$ kubectl describe svc myservice 
Name:              myservice
Namespace:         default
Labels:            <none>
Annotations:       Selector:  app=nginx
Type:              ClusterIP
IP:                None
Port:              <unset>  80/TCP
TargetPort:        80/TCP
Endpoints:         10.244.1.25:80,10.244.2.24:80
Session Affinity:  None
Events:            <none>

[kubeadm@server2 manifest]$ kubectl attach test -it
Defaulting container name to test.
Use 'kubectl describe pod/test -n default' to see all of the containers in this pod.
If you don't see a command prompt, try pressing enter.
/ # nslookup myservice

kubectl 查看 ingress 参数配置 kubernetes ingress controller_nginx_06

kubectl 查看 ingress 参数配置 kubernetes ingress controller_TCP_07

[kubeadm@server2 manifest]$ kubectl run demo -it --image=bind-utils
If you don't see a command prompt, try pressing enter.
bash-4.3# dig myservice.default.svc.cluster.local

kubectl 查看 ingress 参数配置 kubernetes ingress controller_TCP_08

9. Pod滚动更新后，依然可以解析

[kubeadm@server2 manifest]$ kubectl delete -f deployment.yaml

[kubeadm@server2 manifest]$ cat deployment.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: myapp:v2
        ports:
        - containerPort: 80
[kubeadm@server2 manifest]$ kubectl apply -f deployment.yaml 
deployment.apps/nginx-deployment created

[kubeadm@server2 manifest]$ kubectl attach demo -it

kubectl 查看 ingress 参数配置 kubernetes ingress controller_nginx_09

10. LoadBalancer 类型的 Service

从外部访问 Service 的第二种方式，适用于公有云上的 Kubernetes 服务。这时候，你可以指定一个 LoadBalancer 类型的 Service。

[kubeadm@server2 manifest]$ kubectl delete -f service.yaml 
service "myservice" deleted

[kubeadm@server2 manifest]$ cat service.yaml 
kind: Service
apiVersion: v1
metadata:
  name: myservice
spec:
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer

[kubeadm@server2 manifest]$ kubectl apply -f service.yaml 
[kubeadm@server2 manifest]$ kubectl get svc
NAME         TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
kubernetes   ClusterIP      10.96.0.1       <none>        443/TCP        4d2h
myservice    LoadBalancer   10.110.23.204   <pending>     80:31744/TCP   28s

11. 从外部访问的第三种方式叫做ExternalName(pod访问集群外部资源)

[kubeadm@server2 manifest]$ cat service.yaml 
kind: Service
apiVersion: v1
metadata:
  name: myservice
spec:
   type:  ExternalName
   externalName: www.westos.org

[kubeadm@server2 manifest]$ kubectl delete -f service.yaml 
service "myservice" deleted

[kubeadm@server2 manifest]$ kubectl apply -f service.yaml 
service/myservice created

[kubeadm@server2 manifest]$ kubectl get svc
NAME         TYPE           CLUSTER-IP   EXTERNAL-IP      PORT(S)   AGE
kubernetes   ClusterIP      10.96.0.1    <none>           443/TCP   4d2h
myservice    ExternalName   <none>       www.westos.org   <none>    5s

[kubeadm@server2 manifest]$ kubectl attach demo -it
Defaulting container name to demo.
Use 'kubectl describe pod/demo -n default' to see all of the containers in this pod.
If you don't see a command prompt, try pressing enter.
bash-4.3# dig myservice.default.svc.cluster.local

kubectl 查看 ingress 参数配置 kubernetes ingress controller_nginx_10

kubectl 查看 ingress 参数配置 kubernetes ingress controller_IP_11

12. service允许为其分配一个公有IP

[kubeadm@server2 manifest]$ kubectl delete -f service.yaml 
service "myservice" deleted

[kubeadm@server2 manifest]$ cat service.yaml 
kind: Service
apiVersion: v1
metadata:
  name: myservice
spec:
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer
  externalIPs:
    - 172.25.60.10

[kubeadm@server2 manifest]$ kubectl apply -f service.yaml 
service/myservice created
[kubeadm@server2 manifest]$ kubectl get svc
NAME         TYPE           CLUSTER-IP     EXTERNAL-IP    PORT(S)        AGE
kubernetes   ClusterIP      10.96.0.1      <none>         443/TCP        4d2h
myservice    LoadBalancer   10.96.65.119   172.25.60.10   80:30742/TCP   10s

[kubeadm@server2 manifest]$ kubectl describe svc myservice 
Name:                     myservice
Namespace:                default
Labels:                   <none>
Annotations:              Selector:  app=nginx
Type:                     LoadBalancer
IP:                       10.96.65.119
External IPs:             172.25.60.10
Port:                     <unset>  80/TCP
TargetPort:               80/TCP
NodePort:                 <unset>  30742/TCP
Endpoints:                10.244.1.27:80,10.244.2.26:80
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

[root@foundation60 kiosk]# curl 172.25.60.10
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>

13. ingress控制器

一种全局的、为了代理不同后端 Service 而设置的负载均衡服务，就是 Kubernetes 里的Ingress 服务。
Ingress由两部分组成：Ingress controller和Ingress服务。

Ingress Controller 会根据你定义的 Ingress 对象，提供对应的代理能力。业界常用的各种反向代理项目，比如 Nginx、HAProxy、Envoy、Traefik 等，都已经为Kubernetes 专门维护了对应的 Ingress Controller。

kubectl 查看 ingress 参数配置 kubernetes ingress controller_TCP_12

应用ingress controller定义文件（mandatory.yaml ）

[kubeadm@server2 manifest]$ cat mandatory.yaml 
apiVersion: v1
kind: Namespace
metadata:
  name: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx


---


kind: ConfigMap
apiVersion: v1
metadata:
  name: nginx-configuration
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx


---
kind: ConfigMap
apiVersion: v1
metadata:
  name: tcp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx


---
kind: ConfigMap
apiVersion: v1
metadata:
  name: udp-services
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx


---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: nginx-ingress-serviceaccount
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx


---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: nginx-ingress-clusterrole
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - events
    verbs:
      - create
      - patch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "extensions"
      - "networking.k8s.io"
    resources:
      - ingresses/status
    verbs:
      - update


---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: nginx-ingress-role
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      # Defaults to "<election-id>-<ingress-class>"
      # Here: "<ingress-controller-leader>-<nginx>"
      # This has to be adapted if you change either parameter
      # when launching the nginx-ingress-controller.
      - "ingress-controller-leader-nginx"
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - get


---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: nginx-ingress-role-nisa-binding
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: nginx-ingress-role
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx


---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: nginx-ingress-clusterrole-nisa-binding
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nginx-ingress-clusterrole
subjects:
  - kind: ServiceAccount
    name: nginx-ingress-serviceaccount
    namespace: ingress-nginx

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx
      app.kubernetes.io/part-of: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
      annotations:
        prometheus.io/port: "10254"
        prometheus.io/scrape: "true"
    spec:
      # wait up to five minutes for the drain of connections
      terminationGracePeriodSeconds: 300
      serviceAccountName: nginx-ingress-serviceaccount
      nodeSelector:
        kubernetes.io/os: linux
      containers:
        - name: nginx-ingress-controller
          image: registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller:0.30.0
          args:
            - /nginx-ingress-controller
            - --configmap=$(POD_NAMESPACE)/nginx-configuration
            - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
            - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
            - --publish-service=$(POD_NAMESPACE)/ingress-nginx
            - --annotations-prefix=nginx.ingress.kubernetes.io
          securityContext:
            allowPrivilegeEscalation: true
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            # www-data -> 101
            runAsUser: 101
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
          ports:
            - name: http
              containerPort: 80
              protocol: TCP
            - name: https
              containerPort: 443
              protocol: TCP
          livenessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 10
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown

---

apiVersion: v1
kind: LimitRange
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  limits:
  - min:
      memory: 90Mi
      cpu: 100m
    type: Container

改为国内镜像

[kubeadm@server2 manifest]$ sed -i 's#quay.io/kubernetes-ingress-controller/nginx-ingress-controller#registry.cn-hangzhou.aliyuncs.com/google_containers/nginx-ingress-controller#g' mandatory.yaml

[kubeadm@server2 manifest]$ kubectl apply -f mandatory.yaml 
namespace/ingress-nginx created
configmap/nginx-configuration created
configmap/tcp-services created
configmap/udp-services created
serviceaccount/nginx-ingress-serviceaccount created
clusterrole.rbac.authorization.k8s.io/nginx-ingress-clusterrole created
role.rbac.authorization.k8s.io/nginx-ingress-role created
rolebinding.rbac.authorization.k8s.io/nginx-ingress-role-nisa-binding created
clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-clusterrole-nisa-binding created
deployment.apps/nginx-ingress-controller created
limitrange/ingress-nginx created

[kubeadm@server2 manifest]$ kubectl get namespaces 
NAME              STATUS   AGE
default           Active   4d4h
ingress-nginx     Active   19s
kube-node-lease   Active   4d4h
kube-public       Active   4d4h
kube-system       Active   4d4h

[kubeadm@server2 manifest]$ kubectl -n ingress-nginx get pod
NAME                                       READY   STATUS              RESTARTS   AGE
nginx-ingress-controller-5994c9595-9wgvd   0/1     ContainerCreating   0          38s

缺少service服务

[kubeadm@server2 manifest]$ kubectl -n ingress-nginx logs nginx-ingress-controller-5994c9595-9wgvd 
ns:map[string]string{},OwnerReferences:[]OwnerReference{},Finalizers:[],ClusterName:,ManagedFields:[]ManagedFieldsEntry{},}, err services "ingress-nginx" not found

应用ingress-service定义文件

[kubeadm@server2 manifest]$ kubectl -n ingress-nginx get pod --show-labels
NAME                                       READY   STATUS    RESTARTS   AGE   LABELS
nginx-ingress-controller-5994c9595-9wgvd   1/1     Running   0          12m   app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/part-of=ingress-nginx,pod-template-hash=5994c9595

[kubeadm@server2 manifest]$ cat service-nodeport.yaml 
apiVersion: v1
kind: Service
metadata:
  name: ingress-nginx
  namespace: ingress-nginx
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
spec:
  type: NodePort
  ports:
    - name: http
      port: 80
      targetPort: 80
      protocol: TCP
    - name: https
      port: 443
      targetPort: 443
      protocol: TCP
  selector:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx


---

[kubeadm@server2 manifest]$ kubectl apply -f service-nodeport.yaml 
service/ingress-nginx created

[kubeadm@server2 manifest]$ kubectl -n ingress-nginx get svc
NAME            TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx   NodePort   10.106.5.122   <none>        80:31188/TCP,443:31595/TCP   35s

[kubeadm@server2 manifest]$ kubectl -n ingress-nginx describe svc ingress-nginx 
Name:                     ingress-nginx
Namespace:                ingress-nginx
Labels:                   app.kubernetes.io/name=ingress-nginx
                          app.kubernetes.io/part-of=ingress-nginx
Annotations:              Selector:  app.kubernetes.io/name=ingress-nginx,app.kubernetes.io/part-of=ingress-nginx
Type:                     NodePort
IP:                       10.106.5.122
Port:                     http  80/TCP
TargetPort:               80/TCP
NodePort:                 http  31188/TCP
Endpoints:                10.244.1.30:80
Port:                     https  443/TCP
TargetPort:               443/TCP
NodePort:                 https  31595/TCP
Endpoints:                10.244.1.30:443
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

14. 让ingress-nginx访问内部服务

创建ingress服务

[kubeadm@server2 manifest]$ cat service.yaml 
kind: Service
apiVersion: v1
metadata:
  name: myservice
spec:
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  selector:
    app: nginx
  type: ClusterIP

[kubeadm@server2 manifest]$ kubectl get svc
NAME         TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
kubernetes   ClusterIP   10.96.0.1      <none>        443/TCP   4d4h
myservice    ClusterIP   10.111.9.105   <none>        80/TCP    14s

让nginx调度myservice

[kubeadm@server2 manifest]$ cat ingress.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-demo
spec:
  backend:
    serviceName: myservice
    servicePort: 80
#  rules:
#  - host: www1.westos.org
#    http:
#      paths:
#      - path: /
#        backend:
#          serviceName: nginx-svc
#          servicePort: 80

[kubeadm@server2 manifest]$ kubectl apply -f ingress.yaml 
ingress.extensions/ingress-demo created

[kubeadm@server2 manifest]$ kubectl get ingress
NAME           CLASS    HOSTS   ADDRESS   PORTS   AGE
ingress-demo   <none>   *                 80      24s

[kubeadm@server2 manifest]$ kubectl describe ingress ingress-demo 
Name:             ingress-demo
Namespace:        default
Address:          10.106.5.122
Default backend:  myservice:80 (10.244.1.27:80,10.244.2.26:80)
Rules:
  Host        Path  Backends
  ----        ----  --------
  *           *     myservice:80 (10.244.1.27:80,10.244.2.26:80)
Annotations:  Events:
  Type        Reason  Age   From                      Message
  ----        ------  ----  ----                      -------
  Normal      CREATE  38s   nginx-ingress-controller  Ingress default/ingress-demo
  Normal      UPDATE  3s    nginx-ingress-controller  Ingress default/ingress-demo

[kubeadm@server2 manifest]$ kubectl -n ingress-nginx get svc
NAME            TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx   NodePort   10.106.5.122   <none>        80:31188/TCP,443:31595/TCP   15m

kubectl 查看 ingress 参数配置 kubernetes ingress controller_TCP_13

kubectl 查看 ingress 参数配置 kubernetes ingress controller_nginx_14

域名访问

[kubeadm@server2 manifest]$ cat ingress.yaml 
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-demo
spec:
  backend:
    serviceName: myservice
    servicePort: 80
  rules:
  - host: www1.westos.org
    http:
      paths:
      - path: /
        backend:
          serviceName: myservice
          servicePort: 80

[kubeadm@server2 manifest]$ kubectl -n ingress-nginx get pod -o wide
NAME                                       READY   STATUS    RESTARTS   AGE   IP            NODE      NOMINATED NODE   READINESS GATES
nginx-ingress-controller-5994c9595-9wgvd   1/1     Running   0          38m   10.244.1.30   server3   <none>           <none>

[kubeadm@server2 manifest]$ kubectl apply -f ingress.yaml 

[kubeadm@server2 manifest]$ kubectl -n ingress-nginx get svc
NAME            TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx   NodePort   10.106.5.122   <none>        80:31188/TCP,443:31595/TCP   27m

kubectl 查看 ingress 参数配置 kubernetes ingress controller_IP_15

[kubeadm@server2 manifest]$ cat ingress.yaml 
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: ingress-demo
spec:
  backend:
    serviceName: myservice
    servicePort: 80
  rules:
  - host: www1.westos.org
    http:
      paths:
      - path: /
        backend:
          serviceName: myservice
          servicePort: 80

[kubeadm@server2 manifest]$ cat ingress2.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-deployment
  labels:
    app: myapp
spec:
  replicas: 2
  selector:
    matchLabels:
      app: myapp
  template:
    metadata:
      labels:
        app: myapp
    spec:
      containers:
      - name: myapp
        image: myapp:v1
        ports:
        - containerPort: 80
---

kind: Service
apiVersion: v1
metadata:
  name: mynginx
spec:
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  selector:
    app: myapp
  type: ClusterIP

---

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: ingress-example
spec:
  rules:
  - host: www2.westos.org
    http:
      paths:
      - path: /
        backend:
          serviceName: mynginx
          servicePort: 80

[kubeadm@server2 manifest]$ kubectl apply -f ingress2.yaml 
deployment.apps/myapp-deployment created
service/mynginx created
ingress.extensions/ingress-example created

[kubeadm@server2 manifest]$ kubectl get ingress
NAME              CLASS    HOSTS             ADDRESS        PORTS   AGE
ingress-demo      <none>   www1.westos.org   10.106.5.122   80      27m
ingress-example   <none>   www2.westos.org   10.106.5.122   80      79s

[kubeadm@server2 manifest]$ kubectl describe ingress ingress-example
Name:             ingress-example
Namespace:        default
Address:          10.106.5.122
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
  Host             Path  Backends
  ----             ----  --------
  www2.westos.org  
                   /   mynginx:80 (10.244.1.31:80,10.244.2.29:80)
Annotations:       Events:
  Type             Reason  Age   From                      Message
  ----             ------  ----  ----                      -------
  Normal           CREATE  99s   nginx-ingress-controller  Ingress default/ingress-example
  Normal           UPDATE  42s   nginx-ingress-controller  Ingress default/ingress-example

[kubeadm@server2 manifest]$ kubectl -n ingress-nginx get svc
NAME            TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)                      AGE
ingress-nginx   NodePort   10.106.5.122   <none>        80:31188/TCP,443:31595/TCP   42m
[kubeadm@server2 manifest]$ kubectl describe ingress ingress-demo 
Name:             ingress-demo
Namespace:        default
Address:          10.106.5.122
Default backend:  myservice:80 (10.244.1.27:80,10.244.2.26:80)
Rules:
  Host             Path  Backends
  ----             ----  --------
  www1.westos.org  
                   /   myservice:80 (10.244.1.27:80,10.244.2.26:80)
Annotations:       Events:
  Type             Reason  Age                From                      Message
  ----             ------  ----               ----                      -------
  Normal           CREATE  29m                nginx-ingress-controller  Ingress default/ingress-demo
  Normal           UPDATE  21m (x2 over 28m)  nginx-ingress-controller  Ingress default/ingress-demo

[root@foundation60 kiosk]# cat /etc/hosts
172.25.60.4 www1.westos.org www2.westos.org

kubectl 查看 ingress 参数配置 kubernetes ingress controller_TCP_16

kubectl 查看 ingress 参数配置 kubernetes ingress controller_TCP_17

[kubeadm@server2 manifest]$ kubectl get svc myservice
NAME        TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)   AGE
myservice   ClusterIP   10.111.9.105   <none>        80/TCP    50m

[kubeadm@server2 manifest]$ kubectl get svc mynginx
NAME      TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)   AGE
mynginx   ClusterIP   10.109.203.127   <none>        80/TCP    21m

[kubeadm@server2 manifest]$ kubectl describe svc myservice 
Name:              myservice
Namespace:         default
Labels:            <none>
Annotations:       Selector:  app=nginx
Type:              ClusterIP
IP:                10.111.9.105
Port:              <unset>  80/TCP
TargetPort:        80/TCP
Endpoints:         10.244.1.27:80,10.244.2.26:80
Session Affinity:  None
Events:            <none>

[kubeadm@server2 manifest]$ kubectl describe svc mynginx
Name:              mynginx
Namespace:         default
Labels:            <none>
Annotations:       Selector:  app=myapp
Type:              ClusterIP
IP:                10.109.203.127
Port:              <unset>  80/TCP
TargetPort:        80/TCP
Endpoints:         10.244.1.31:80,10.244.2.29:80
Session Affinity:  None
Events:            <none>


[kubeadm@server2 manifest]$ cat ingress3.yaml 
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: simple-fanout-example
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - host: www3.westos.org
    http:
      paths:
      - path: /v1
        backend:
          serviceName: mynginx
          servicePort: 80
      - path: /v2
        backend:
          serviceName: myservice
          servicePort: 80

[kubeadm@server2 manifest]$ kubectl apply -f ingress3.yaml 
ingress.networking.k8s.io/simple-fanout-example created

[kubeadm@server2 manifest]$ kubectl describe ingress simple-fanout-example 
Name:             simple-fanout-example
Namespace:        default
Address:          10.106.5.122
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
  Host             Path  Backends
  ----             ----  --------
  www3.westos.org  
                   /v1   mynginx:80 (10.244.1.31:80,10.244.2.29:80)
                   /v2   myservice:80 (10.244.1.27:80,10.244.2.26:80)
Annotations:       nginx.ingress.kubernetes.io/rewrite-target: /
Events:
  Type    Reason  Age   From                      Message
  ----    ------  ----  ----                      -------
  Normal  CREATE  76s   nginx-ingress-controller  Ingress default/simple-fanout-example
  Normal  UPDATE  29s   nginx-ingress-controller  Ingress default/simple-fanout-example

[root@foundation60 kiosk]# cat /etc/hosts
172.25.60.4 www1.westos.org www2.westos.org www3.westos.org

[root@foundation60 kiosk]# curl www3.westos.org:31188/v1
Hello MyApp | Version: v1 | <a href="hostname.html">Pod Name</a>
[root@foundation60 kiosk]# curl www3.westos.org:31188/v2
Hello MyApp | Version: v2 | <a href="hostname.html">Pod Name</a>

本文章为转载内容，我们尊重原作者对文章享有的著作权。如有内容错误或侵权问题，欢迎原作者联系我们进行内容更正或删除文章。