k8s集群中的ingress---基于traefik

原创

dongyali521521 2019-08-21 08:48:17 ©著作权

©著作权归作者所有：来自51CTO博客作者dongyali521521的原创作品，请联系作者获取转载授权，否则将追究法律责任

为了对外发布pod内的应用，k8s支持两种负载均衡机制 1、一种是service，用于实现四层TCP负载均衡 service主要实现集群内部通信，以及基于四层的内外通信（如端口） 2、另一种是ingress，用户实现七层HTTP负载均衡 ingress主要实现基于七层的内外通信（如URL） ingress仅仅是一组路由规则的集合，它需要借助ingress控制器才能发挥作用 ingress控制器不受controller-manager管理，它作为一个附件直接运行在k8s集群上 ingress控制器本身也是以pod形式运行，它与被代理的pod运行在同一个网络和service不同的是，要使用ingress，必须先创建ingress-controller这个pod和基于该pod的svc 对于小规模的应用我们使用 NodePort 或许能够满足我们的需求，但是当你的应用越来越多的时候，你就会发现对于 NodePort 的管理就非常麻烦了，这个时候使用 ingress 就非常方便了，可以避免管理大量的 Port。

igress类型 1、单service资源型 2、基于URL路径进行转发 3、基于虚拟主机进行转发 4、TLS类型 ingress控制器可以由如下反向代理程序实现： 1、haproxy 2、nginx 3、envoy 4、traefik 5、Vulcand

创建基于treafik的ingress 1、创建rbac认证

apiVersion: v1
kind: ServiceAccount
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
  name: traefik-ingress-controller
  namespace: kube-system

$ kubectl create -f rbac.yaml serviceaccount "traefik-ingress-controller" created clusterrole.rbac.authorization.k8s.io "traefik-ingress-controller" created clusterrolebinding.rbac.authorization.k8s.io "traefik-ingress-controller" created

2、创建基于treafik的ingress控制器pod及svc 将该控制器pod部署在master上 $ docker pull traefik $ vim traefik.yaml

kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
  labels:
    k8s-app: traefik-ingress-lb
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: traefik-ingress-lb
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
      tolerations:
      - operator: "Exists"        #允许污点
      nodeSelector:
        kubernetes.io/hostname: master        #部署在master上
      containers:
      - image: traefik
        name: traefik-ingress-lb
        ports:
        - name: http
          containerPort: 80
					hostPort: 80        #外网访问时不用使用nodePort端口，直接使用域名即可
        - name: admin
          containerPort: 8080
        args:
        - --api
        - --kubernetes
        - --logLevel=INFO
---
kind: Service
apiVersion: v1
metadata:
  name: traefik-ingress-service
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
    - protocol: TCP
      port: 80
      name: web
    - protocol: TCP
      port: 8080
      name: admin
  type: NodePort

因为traefik容器中有两个端口，80和8080（管理端口），所以其对应的服务中也需要两个端口80和8080. $ kubectl apply -f traefik.yaml deployment.extensions "traefik-ingress-controller" created service "traefik-ingress-service" created $ kubectl get svc -n kube-system traefik-ingress-service NodePort 10.100.222.78 <none> 80:31657/TCP,8080:31572/TCP 79d 通过svc访问traefik的管理界面 http://192.168.1.243:31572/

3、为上述ingress控制器及其svc本身（8080）创建ingress实例

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: traefik-web-ui
  namespace: kube-system
  annotations:
    kubernetes.io/ingress.class: traefik
spec:
  rules:
  - host: traefik.example.com
    http:
      paths:
      - backend:
          serviceName: traefik-ingress-service
          servicePort: 8080

模拟dns解析 $ vim /etc/hosts 192.168.1.243 traefik.example.com 因为pod中有hostPort: 80 ，所以能够以ingress的方式直接使用域名访问traefik的管理界面 https://traefik.example.com 如果你有多个master的话，可以在每个master上部署一个 ingress-controller 服务，然后在master前面挂一个负载均衡器，比如 nginx，将所有的master均作为这个负载均衡器的后端，这样就可以实现 ingress-controller 的高可用和负载均衡了。

4、定义后端普通应用pod及其svc svc的type为ClusterIP

kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: svc1
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: svc1
    spec:
      containers:
      - name: svc1
        image: cnych/example-web-service
        env:
        - name: APP_SVC
          value: svc1
        ports:
        - containerPort: 8080
          protocol: TCP
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: svc2
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: svc2
    spec:
      containers:
      - name: svc2
        image: cnych/example-web-service
        env:
        - name: APP_SVC
          value: svc2
        ports:
        - containerPort: 8080
          protocol: TCP
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: svc3
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: svc3
    spec:
      containers:
      - name: svc3
        image: cnych/example-web-service
        env:
        - name: APP_SVC
          value: svc3
        ports:
        - containerPort: 8080
          protocol: TCP
---
kind: Service
apiVersion: v1
metadata:
  labels:
    app: svc1
  name: svc1
spec:
  type: ClusterIP
  ports:
  - port: 8080
    name: http
  selector:
    app: svc1
---
kind: Service
apiVersion: v1
metadata:
  labels:
    app: svc2
  name: svc2
spec:
  type: ClusterIP
  ports:
  - port: 8080
    name: http
  selector:
    app: svc2
---
kind: Service
apiVersion: v1
metadata:
  labels:
    app: svc3
  name: svc3
spec:
  type: ClusterIP
  ports:
  - port: 8080
    name: http
  selector:
    app: svc3

$ kubectl create -f backend.yaml deployment.extensions "svc1" created deployment.extensions "svc2" created deployment.extensions "svc3" created service "svc1" created service "svc2" created service "svc3" created

5、为上述普通应用pod及其svc定义ingress策略 ingress策略的后端是应用pod的svc

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: example-web-app
  annotations:
    kubernetes.io/ingress.class: "traefik"
spec:
  rules:
  - host: www.example.com
    http:
      paths:
      - path: /s1
        backend:
          serviceName: svc1
          servicePort: 8080
      - path: /s2
        backend:
          serviceName: svc2
          servicePort: 8080
      - path: /
        backend:
          serviceName: svc3
          servicePort: 8080

$ kubectl create -f example-ingress.yaml ingress.extensions "example-web-app" created $ kubectl get ingress $ kubectl describe ingress example-web-app 模拟dns $ vim /etc/hosts 192.168.1.243 www.example.com http://www.example.com ---访问svc3 http://www.example.com/s1 ---访问svc1 http://www.example.com/s2 ---访问svc2

6、使traefik ingress支持TLS 要使其支持tls需要三个方面的支持一、生成ca证书 $ mkdir /ssl $ cd /ssl $ openssl req -newkey rsa:2048 -nodes -keyout tls.key -x509 -days 365 -out tls.crt $ ls tls.crt tls.key 然后创建secret用于存储证书 $ kubectl create secret generic traefik-cert --from-file=tls.crt --from-file=tls.key -n kube-system $ kubectl get secret -n kube-system |grep traefik 二、增加默认配置文件traefik.toml 该文件和traefik pod文件在同一个目录 $ vim traefik.toml

defaultEntryPoints = ["http", "https"]

[entryPoints]
  [entryPoints.http]
  address = ":80"
    [entryPoints.http.redirect]
      entryPoint = "https"
  [entryPoints.https]
  address = ":443"
    [entryPoints.https.tls]
      [[entryPoints.https.tls.certificates]]
      CertFile = "/ssl/tls.crt"
      KeyFile = "/ssl/tls.key"

创建configmap用于存储该配置文件 $ kubectl create configmap traefik-conf --from-file=traefik.toml -n kube-system $ kubectl get configmap -n kube-system |grep traefik 三、修改第2步中的 traefik pod 的 yaml文件 $ vim traefik.yaml

kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: traefik-ingress-controller
  namespace: kube-system
  labels:
    k8s-app: traefik-ingress-lb
spec:
  replicas: 1
  selector:
    matchLabels:
      k8s-app: traefik-ingress-lb
  template:
    metadata:
      labels:
        k8s-app: traefik-ingress-lb
        name: traefik-ingress-lb
    spec:
      serviceAccountName: traefik-ingress-controller
      terminationGracePeriodSeconds: 60
      volumes:
      - name: ssl
        secret:
          secretName: traefik-cert
      - name: config
        configMap:
          name: traefik-conf
      tolerations:
      - operator: "Exists"
      nodeSelector:
        kubernetes.io/hostname: master
      containers:
      - image: traefik
        name: traefik-ingress-lb
        volumeMounts:
        - mountPath: "/ssl"
          name: "ssl"
        - mountPath: "/config"
          name: "config"
        ports:
        - name: http
          containerPort: 80
          hostPort: 80
        - name: https
          containerPort: 443
          hostPort: 443
        - name: admin
          containerPort: 8080
        args:
        - --configfile=/config/traefik.toml
        - --api
        - --kubernetes
        - --logLevel=INFO

$ kubectl apply -f traefik.yaml $ kubectl logs -f traefik-ingress-controller-7dcfd9c6df-v58k7 -n kube-system time="2018-08-26T11:26:44Z" level=info msg="Server configuration reloaded on :80" time="2018-08-26T11:26:44Z" level=info msg="Server configuration reloaded on :443" time="2018-08-26T11:26:44Z" level=info msg="Server configuration reloaded on :8080"