理论了解

1、ingress简介

kubernetes官方文档

Ingress 是 kubernetes API 中的标准资源类型之一，ingress 实现的功能是在应用层对客户端请求的 host 名称或请求的 URL 路径把请求转发到指定的 service 资源的规则，即用于将 kubernetes 集群外部的请求资源转发之集群内部的 service，再被service 转发之pod处理客户端的请求。

2、ingress控制器简介

kubernegtes官方文档

Ingress 资源需要指定监听地址、请求的 host 和 URL 等配置，然后根据这些规则的匹配机制将客户端的请求进行转发，这种能够为 ingress 配置资源监听并转发流量的组件称为ingress 控制器(ingress controller)，ingress controller 是 kubernetes 的一个附件 ，需要单独部署。

3、 Ingress 选型

kubernetes官方文档

4、 Ingress使用逻辑图

k8s集群部分机器，没有把所有节点都画上，意思一下，理解领会精神就行。

使用Ingress完成效果

1、实现多个域名的http请求流量转发

2、http实现url请求流量转发

3、实现多域名的https请求流量转发

4、https实现url请求流量转发

资源准备

1、k8s集群信息

ingress nodeport 测试 ingress filtering_nginx


NAME STATUS ROLES AGE VERSION 192.168.101.100 Ready,SchedulingDisabled master 46h v1.20.2 192.168.101.101 Ready,SchedulingDisabled master 46h v1.20.2 192.168.101.102 Ready,SchedulingDisabled master 46h v1.20.2 192.168.101.103 Ready node 46h v1.20.2 192.168.101.104 Ready node 46h v1.20.2 192.168.101.105 Ready node 46h v1.20.2 192.168.101.106 Ready node 46h v1.20.2


2、公网域名

1、配置公网域名到内网服务器

图省事，直接用*.ingress，给一组业务使用。以后不用一个一个配置test02、test03到04域名

保证服务器有上网功能

ingress nodeport 测试 ingress filtering_容器_02

test01.ingress.opsvv.com #测试app1页面

test02.ingress.opsvv.com #测试app1页面

#重复操作

test03.ingress.opsvv.com #测试app3页面

test04.ingress.opsvv.com #测试app4页面

没有公网域名可以使用主机名 hosts或者powerdns、bind等工具做内部解析

2、测试域名是否生效

测试域名是已经生效的

ping -c 1 test01.ingress.opsvv.com

ping -c 1 test02.ingress.opsvv.com

ping -c 1 test03.ingress.opsvv.com

ping -c 1 test04.ingress.opsvv.com

ingress nodeport 测试 ingress filtering_kubernetes_03

3、公网证书

腾讯云免费证书

ingress nodeport 测试 ingress filtering_docker_04

没有公网证书可以使用Openssl自签名证书

Openssl生成自签名证书

K8s集群配置Ingress

1、Service类型了解

1、ClusterIP

默认的类型，用于 k8s 内部之间的服务访问，即通过内部的service ip 实现服务间的访问，service IP 仅可以在内部访问，不能从外部访问。

2、NodePort

在 cluster IP 的基础之上，通过在每个 node 节点监听一个可以指定宿主机端口(nodePort)来暴露服务，从而允许外部 client 访问 k8s 集群中的服务，nodePort 把外部client的请求转发至 service 进行处理。

3、LoadBalancer

主要在公有云如阿里云、AWS 上使用，LoadBalancer 构建在nodePort 基础之上，通过公有云服务商提供的负载均衡器将 k8s 集群中的服务暴露给集群外部的client访问。

2、部署ingress controller

1、部署 ingress controller

github地址

Nodeport方式部署，了解或不想了解都可以

2、镜像下载地址

镜像在国外不方便下载，已经改成了私有仓库


docker pull imagespull.opsvv.com/kubernetes/ingress/ingress-nginx:v1.2.0 
docker pull imagespull.opsvv.com/kubernetes/ingress/kube-webhook-certgen:v1.1.1 
docker pull imagespull.opsvv.com/kubernetes/ingress/kube-webhook-certgen:v1.1.1


或者浏览器images.tar csdn免费下载

ingress-nginx-controller-v1.2.0-images

3、不废话了，开搞开搞。

使用daemonset方式部署controller，master和node都ingress controller Pod服务

4、使用yaml文件部署

cd /opt/suops/k8s/ingress-files/fanht-ops-nginx-ingress-deploy/

yaml文件需要注意的地方，设置为True,Pod才能把80和443端口监听到宿主机上

ingress nodeport 测试 ingress filtering_kubernetes_05

kubectl apply -f ingress-nginx-controller-v1.2.0_daemonset.yaml


apiVersion: v1
kind: Namespace
metadata:
  labels:
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  name: ingress-nginx
---
apiVersion: v1
automountServiceAccountToken: true
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.2.0
  name: ingress-nginx
  namespace: ingress-nginx
---
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.2.0
  name: ingress-nginx-admission
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.2.0
  name: ingress-nginx
  namespace: ingress-nginx
rules:
- apiGroups:
  - ""
  resources:
  - namespaces
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - configmaps
  - pods
  - secrets
  - endpoints
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - services
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses/status
  verbs:
  - update
- apiGroups:
  - networking.k8s.io
  resources:
  - ingressclasses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resourceNames:
  - ingress-controller-leader
  resources:
  - configmaps
  verbs:
  - get
  - update
- apiGroups:
  - ""
  resources:
  - configmaps
  verbs:
  - create
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - patch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.2.0
  name: ingress-nginx-admission
  namespace: ingress-nginx
rules:
- apiGroups:
  - ""
  resources:
  - secrets
  verbs:
  - get
  - create
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.2.0
  name: ingress-nginx
rules:
- apiGroups:
  - ""
  resources:
  - configmaps
  - endpoints
  - nodes
  - pods
  - secrets
  - namespaces
  verbs:
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - nodes
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - services
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - events
  verbs:
  - create
  - patch
- apiGroups:
  - networking.k8s.io
  resources:
  - ingresses/status
  verbs:
  - update
- apiGroups:
  - networking.k8s.io
  resources:
  - ingressclasses
  verbs:
  - get
  - list
  - watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.2.0
  name: ingress-nginx-admission
rules:
- apiGroups:
  - admissionregistration.k8s.io
  resources:
  - validatingwebhookconfigurations
  verbs:
  - get
  - update
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.2.0
  name: ingress-nginx
  namespace: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-nginx
subjects:
- kind: ServiceAccount
  name: ingress-nginx
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.2.0
  name: ingress-nginx-admission
  namespace: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
  name: ingress-nginx-admission
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.2.0
  name: ingress-nginx
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ingress-nginx
subjects:
- kind: ServiceAccount
  name: ingress-nginx
  namespace: ingress-nginx
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.2.0
  name: ingress-nginx-admission
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ingress-nginx-admission
subjects:
- kind: ServiceAccount
  name: ingress-nginx-admission
  namespace: ingress-nginx
---
apiVersion: v1
data:
  allow-snippet-annotations: "true"
kind: ConfigMap
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.2.0
  name: ingress-nginx-controller
  namespace: ingress-nginx
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.2.0
  name: ingress-nginx-controller-admission
  namespace: ingress-nginx
spec:
  ports:
  - appProtocol: https
    name: https-webhook
    port: 443
    targetPort: webhook
  selector:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  type: ClusterIP
---
apiVersion: apps/v1
#kind: Deployment 将Deployment方式改成DaemonSet
kind: DaemonSet
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.2.0
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  minReadySeconds: 0
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app.kubernetes.io/component: controller
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/name: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/component: controller
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/name: ingress-nginx
    spec:
      hostNetwork: true
      hostPID: true
      containers:
      - args:
        - /nginx-ingress-controller
        - --election-id=ingress-controller-leader
        - --controller-class=k8s.io/ingress-nginx
        - --ingress-class=nginx
        - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
        - --validating-webhook=:8443
        - --validating-webhook-certificate=/usr/local/certificates/cert
        - --validating-webhook-key=/usr/local/certificates/key
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: LD_PRELOAD
          value: /usr/local/lib/libmimalloc.so
        image: imagespull.opsvv.com/kubernetes/ingress/ingress-nginx:v1.2.0 
        imagePullPolicy: IfNotPresent
        lifecycle:
          preStop:
            exec:
              command:
              - /wait-shutdown
        livenessProbe:
          failureThreshold: 5
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        name: controller
        ports:
        - containerPort: 80
          name: http
          protocol: TCP
        - containerPort: 443
          name: https
          protocol: TCP
        - containerPort: 8443
          name: webhook
          protocol: TCP
        readinessProbe:
          failureThreshold: 3
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          periodSeconds: 10
          successThreshold: 1
          timeoutSeconds: 1
        resources:
          requests:
            cpu: 100m
            memory: 90Mi
        securityContext:
          allowPrivilegeEscalation: true
          capabilities:
            add:
            - NET_BIND_SERVICE
            drop:
            - ALL
          runAsUser: 101
        volumeMounts:
        - mountPath: /usr/local/certificates/
          name: webhook-cert
          readOnly: true
      dnsPolicy: ClusterFirst
      nodeSelector:
        kubernetes.io/os: linux
      serviceAccountName: ingress-nginx
      terminationGracePeriodSeconds: 300
      volumes:
      - name: webhook-cert
        secret:
          secretName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.2.0
  name: ingress-nginx-admission-create
  namespace: ingress-nginx
spec:
  template:
    metadata:
      labels:
        app.kubernetes.io/component: admission-webhook
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
        app.kubernetes.io/version: 1.2.0
      name: ingress-nginx-admission-create
    spec:
      containers:
      - args:
        - create
        - --host=ingress-nginx-controller-admission,ingress-nginx-controller-admission.$(POD_NAMESPACE).svc
        - --namespace=$(POD_NAMESPACE)
        - --secret-name=ingress-nginx-admission
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        image: imagespull.opsvv.com/kubernetes/ingress/kube-webhook-certgen:v1.1.1 
        imagePullPolicy: IfNotPresent
        name: create
        securityContext:
          allowPrivilegeEscalation: false
      nodeSelector:
        kubernetes.io/os: linux
      restartPolicy: OnFailure
      securityContext:
        fsGroup: 2000
        runAsNonRoot: true
        runAsUser: 2000
      serviceAccountName: ingress-nginx-admission
---
apiVersion: batch/v1
kind: Job
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.2.0
  name: ingress-nginx-admission-patch
  namespace: ingress-nginx
spec:
  template:
    metadata:
      labels:
        app.kubernetes.io/component: admission-webhook
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/part-of: ingress-nginx
        app.kubernetes.io/version: 1.2.0
      name: ingress-nginx-admission-patch
    spec:
      containers:
      - args:
        - patch
        - --webhook-name=ingress-nginx-admission
        - --namespace=$(POD_NAMESPACE)
        - --patch-mutating=false
        - --secret-name=ingress-nginx-admission
        - --patch-failure-policy=Fail
        env:
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        image: imagespull.opsvv.com/kubernetes/ingress/kube-webhook-certgen:v1.1.1 
        imagePullPolicy: IfNotPresent
        name: patch
        securityContext:
          allowPrivilegeEscalation: false
      nodeSelector:
        kubernetes.io/os: linux
      restartPolicy: OnFailure
      securityContext:
        fsGroup: 2000
        runAsNonRoot: true
        runAsUser: 2000
      serviceAccountName: ingress-nginx-admission
---
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.2.0
  name: nginx
spec:
  controller: k8s.io/ingress-nginx
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  labels:
    app.kubernetes.io/component: admission-webhook
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.2.0
  name: ingress-nginx-admission
webhooks:
- admissionReviewVersions:
  - v1
  clientConfig:
    service:
      name: ingress-nginx-controller-admission
      namespace: ingress-nginx
      path: /networking/v1/ingresses
  failurePolicy: Fail
  matchPolicy: Equivalent
  name: validate.nginx.ingress.kubernetes.io
  rules:
  - apiGroups:
    - networking.k8s.io
    apiVersions:
    - v1
    operations:
    - CREATE
    - UPDATE
    resources:
    - ingresses
  sideEffects: None


5、验收，检查服务状态

自动创建了ingress-nginx的namespace

ingress nodeport 测试 ingress filtering_kubernetes_06

检查pod状态

ingress nodeport 测试 ingress filtering_kubernetes_07

观察服务已经正常启动，并且使用机器80端口

netstat -ntlp|egrep -a "80|443"

ingress nodeport 测试 ingress filtering_docker_08

浏览器访问

没有配置域名，使用IP访问任意节点的80和443端口

ingress nodeport 测试 ingress filtering_kubernetes_09

ingress nodeport 测试 ingress filtering_kubernetes_10

ingress nodeport 测试 ingress filtering_kubernetes_11

6、验收结果

有划水嫌疑

ingress controller服务正常，功能正常。

Ingress 配置http代理SVC

不演示乱七八糟生产环境不用的配置，直接上干货。

接下来yaml文件部署ingress跟逻辑图片对应操作

ingress nodeport 测试 ingress filtering_容器_12

1、启动四个测试Tomcat Pod

Namespace ingress-test，四个测试pod对应的svc如下

域名

Svc

deployment

页面内容

test01.ingress.opsvv.com

server01-service

test01-deployment

test01

test02.ingress.opsvv.com

server02-service

test02-deployment

test02

test03.ingress.opsvv.com

server03-service

test03-deployment

test03

test04.ingress.opsvv.com

server04-service

test04-deployment

test04

创建namespace和pod

02-04yaml文件，改一下pod名称和svc名称即可


kind: Deployment #apiVersion: extensions/v1beta1 apiVersion: apps/v1 metadata: labels: app: test01-deployment-label name: test01-deployment namespace: test-ingress spec: replicas: 1 selector: matchLabels: app: test01-selector template: metadata: labels: app: test01-selector spec: containers: - name: test01-container image: tomcat:7.0.94-alpine imagePullPolicy: Always ports: - containerPort: 8080 protocol: TCP name: http env: - name: "password" value: "123456" - name: "age" value: "18" resources: limits: cpu: 1 memory: "212Mi" requests: cpu: 200m memory: "212Mi" --- kind: Service apiVersion: v1 metadata: labels: app: test01-service-label name: test01-service namespace: test-ingress spec: #使用默认svc cluster模式 ports: - name: http port: 80 protocol: TCP targetPort: 8080 selector: app: test01-selector


kubectl create namespace test-ingress kubectl apply -f ./ kubectl get pods -ntest-ingress kubectl get svc -ntest-ingress


ingress nodeport 测试 ingress filtering_docker_13

ingress nodeport 测试 ingress filtering_docker_14

2、拷贝测试页面到Tomcat Pod

Tomcat容器内默认web路径

/usr/local/tomcat/webapps/


kubectl cp app01/ test01-deployment-cdf97dd89-6nhwr:/usr/local/tomcat/webapps/ -ntest-ingress kubectl cp app02/ test02-deployment-5fd5dd7ccf-ghtlc:/usr/local/tomcat/webapps/ -ntest-ingress kubectl cp app03/ test03-deployment-5cfd4cb695-jbmgc:/usr/local/tomcat/webapps/ -ntest-ingress kubectl cp app04 test04-deployment-855d9d46f5-bwcbc:/usr/local/tomcat/webapps/ -ntest-ingress


ingress nodeport 测试 ingress filtering_容器_15

3、验证前端index.jsp是否生效


cat app01/index.jsp cat app02/index.jsp cat app03/index.jsp cat app04/index.jsp kubectl get svc -ntest-ingress curl http://10.68.210.83:80/app01/index.jsp curl http://10.68.52.23:80/app02/index.jsp curl http://10.68.171.97:80/app03/index.jsp curl http://10.68.184.250:80/app04/index.jsp


ingress nodeport 测试 ingress filtering_kubernetes_16

3、创建Ingress，使用不通域名代理到svc

参数解释

ingress nodeport 测试 ingress filtering_容器_17

ingress nodeport 测试 ingress filtering_容器_18


apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: tomcat-web namespace: test-ingress annotations: kubernetes.io/ingress.class: "nginx" nginx.ingress.kubernetes.io/use-regex: "true" nginx.ingress.kubernetes.io/proxy-connect-timeout: "600" nginx.ingress.kubernetes.io/proxy-send-timeout: "600" nginx.ingress.kubernetes.io/proxy-read-timeout: "600" nginx.ingress.kubernetes.io/proxy-body-size: "10m" #nginx.ingress.kubernetes.io/rewrite-target: / nginx.ingress.kubernetes.io/app-root: /index.html spec: rules: - host: test01.ingress.opsvv.com http: paths: - pathType: Prefix path: "/" backend: service: name: server01-serverice port: number: 80 - host: test02.ingress.opsvv.com http: paths: - pathType: Prefix path: "/" backend: service: name: test02-service port: number: 80 - host: test03.ingress.opsvv.com http: paths: - pathType: Prefix path: "/" backend: service: name: test03-service port: number: 80 - host: test04.ingress.opsvv.com http: paths: - pathType: Prefix path: "/" backend: service: name: test04-service port: number: 80


4、创建ingress,验证代理配置

命令行验证

域名是否代理到了service


kubectl apply -f ingress-hosts.yml kubectl get ingress -ntest-ingress curl http://test01.ingress.opsvv.com/app01/ curl http://test02.ingress.opsvv.com/app02/ curl http://test03.ingress.opsvv.com/app03/ curl http://test04.ingress.opsvv.com/app04/


ingress nodeport 测试 ingress filtering_kubernetes_19

浏览器验证，因为使用的80端口，提示不安全是正常的

ingress nodeport 测试 ingress filtering_容器_20

ingress nodeport 测试 ingress filtering_ingress nodeport 测试_21

ingress nodeport 测试 ingress filtering_nginx_22

ingress nodeport 测试 ingress filtering_kubernetes_23

http 域名代理配置完成。

配置Http正则转发

1、uri规划

类似nginx ^~ /location ,

使用一个新的域名，方便跟之前配置过的代理做区分

域名

uri

代理后端service

uripath.ingress.opsvv.com

/app01

test01-service

uripath.ingress.opsvv.com

/app02

test02-service

uripath.ingress.opsvv.com

/app03

test03-service

uripath.ingress.opsvv.com

/app04

test04-service

2、修改ingress-uri.yaml文件

ingress nodeport 测试 ingress filtering_ingress nodeport 测试_24


apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: tomcat-uripath namespace: test-ingress annotations: kubernetes.io/ingress.class: "nginx" nginx.ingress.kubernetes.io/use-regex: "true" nginx.ingress.kubernetes.io/proxy-connect-timeout: "600" nginx.ingress.kubernetes.io/proxy-send-timeout: "600" nginx.ingress.kubernetes.io/proxy-read-timeout: "600" nginx.ingress.kubernetes.io/proxy-body-size: "10m" nginx.ingress.kubernetes.io/app-root: /index.html spec: rules: - host: uritest.ingress.opsvv.com http: paths: - pathType: Prefix path: "/app01" backend: service: name: test01-service port: number: 80 - pathType: Prefix path: "/app02" backend: service: name: test02-service port: number: 80 - pathType: Prefix path: "/app03" backend: service: name: test03-service port: num: 80 - pathType: Prefix path: "/app04" backend: service: name: test04-service port: number: 80


3、创建ingress并验证


kubectl apply -f ingress-uri.yaml kubectl get ingress -ntest-ingress


ingress nodeport 测试 ingress filtering_ingress nodeport 测试_25


curl http://uritest.ingress.opsvv.com/app01/ curl http://uritest.ingress.opsvv.com/app02/ curl http://uritest.ingress.opsvv.com/app03/ curl http://uritest.ingress.opsvv.com/app04/


ingress nodeport 测试 ingress filtering_容器_26

浏览器验证

ingress nodeport 测试 ingress filtering_docker_27

ingress nodeport 测试 ingress filtering_ingress nodeport 测试_28

ingress nodeport 测试 ingress filtering_nginx_29

ingress nodeport 测试 ingress filtering_docker_30

4、Ingress Http总结

ingress http侧配置和使用都已经完成

后续补充 TCP配置，这一块在生产中用到的挺多的。

Ingress 配置https代理Svc

跟http使用99%都一样，就是多了一个ssl证书而已

因为ingress controller已经在每个节点上都监听了机器层面的44端口，只需要配置yaml文件即可。

ingress nodeport 测试 ingress filtering_ingress nodeport 测试_31

1、多域名转发不同的svc

1、域名规划

因为是公网的域名和个人免费证书，申请比较麻烦。只用两个域名来演示。

腾讯云签发免费证书还是挺快的。

域名

转发到service

test01.ingress.opsvv.com

test01-service

test02.ingress.opsvv.com

test02-service

ingress nodeport 测试 ingress filtering_docker_32

下载证书服务器类型尽量选Nginx

ingress nodeport 测试 ingress filtering_ingress nodeport 测试_33

没有公网证书可以使用Openssl自签名证书，生成方法如下

Openssl生成自签名证书

1、导入证书创建secret

1、解压证书


unzip test01.ingress.opsvv.com_nginx.zip unzip test02.ingress.opsvv.com_nginx.zip


ingress nodeport 测试 ingress filtering_nginx_34

2、创建secret


#创建test01 secret cd test01.ingress.opsvv.com_nginx/ kubectl create secret tls tls-secret-tomcat-web-test01 --cert=test01.ingress.opsvv.com_bundle.crt --key=test01.ingress.opsvv.com.key -n test-ingress kubectl get secret -ntest-ingress cd .. #创建test02 secret cd test02.ingress.opsvv.com_nginx/ kubectl create secret tls tls-secret-tomcat-web-test02 --cert=test02.ingress.opsvv.com_bundle.crt --key=test02.ingress.opsvv.com.key -n test-ingress kubectl get secret -ntest-ingress


ingress nodeport 测试 ingress filtering_kubernetes_35

3、配置Ingress https代理

配置文件解释：：去！

ingress nodeport 测试 ingress filtering_ingress nodeport 测试_36

ingress nodeport 测试 ingress filtering_ingress nodeport 测试_37

创建之前需要注意，千万不要出现冲突的配置。

例如http和https用的域名一样，匹配规则一样。会导致冲突创建ingress配置文件冲突


kubectl delete ingress tomcat-web -ntest-ingress kubectl delete ingress tomcat-uripath -ntest-ingress kubectl apply -f ingress-https-hosts.yaml


ingress nodeport 测试 ingress filtering_ingress nodeport 测试_38

4、验证https是否生效

linux验证

ingress nodeport 测试 ingress filtering_kubernetes_39

浏览器验证

ingress nodeport 测试 ingress filtering_容器_40

ingress nodeport 测试 ingress filtering_容器_41

2、使用域名配置规则匹配转发

域名

uri

service

ssl.ingress.opsvv.com

/app01

test01-service

ssl.ingress.opsvv.com

/app02

test02-service

1、导入证书


cd ssl-cert/ ls rz unzip ssl.ingress.opsvv.com_nginx.zip cd ssl.ingress.opsvv.com_nginx/


ingress nodeport 测试 ingress filtering_docker_42

2、创建secret

secret名称 tls-secret-tomcat-web-ssl

使用不通的域名来实现效果


kubectl create secret tls tls-secret-tomcat-web-ssl --cert=ssl.ingress.opsvv.com_bundle.crt --key=ssl.ingress.opsvv.com.key -n test-ingress kubectl get secret -ntest-ingress


ingress nodeport 测试 ingress filtering_docker_43

3、配置https uri yaml

ingress nodeport 测试 ingress filtering_kubernetes_44


apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: tomcat-web-https-uri namespace: test-ingress annotations: kubernetes.io/ingress.class: "nginx" ##指定Ingress Controller的类型 nginx.ingress.kubernetes.io/ssl-redirect: 'false' spec: tls: - hosts: - ssl.ingress.opsvv.com secretName: tls-secret-tomcat-web-ssl rules: - host: ssl.ingress.opsvv.com http: paths: - pathType: Prefix path: "/test01" backend: service: name: test01-service port: number: 80 - pathType: Prefix path: "/test02" backend: service: name: test02-service port: number: 80 - pathType: Prefix path: "/test03" backend: service: name: test03-service port: number: 80 - pathType: Prefix path: "/test04" backend: service: name: test04-service port: number: 80


3、ingress并验证


kubectl apply -f ingress-https-uri.ya kubectl get ingress -ntest-ingress


ingress nodeport 测试 ingress filtering_ingress nodeport 测试_45

linux机器验证

ingress nodeport 测试 ingress filtering_kubernetes_46

浏览器验证

ingress nodeport 测试 ingress filtering_nginx_47

ingress nodeport 测试 ingress filtering_ingress nodeport 测试_48

3、ingress https总结

https代理后端service 配置完成

https 通过uri匹配后端service 完成

在线更新Ingress证书

1、推荐方案一

创建新的secret

修改ingress 使用到的secret证书，需要提前统计都有哪些ingress使用，避免出现遗漏

kubectl apply ingress.yaml 生效

2、方案二简单粗暴 (比较low)，了解即可

1、以tls-secret-tomcat-web-ssl 为例

直接修改secret 内容

ingress nodeport 测试 ingress filtering_ingress nodeport 测试_49

3、了解知识base64字符串加解密

secret中存储cert和key都是经过base64加密的


1、base64加密 echo "str" | base64 2、base64解密 echo c3RyCg== | base64 -d


4、直接修改secret内容

1、备份secret私钥内容

kubectl get secret tls-secret-tomcat-web-ssl -o yaml -ntest-ingress >tls-secret-tomcat-web-ssl.bak


apiVersion: v1 data: tls.crt: 俺把crt内容删了 tls.key: 俺把key内容删了


1、查看ssl证书文件 base64加密格式

cat ssl.ingress.opsvv.com_bundle.crt |base64

2、编辑secret直接替换

tls.crt

tls.key

3、生产不建议这么做，如果替换失败会导致很多使用证书地方出现访问报错。s级故障香不香？年终奖还想不想要