k8s 实战

  • 备注: 用到的源代码可以联系QQ:122725501 索取即可

传统部署与k8s部署的区别

  • 传统部署

kubernetes(十三) k8s 业务上线流程(手动版)

  • k8s 部署架构

kubernetes(十三) k8s 业务上线流程(手动版)

  • 项目迁移到k8s的流程

kubernetes(十三) k8s 业务上线流程(手动版)

制作镜像

  • 镜像分类
    • 基础镜像
    • 环境镜像
    • 项目镜像

kubernetes(十三) k8s 业务上线流程(手动版)

控制器管理POD

  • Deployment:无状态部署,例如Web,微服务,API
  • StatefulSet:有状态部署,例如数据库,ZK,ETCD
  • DaemonSet:守护进程部署,例如监控Agent、日志Agent
  • Job & CronJob:批处理,例如数据库备份,邮件通知

Pod数据持久化

  • 容器部署过程中一般有如下的三种数据
    • 启动时需要的初始数据,可以是配置文件
    • 启动过程中产生的初始化数据,该临时数据需要多个容器间共享
    • 启动过程中产生的业务数据

kubernetes(十三) k8s 业务上线流程(手动版)

暴露应用

使用Service ClusterIP类型暴露集群内部的应用访问

  • service定义了Pod逻辑集合和访问这个集合的策略
  • service引入为了解决Pod的动态变化,提供服务发现和负载均衡
  • 使用coreDNS解析Service名称

kubernetes(十三) k8s 业务上线流程(手动版)

对外发布应用

  • 使用ingress对外暴露应用
    • 通过Service关联Pod
    • 基于域名访问
    • 通过Ingress Controller实现pod的负载均衡
    • 支持TCP/UDP四层和Http七层

kubernetes(十三) k8s 业务上线流程(手动版)

部署Java/PHP项目

部署Java项目

  • 构建环境镜像
$ mkdir ~/base_env/ && cd ~/base_env
$ vim Dockerfile
FROM centos:7
LABEL maintainer="waanghui@qq.com"
ENV tomcat_version="8.5.57"
  • 代码提交至代码仓库: http://192.168.56.19:9999/root/java-demo.git (如何部署代码仓库可以联系QQ 122725501协助:)

  • 代码编译

    $ vi /etc/maven/settings.xml
      <mirrors>
           <mirror>
             <id>central</id>
             <mirrorOf>central</mirrorOf>
             <name>aliyun maven</name>
             <url>https://maven.aliyun.com/repository/public</url>
           </mirror>
       </mirrors>
    $ cd java-demo
    $ mvn clean package -D maven.test.skip=true
  • 构建镜像并推送到镜像仓库
$  vim Dockerfile 
FROM hub.cropy.cn/base/tomcat:v1
LABEL maintainer="wanghui@qq.com"
RUN rm -rf /usr/local/tomcat/webapps/*
COPY target/*.war /usr/local/tomcat/webapps/ROOT.war

$ docker build -t hub.cropy.cn/demo/java-demo:v1 .
$ docker push hub.cropy.cn/demo/java-demo:v1
  • k8s服务编排部署项目

    • 创建命名空间,资源隔离,权限控制
    $ vim namespace.yaml 
    apiVersion: v1
    kind: Namespace
    metadata:
    name: test
    $ kubectl apply -f namespace.yaml 
    • 使用deploy控制器创建Pod
    # 需要创建hub仓库的secret
    $ kubectl create secret docker-registry docker-regsitry-auth --docker-server="hub.cropy.cn" --docker-username="admin" --docker-password="Harbor12345" -n test
    $ vim deployment.yaml 
    apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: java-demo
    namespace: test
    spec:
    replicas: 2
    selector:
      matchLabels:
        project: www
        app: java-demo
    template:
      metadata:
        labels:
          project: www
          app: java-demo
      spec:
        imagePullSecrets:
        - name: "docker-regsitry-auth"
        containers:
        - image: hub.cropy.cn/demo/java-demo:v1
          name: java-demo
          imagePullPolicy: Always
          ports:
          - containerPort: 8080
            name: web
            protocol: TCP
          resources:
            requests:
              cpu: 0.5
              memory: 0.5Gi
            limits:
              cpu: 1
              memory: 2Gi
          livenessProbe:
            httpGet:
              path: /
              port: 8080
            initialDelaySeconds: 60
            timeoutSeconds: 20
          readinessProbe:
            httpGet:
              path: /
              port: 8080
            initialDelaySeconds: 60
            timeoutSeconds: 20
    $ kubectl apply -f deployment.yaml     
    $ kubectl get pod -n test -o wide
    • 使用service暴露应用
    $ vim service.yaml 
    apiVersion: v1
    kind: Service
    metadata:
    labels:
      app: java-demo
    name: java-demo
    namespace: test
    spec:
    ports:
    - port: 80
      protocol: TCP
      targetPort: 8080
      nodePort: 30018
    selector:
      app: java-demo
      project: www
    type: NodePort
    
    $ kubectl apply -f service.yaml
    $ kubectl get ep -n test

    node节点访问测试

    • 使用ingress对外暴露
    $ vim ingress.yaml 
    apiVersion: extensions/v1beta1
    kind: Ingress
    metadata:
    name: java-demo
    namespace: test
    spec:
    rules:
    - host: java.cropy.cn
      http:
        paths:
        - path: /
          backend:
            serviceName: java-demo
            servicePort: 80
    $ kubectl apply -f ingress.yaml 
    $ kubectl  get ing -n test

    外网用户加入k8s任意集群节点的ip:java.cropy.cn 到hosts完成解析,就可以使用域名访问了

  • 外部环境和中间件准备(当前准备mysql)
$ helm install java-demo-db --set persistence.storageClass="managed-nfs-storage" stable/mysql
$ MYSQL_ROOT_PASSWORD=$(kubectl get secret --namespace default java-demo-db-mysql -o jsonpath="{.data.mysql-root-password}" | base64 --decode; echo)
$ echo $MYSQL_ROOT_PASSWORD    #获取数据库密码
$ mysql -h java-demo-db-mysql -p     #-h 这里的host是mysql的svc
E2szJIvuhI
$ kubectl get pods
NAME                                      READY   STATUS    RESTARTS   AGE
java-demo-db-mysql-745b9688b8-7f5fn       1/1     Running   0          93s
$ kubectl cp db/tables_ly_tomcat.sql java-demo-db-mysql-745b9688b8-7f5fn:/  #拷贝sql文件到数据库容器,然后导入数据
$ kubectl exec -it java-demo-db-mysql-745b9688b8-7f5fn -- bash
root@java-demo-db-mysql-745b9688b8-7f5fn:/# mysql -uroot -pE2szJIvuhI  
mysql> source /tables_ly_tomcat.sql;
  • 修改源代码中的配置文件
$ vim src/main/resources/application.yml 
server:
  port: 8080
spring:
  datasource:
    url: jdbc:mysql://java-demo-db-mysql.default:3306/test?characterEncoding=utf-8
    username: root
    password: E2szJIvuhI
    driver-class-name: com.mysql.jdbc.Driver
  freemarker:
    allow-request-override: false
    cache: true
    check-template-location: true
    charset: UTF-8
    content-type: text/html; charset=utf-8
    expose-request-attributes: false
    expose-session-attributes: false
    expose-spring-macro-helpers: false
    suffix: .ftl
    template-loader-path:
      - classpath:/templates/
$ mvn clean package -D maven.test.skip=true     #重新打包
$ docker build -t hub.cropy.cn/demo/java-demo:v2 . #重新构建
$ docker push hub.cropy.cn/demo/java-demo:v2     #镜像上传
  • 重新发布和回滚
$ vim k8s-yaml/deployment.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: java-demo
  namespace: test
spec:
  replicas: 3
  selector:
    matchLabels:
      project: www
      app: java-demo
  template:
    metadata:
      labels:
        project: www
        app: java-demo
    spec:
      imagePullSecrets:
      - name: "docker-regsitry-auth"
      containers:
      - image: hub.cropy.cn/demo/java-demo:v2
        name: java-demo
        imagePullPolicy: Always
        ports:
        - containerPort: 8080
          name: web
          protocol: TCP
        resources:
          requests:
            cpu: 0.5
            memory: 0.5Gi
          limits:
            cpu: 1
            memory: 2Gi
        livenessProbe:
          httpGet:
            path: /
            port: 8080
          initialDelaySeconds: 60
          timeoutSeconds: 20
        readinessProbe:
          httpGet:
            path: /
            port: 8080
          initialDelaySeconds: 60
          timeoutSeconds: 20
$ kubectl apply -f k8s-yaml/deployment.yaml
$ kubectl rollout history deploy/java-demo -n test    #查看历史
$ kubectl rollout undo deploy/java-demo -n test       #回滚至上一版本

如果项目不能正常写入数据,可以到myqsl添加授权即可

授权root有所有远程权限:

grant all on java.* to 'root'@'%' identified by 'E2szJIvuhI';
  • 测试
加入客户端主机名解析,然后浏览器直接访问即可

192.168.56.14 java.cropy.cn

部署php项目

  • 下载代码
$ git clone http://192.168.56.19:9999/root/php-demo.git
  • 构建镜像
$ vim Dockerfile
FROM lizhenliang/nginx-php:latest
LABEL maintainer="php.cropy.cn"
ADD . /usr/local/nginx/html 
$ docker build -t hub.cropy.cn/demo/php-demo:v1 .
$ docker push hub.cropy.cn/demo/php-demo:v1
  • k8s部署编排配置

    • 创建命名空间,资源隔离,权限控制
    $ vim namespace.yaml 
    apiVersion: v1
    kind: Namespace
    metadata:
    name: test
    $ kubectl apply -f namespace.yaml 
    • 使用deploy控制器创建Pod
    $ vim deployment.yaml 
    apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: php-demo
    namespace: test
    spec:
    replicas: 3
    selector:
      matchLabels:
        project: www
        app: php-demo
    template:
      metadata:
        labels:
          project: www
          app: php-demo
      spec:
        imagePullSecrets:
        - name: docker-regsitry-auth
        containers:
        - name: nginx
          image: hub.cropy.cn/demo/php-demo:v1
          ports:
          - containerPort: 80
            name: web
            protocol: TCP
          resources:
            requests:
              cpu: 0.5
              memory: 256Mi
            limits:
              cpu: 1
              memory: 1Gi
          livenessProbe:
            httpGet:
              path: /status.php
              port: 80
            initialDelaySeconds: 6
            timeoutSeconds: 20
          readinessProbe:
            httpGet:
              path: /status.php
              port: 80
            initialDelaySeconds: 6
            timeoutSeconds: 20
    $ kubectl apply -f deployment.yaml     
    $ kubectl get pod -n test -o wide
    • 使用service暴露应用
    $ vim service.yaml 
    apiVersion: v1
    kind: Service
    metadata:
    name: php-demo
    namespace: test
    spec:
    selector:
      project: www
      app: php-demo
    ports:
    - name: web
      port: 80
      targetPort: 80
    $ kubectl apply -f service.yaml
    $ kubectl get ep -n test

    node节点访问测试

    • 使用ingress对外暴露
    $ vim ingress.yaml 
    apiVersion: extensions/v1beta1
    kind: Ingress
    metadata:
    name: php-demo
    namespace: test
    spec:
    rules:
      - host: php.cropy.cn
        http:
          paths:
          - path: /
            backend:
              serviceName: php-demo
              servicePort: 80
    $ kubectl apply -f ingress.yaml 
    $ kubectl  get ing -n test

    上述流程就是上线php项目的流程

    • 外部环境和中间件准备(当前准备mysql)
    $ helm install php-demo-db --set persistence.storageClass="managed-nfs-storage" stable/mysql    
    $ MYSQL_ROOT_PASSWORD=$(kubectl get secret --namespace default php-demo-db-mysql -o jsonpath="{.data.mysql-root-password}" | base64 --decode; echo)
    $ echo $MYSQL_ROOT_PASSWORD
    cHDi5ikYAj
    $ kubectl get pod
    NAME                                      READY   STATUS    RESTARTS   AGE
    nfs-client-provisioner-75f9fd7b57-67jtk   1/1     Running   1          4d7h
    php-demo-db-mysql-5c7fb76866-wn8rd        1/1     Running   0          115s
    $ kubectl exec -it php-demo-db-mysql-5c7fb76866-wn8rd -- bash
    root@php-demo-db-mysql-5c7fb76866-wn8rd:/# mysql -uroot -pcHDi5ikYAj
    mysql> create database wp;
    • 修改代码配置文件,重新打包
    $ vim wp-config.php
    define('DB_NAME', 'wp');
    
    /** MySQL数据库用户名 */
    define('DB_USER', 'root');
    
    /** MySQL数据库密码 */
    define('DB_PASSWORD', 'cHDi5ikYAj');
    
    /** MySQL主机 */
    define('DB_HOST', 'php-demo-db-mysql.default');
    
    $ docker build -t hub.cropy.cn/demo/php-demo:v2 .
    $ docker push hub.cropy.cn/demo/php-demo:v2 .
    • 更新&回滚
    $ vim deployment.yaml   #更新到v2版本
    apiVersion: apps/v1
    kind: Deployment
    metadata:
    name: php-demo
    namespace: test
    spec:
    replicas: 3
    selector:
      matchLabels:
        project: www
        app: php-demo
    template:
      metadata:
        labels:
          project: www
          app: php-demo
      spec:
        imagePullSecrets:
        - name: docker-regsitry-auth
        containers:
        - name: nginx
          image: hub.cropy.cn/demo/php-demo:v2
          ports:
          - containerPort: 80
            name: web
            protocol: TCP
          resources:
            requests:
              cpu: 0.5
              memory: 256Mi
            limits:
              cpu: 1
              memory: 1Gi
          livenessProbe:
            httpGet:
              path: /status.php
              port: 80
            initialDelaySeconds: 6
            timeoutSeconds: 20
          readinessProbe:
            httpGet:
              path: /status.php
              port: 80
            initialDelaySeconds: 6
            timeoutSeconds: 20
    $ kubectl apply -f deployment.yaml 
    • 测试
    加入客户端主机名解析,然后浏览器直接访问即可
    
    192.168.56.14 php.cropy.cn