KubeSphere 的流水线创建

1. 创建凭证

这里我们要创建3个凭证,分别是:

1. sonarqube的token
1. gitlab的用户名密码(或者gitee的用户名密码)
1. github的用户名密码(或者阿里云容器镜像服务的用户名密码)

kubesphere流水线参入参数不弹框 kubesphere 流水线_k8s

构建镜像时发现github和dockerhub拉取上传有问题,就改为了国内的gitee和阿里,于是最终采用了阿里云+gitee

kubesphere流水线参入参数不弹框 kubesphere 流水线_kubesphere_02

2.创建项目

2.1 创建kubesphere-sample-dev项目

使用project-admin用户登录kubesphere,创建项目

kubesphere流水线参入参数不弹框 kubesphere 流水线_kubesphere_03

kubesphere-sample-dev

kubesphere流水线参入参数不弹框 kubesphere 流水线_k8s_04

kubesphere流水线参入参数不弹框 kubesphere 流水线_kubesphere_05

2.2 邀请成员

kubesphere流水线参入参数不弹框 kubesphere 流水线_jenkins_06

kubesphere流水线参入参数不弹框 kubesphere 流水线_kubesphere_07

3. 创建流水线

3.1 进入devops项目

使用project-regular用户登录

kubesphere流水线参入参数不弹框 kubesphere 流水线_git_08

3.2 创建流水线

kubesphere流水线参入参数不弹框 kubesphere 流水线_maven_09

流水线名字

kubesphere-sample-dev-pipeline

kubesphere流水线参入参数不弹框 kubesphere 流水线_git_10

3.3 添加构建参数

构建参数


作用

REGISTRY

registry.cn-shanghai.aliyuncs.com

镜像仓库地址

DOCKERHUB_NAMESPACE

devops-maven-sample

镜像仓库namespace

APP_NAME

devops-sample

镜像名

kubesphere流水线参入参数不弹框 kubesphere 流水线_k8s_11

后来发现dockerhub镜像上传时比较麻烦就把仓库改到了阿里

kubesphere流水线参入参数不弹框 kubesphere 流水线_jenkins_12

4. 编辑流水线

4.1 流水线编辑

kubesphere流水线参入参数不弹框 kubesphere 流水线_maven_13

kubesphere流水线参入参数不弹框 kubesphere 流水线_jenkins_14

kubesphere流水线参入参数不弹框 kubesphere 流水线_k8s_15

4.2 Fork项目

项目来源:https://github.com/kubesphere/devops-maven-sample
将该项目同步到gitee中.(懒得重新截图了,步骤差不多)

kubesphere流水线参入参数不弹框 kubesphere 流水线_kubesphere_16

kubesphere流水线参入参数不弹框 kubesphere 流水线_kubesphere_17

修改Dockerfile-online的镜像

FROM openjdk:8

WORKDIR /home

COPY target/*.jar /home

ENTRYPOINT java -jar *.jar

4.3 代码克隆

git仓库地址

https://gitee.com/qqmiller/devops-maven-sample.git

kubesphere流水线参入参数不弹框 kubesphere 流水线_kubesphere_18

kubesphere流水线参入参数不弹框 kubesphere 流水线_kubesphere_19

4.4 单元测试

kubesphere流水线参入参数不弹框 kubesphere 流水线_git_20

选择指定容器

kubesphere流水线参入参数不弹框 kubesphere 流水线_git_21

添加嵌套步骤

kubesphere流水线参入参数不弹框 kubesphere 流水线_git_22

mvn clean package

kubesphere流水线参入参数不弹框 kubesphere 流水线_kubesphere_23

Jenkinsfile

kubesphere流水线参入参数不弹框 kubesphere 流水线_kubesphere_24

4.5 代码分析

4.5.1 添加maven容器

Code Analysis

kubesphere流水线参入参数不弹框 kubesphere 流水线_maven_25

指定容器

kubesphere流水线参入参数不弹框 kubesphere 流水线_kubesphere_26

4.5.2 添加SonarQube凭证

嵌套步骤

kubesphere流水线参入参数不弹框 kubesphere 流水线_maven_27

添加凭证

SONAR_TOKEN

kubesphere流水线参入参数不弹框 kubesphere 流水线_k8s_28

4.5.3 添加Sonarqube配置

添加嵌套步骤

这里别选错,是在添加凭证里面添加嵌套步骤

kubesphere流水线参入参数不弹框 kubesphere 流水线_kubesphere_29

添加sonarqube

kubesphere流水线参入参数不弹框 kubesphere 流水线_jenkins_30

继续添加sonarqube的嵌套步骤

kubesphere流水线参入参数不弹框 kubesphere 流水线_git_31

添加Shell命令

mvn sonar:sonar -Dsonar.login=$SONAR_TOKEN

kubesphere流水线参入参数不弹框 kubesphere 流水线_k8s_32

4.5.4 添加SonarQube超时

第三个

kubesphere流水线参入参数不弹框 kubesphere 流水线_k8s_33

添加超时

kubesphere流水线参入参数不弹框 kubesphere 流水线_git_34

4.5.5 代码质量检查

超时中添加嵌套步骤

kubesphere流水线参入参数不弹框 kubesphere 流水线_jenkins_35

添加代码质量检查

kubesphere流水线参入参数不弹框 kubesphere 流水线_kubesphere_36

这部分的步骤不叫多,截图如下

kubesphere流水线参入参数不弹框 kubesphere 流水线_k8s_37

Jenkinsfile部分

kubesphere流水线参入参数不弹框 kubesphere 流水线_kubesphere_38

4.6 构建镜像并推送至仓库

此步骤构建镜像,完成后将镜像推送至仓库

kubesphere流水线参入参数不弹框 kubesphere 流水线_jenkins_39

4.6.1 添加maven容器

kubesphere流水线参入参数不弹框 kubesphere 流水线_kubesphere_40

4.6.2 添加项目构建

添加嵌套

kubesphere流水线参入参数不弹框 kubesphere 流水线_maven_41

添加shell

mvn -Dmaven.test.skip=true clean package

kubesphere流水线参入参数不弹框 kubesphere 流水线_kubesphere_42

4.6.3 根据Dockerfile构建镜像

kubesphere流水线参入参数不弹框 kubesphere 流水线_kubesphere_43

构建镜像命令

REGISTRY, DOCKERHUB_NAMESPACE ,APP_NAME这3个变量就是在3.3节定义的那3个传参

docker build -f Dockerfile-online -t $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:SNAPSHOT-$BUILD_NUMBER .

kubesphere流水线参入参数不弹框 kubesphere 流水线_kubesphere_44

4.6.4 添加Git仓库凭证

由于上传需要验证,这一步添加Git凭证

添加嵌套步骤

kubesphere流水线参入参数不弹框 kubesphere 流水线_maven_45

上面是密码,下面是用户名

DOCKER_PASSWORD
DOCKER_USERNAME

kubesphere流水线参入参数不弹框 kubesphere 流水线_git_46

4.6.5 镜像仓库登陆

kubesphere流水线参入参数不弹框 kubesphere 流水线_maven_47

添加shell

echo "$DOCKER_PASSWORD" | docker login $REGISTRY -u "$DOCKER_USERNAME" --passwd-stdin

kubesphere流水线参入参数不弹框 kubesphere 流水线_kubesphere_48

4.6.6 镜像上传

kubesphere流水线参入参数不弹框 kubesphere 流水线_git_49

嵌套shell

docker push $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:SNAPSHOT-$BUILD_NUMBER

kubesphere流水线参入参数不弹框 kubesphere 流水线_maven_50

这里先跑一遍流水线,发现由于github和docker.io无法正常拉取上传关系,造成流水线失败.将代码转到gitee,镜像仓库改为阿里后构建完成流水线并将代码上传至阿里镜像仓库.

kubesphere流水线参入参数不弹框 kubesphere 流水线_jenkins_51

sonarqube检测结果

kubesphere流水线参入参数不弹框 kubesphere 流水线_k8s_52

阿里云镜像仓库也拿到了构建完成的镜像

kubesphere流水线参入参数不弹框 kubesphere 流水线_k8s_53

Jenkinsfile

pipeline {
  agent {
    node {
      label 'maven'
    }

  }
  stages {
    stage('CheckOut SCM') {
      agent none
      steps {
        git(url: 'https://gitee.com/qqmiller/devops-maven-sample.git', credentialsId: 'gitee', branch: 'master', changelog: true, poll: false)
      }
    }

    stage('Unit Test') {
      agent none
      steps {
        container('maven') {
          sh 'mvn clean package'
        }

      }
    }

    stage('Code Analysis') {
      agent none
      steps {
        container('maven') {
          withCredentials([string(credentialsId : 'sonar' ,variable : 'SONAR_TOKEN' ,)]) {
            withSonarQubeEnv('sonar') {
              sh 'mvn sonar:sonar -Dsonar.login=$SONAR_TOKEN'
            }

          }

          timeout(unit: 'MINUTES', activity: true, time: 30) {
            waitForQualityGate 'true'
          }

        }

      }
    }

    stage('Build and Push') {
      agent none
      steps {
        container('maven') {
          sh 'mvn -Dmaven.test.skip=true clean package'
          sh 'docker build -f Dockerfile-online -t $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:v$BUILD_NUMBER .'
          withCredentials([usernamePassword(credentialsId : 'aliyun' ,passwordVariable : 'DOCKER_PASSWORD' ,usernameVariable : 'DOCKER_USERNAME' ,)]) {
            sh 'echo "$DOCKER_PASSWORD" | docker login $REGISTRY -u "$DOCKER_USERNAME" --password-stdin'
            sh 'docker push $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:v$BUILD_NUMBER'
          }

        }

      }
    }

  }
}

4.7 生成制品

kubesphere流水线参入参数不弹框 kubesphere 流水线_jenkins_54

target/*.jar

kubesphere流水线参入参数不弹框 kubesphere 流水线_jenkins_55

4.8 应用部署

服务最终会被部署到kubesphere-sample-dev命名空间下

root@ks-master:~# kubectl get ns|grep kubesphere-sample-dev
kubesphere-sample-dev             Active   18h
root@ks-master:~# kubectl get pods -n kubesphere-sample-dev
No resources found in kubesphere-sample-dev namespace.

目前这个命名空间下是空的

4.8.1 审核

kubesphere流水线参入参数不弹框 kubesphere 流水线_kubesphere_56

添加步骤

kubesphere流水线参入参数不弹框 kubesphere 流水线_maven_57

4.8.2 添加kubeconfig凭证

继续添加步骤

kubesphere流水线参入参数不弹框 kubesphere 流水线_git_58

指定maven容器

kubesphere流水线参入参数不弹框 kubesphere 流水线_kubesphere_59

添加嵌套步骤

kubesphere流水线参入参数不弹框 kubesphere 流水线_git_60

添加凭证

KUBECONFIG_CONTENT

kubesphere流水线参入参数不弹框 kubesphere 流水线_git_61

4.8.3 部署应用

添加嵌套步骤

kubesphere流水线参入参数不弹框 kubesphere 流水线_git_62

mkdir ~/.kube
echo "$KUBECONFIG_CONTENT" > ~/.kube/config
envsubst < deploy/dev-ol/devops-sample.yaml | kubectl apply -f -
envsubst < deploy/dev-ol/devops-sample-svc.yaml | kubectl apply -f -

envsubst作用是变量填充

kubesphere流水线参入参数不弹框 kubesphere 流水线_maven_63

Jenkinsfile

pipeline {
  agent {
    node {
      label 'maven'
    }

  }
  stages {
    stage('CheckOut SCM') {
      agent none
      steps {
        git(url: 'https://gitee.com/qqmiller/devops-maven-sample.git', credentialsId: 'gitee', branch: 'master', changelog: true, poll: false)
      }
    }

    stage('Unit Test') {
      agent none
      steps {
        container('maven') {
          sh 'mvn clean package'
        }

      }
    }

    stage('Code Analysis') {
      agent none
      steps {
        container('maven') {
          withCredentials([string(credentialsId : 'sonar' ,variable : 'SONAR_TOKEN' ,)]) {
            withSonarQubeEnv('sonar') {
              sh 'mvn sonar:sonar -Dsonar.login=$SONAR_TOKEN'
            }

          }

          timeout(unit: 'MINUTES', activity: true, time: 30) {
            waitForQualityGate 'true'
          }

        }

      }
    }

    stage('Build and Push') {
      agent none
      steps {
        container('maven') {
          sh 'mvn -Dmaven.test.skip=true clean package'
          sh 'docker build -f Dockerfile-online -t $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:v$BUILD_NUMBER .'
          withCredentials([usernamePassword(credentialsId : 'aliyun' ,passwordVariable : 'DOCKER_PASSWORD' ,usernameVariable : 'DOCKER_USERNAME' ,)]) {
            sh 'echo "$DOCKER_PASSWORD" | docker login $REGISTRY -u "$DOCKER_USERNAME" --password-stdin'
            sh 'docker push $REGISTRY/$DOCKERHUB_NAMESPACE/$APP_NAME:v$BUILD_NUMBER'
          }

        }

      }
    }
    stage('Artifacts') {
      agent none
      steps {
        archiveArtifacts 'target/*.jar'
      }
    }
    stage('Deploy to Dev') {
      agent none
      steps {
        input(message: '''发布到开发环境
@project-admin''', submitter: 'project-admin')
        container('maven') {
          withCredentials([kubeconfigContent(credentialsId : 'kubeconfig' ,variable : 'KUBECONFIG_CONTENT' ,)]) {
            sh '''mkdir ~/.kube
echo "$KUBECONFIG_CONTENT" > ~/.kube/config
envsubst < deploy/dev-ol/devops-sample.yaml | kubectl apply -f -
envsubst < deploy/dev-ol/devops-sample-svc.yaml | kubectl apply -f -'''
          }
        }
      }
    }
  }
}

5. 运行流水线

运行测试下

运行到Deploy to Dev步骤时需要审批继续

kubesphere流水线参入参数不弹框 kubesphere 流水线_git_64

在project-admin用户下点击proceed

kubesphere流水线参入参数不弹框 kubesphere 流水线_kubesphere_65

任务就被继续执行

kubesphere流水线参入参数不弹框 kubesphere 流水线_kubesphere_66

制品

kubesphere流水线参入参数不弹框 kubesphere 流水线_maven_67

查看服务端口

root@ks-master:~# kubectl get pods,svc -n kubesphere-sample-dev
NAME                                 READY   STATUS    RESTARTS   AGE
pod/ks-sample-dev-597764c9d8-lqctx   1/1     Running   0          94s

NAME                    TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
service/ks-sample-dev   NodePort   10.233.46.39   <none>        8080:30861/TCP   66m

kubesphere流水线参入参数不弹框 kubesphere 流水线_k8s_68