实验要求

对一个现有django项目实现持续话的集成部署

具体实现方式

1 安装BlueOcean

我们这里使用 BlueOcean 这种方式来完成此处 CI/CD 的工作，BlueOcean 是 Jenkins 团队从用户体验角度出发，专为 Jenkins Pipeline 重新设计的一套 UI 界面，仍然兼容以前的 fressstyle 类型的 job，BlueOcean 具有以下的一些特性。

BlueOcean 插件：登录 Jenkins Web UI -> 点击左侧的 Manage Jenkins -> Manage Plugins -> Available -> 搜索查找 BlueOcean -> 点击下载安装并重启

jenkins可以和禅道一起用吗_ci

2.新建一个jenkins的流水线项目

jenkins可以和禅道一起用吗_ci_02


添加一个触发器，给gitlab提交代码时触发自动构建

jenkins可以和禅道一起用吗_ci_03


在gitlab项目下面添加webhooks，这样每次git push就能自动触发

jenkins可以和禅道一起用吗_ide_04

3.现有django项目qfcsspiders下增加相关文件，结构如下

qfcsspiders/
├── Dockerfile     ###docker镜像
├── Jenkinsfile    ###jenkins的pipline直接调用项目
├── manifests
│   ├── k8s.yaml  ###k8s的Deployment的yaml文件
│   ├── svc.yaml  ###k8s的Service的yaml文件
├── sonar-project.properties   ####sonar扫描的文件
├── robot.txt                 #### robot的测试用例
Dockerfile
FROM python:3.6
RUN echo "nameserver 114.114.114.114">>/etc/resolv.conf
RUN chmod o+r /etc/resolv.conf
RUN cp /etc/apt/sources.list /etc/apt/sources.list.bak && sed -i "s@http://deb.debian.org@http://mirrors.aliyun.com@g" /etc/apt/sources.list && rm -rf /var/lib/apt/lists/*
RUN apt-get update
RUN mkdir -p /data/qfcsspiders/
COPY . /data/qfcsspiders/
WORKDIR /data/qfcsspiders
RUN pip install -r requirements.txt -i https://mirrors.aliyun.com/pypi/simple/
EXPOSE 8000
CMD ["python", "manage.py", "runserver", "0.0.0.0:8000"]
k8s.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: qfcsspiders
  namespace: python
  labels:
    app: qfcsspiders
spec:
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
    type: RollingUpdate
  selector:
    matchLabels:
      app: qfcsspiders
  template:
    metadata:
      labels:
        app: qfcsspiders
    spec:
      restartPolicy: Always
      containers:
      - image: <IMAGE_URL>
        name: qfcsspiders
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 8000
svc.yaml
apiVersion: v1
kind: Service
metadata:
  name: qfcsspiders
  namespace: python
spec:
  selector:
    app: qfcsspiders
  type:  NodePort
  ports:
  - name: http
    port: 8000
    targetPort:  8000
    nodePort: 31800
sonar-project.properties
sonar.projectKey=qfcsspiders
sonar.projectName=qfcsspiders
sonar.projectVersion=1.0
sonar.coverage.dtdVerification=false
# 下面是扫描的目录
sonar.sources=qfcsspiders,app01,spiders,static,templates
#sonar.language=python
robot.txt
*** Settings ***
Library           RequestsLibrary
Library           SeleniumLibrary

*** Variables ***
${demo_url}       http://192.168.200.115:31800/login/

*** Test Cases ***
api
    [Tags]  critical
    Create Session    api    ${demo_url}
    ${alarm_system_info}    RequestsLibrary.Get Request    api    /
    log    ${alarm_system_info.status_code}
    log    ${alarm_system_info.content}
    should be true    ${alarm_system_info.status_code} == 200
Jenkinsfile
timeout(time: 600, unit: 'SECONDS') {
def label = "slave-${UUID.randomUUID().toString()}"
def proName ="qfcsspiders"
try{
podTemplate(label: label,cloud: 'k8s115', containers: [
  containerTemplate(name: 'node', image: 'node:alpine', command: 'cat', ttyEnabled: true),
  containerTemplate(name: 'docker', image: 'docker', command: 'cat', ttyEnabled: true),
  containerTemplate(name: 'kubectl', image: 'cnych/kubectl', command: 'cat', ttyEnabled: true),
  containerTemplate(name: 'tools', image: '192.168.200.218/library/tool:v3', command: 'cat', ttyEnabled: true)
], volumes: [
  hostPathVolume(mountPath: '/home/jenkins/.kube', hostPath: '/root/.kube'),
  hostPathVolume(mountPath: '/var/run/docker.sock', hostPath: '/var/run/docker.sock')
]) {
  node(label) {
    def myRepo = checkout scm
    def gitCommit = myRepo.GIT_COMMIT
    def gitBranch = myRepo.GIT_BRANCH.split('/')[-1]
    def imageTag = sh(script: "git rev-parse --short HEAD", returnStdout: true).trim()
    def dockerRegistryUrl = "192.168.200.118"
    def imageEndpoint = "library/qfcsspiders"
    def image = "${dockerRegistryUrl}/${imageEndpoint}"

    stage('Clone') {
      echo "1.Prepare Stage"
	  checkout scm
	  imageTag = "${gitBranch}-${imageTag}"
	  imageurl = "${image}:${imageTag}"

    }
   stage('Test') {
     echo "2.Test Stage"
     echo gitBranch
   }

    stage('CI'){
            stage('Unit Test') {
                    echo "Unit Test Stage Skip..."
            }

            stage('Code Scan') {
                    container('tools') {
                        withSonarQubeEnv('mysonar') {
                            sh 'sonar-scanner -X'
                            sleep 3
                        }
                        script {
                            timeout(1) {
                                def qg = waitForQualityGate('mysonar')
                                if (qg.status != 'OK') {
                                    error "未通过Sonarqube的代码质量阈检查，请及时修改！failure: ${qg.status}"
                                }
                            }
                        }
                    }

            }

    }


    stage('构建 Docker 镜像') {
	  withCredentials([usernamePassword(credentialsId: 'dockerhub', passwordVariable: 'password', usernameVariable: 'username')]) {
          container('docker') {
            echo "3. 构建 Docker 镜像阶段"
            sh """
              docker login ${dockerRegistryUrl} -u ${username} -p ${password}
              docker build -t ${image}:${imageTag} .
              docker push ${image}:${imageTag}
              """
          }
      }
    }
    stage('运行 Kubectl') {
      container('kubectl') {
// 	    if (gitBranch == 'main') {
//             input "确认要部署线上环境吗？"
//         }

        echo "查看 K8S 集群 Pod 列表"
        sh "kubectl get pods"
        sh """
          sed -i "s#<IMAGE_URL>#${imageurl}#g" manifests/k8s.yaml
          kubectl apply -f manifests/k8s.yaml
          kubectl apply -f manifests/svc.yaml
        """
      }
    }

    stage('接口测试') {
     container('tools') {
            sleep 60
            sh 'robot -i critical  -d artifacts/ robot.txt|| echo ok'
            echo "R ${currentBuild.result}"
            step([
                $class : 'RobotPublisher',
                outputPath: 'artifacts/',
                outputFileName : "output.xml",
                disableArchiveOutput : false,
                passThreshold : 80,
                unstableThreshold: 20.0,
                onlyCritical : true,
                otherFiles : "*.png"
            ])
            echo "R ${currentBuild.result}"
            archiveArtifacts artifacts: 'artifacts/*', fingerprint: true
            }
     }


  }
}

}catch(Exception e) {
        currentBuild.result = "FAILURE"
    }finally {
        echo "发送信息"
        def label2 = "slave-${UUID.randomUUID().toString()}"
        def currResult = currentBuild.result ?: 'SUCCESS'
        podTemplate(label: label2, cloud: 'k8s115',containers: [
         containerTemplate(name: 'tools', image: '192.168.200.218/library/tool:v3', command: 'cat', ttyEnabled: true)
      ]){
            node(label2) {
            // 判断执行任务状态，根据不同状态发送企业微信
            stage('微信通知'){
               container('tools'){
                if (currResult == 'SUCCESS') {
                sh """
                curl 'https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=yourkey' \
                    -H 'Content-Type: application/json' \
                    -d '{
                        "msgtype": "markdown",
                        "markdown": {
                            "content": "<font color='info'> 构建成功   </font>  \n **项目名称**：${proName}  \n**构建number**：${BUILD_NUMBER} "
                        }
                    }'
                """
                }else {
                sh """
                curl 'https://qyapi.weixin.qq.com/cgi-bin/webhook/send?key=yourkey' \
                    -H 'Content-Type: application/json' \
                    -d '{
                        "msgtype": "markdown",
                        "markdown": {
                            "content": "<font color='warning'>😖❌ 构建失败 ❌😖</font>  \n**项目名称**：${proName}  \n**构建number**：${BUILD_NUMBER}"
                        }
                    }'
                """
            }
          }
        }
     }
    }
}
}

需要注意的是

  • harbor地址192.168.200.118后期需要替换成https的域名
  • 其中 credentialsId: 'dockerhub’需要新建harbor的账号密码用于docker login使用
  • 其中tool镜像需要单独需要定制化才能使用

4.制作tool镜像

目录结构如下

[root@node02 tool-docker]# ls
config  Dockerfile  kubectl  requirements.txt  sonar-scanner  sonar-scanner-cli-4.2.0.1873-linux.zip

其中config文件需要再k8s的master的 /root/.kube/config
requirements.txt文件为（自动化测试测试robotframework）

robotframework
robotframework-seleniumlibrary
robotframework-databaselibrary
robotframework-request

sonar-scanner文件夹为sonar的扫描工具，下周使用参考：
这边需要解压后放到docker容器中

下面是具体的Dockerfile

FROM alpine
LABEL maintainer="chenzhenhua"
USER root

RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.tuna.tsinghua.edu.cn/g' /etc/apk/repositories && \
    apk update && \
    apk add  --no-cache openrc docker git curl tar gcc g++ make \
    bash shadow openjdk11  openssl-dev libffi-dev python3  python3-dev  py-pip\
    libstdc++ harfbuzz nss freetype ttf-freefont chromium chromium-chromedriver subversion  && \
    mkdir -p /root/.kube && \
    usermod -a -G docker root

COPY config /root/.kube/

COPY requirements.txt /

RUN pip3 install -i http://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com -r requirements.txt


RUN rm -rf /var/cache/apk/* && \
    rm -rf ~/.cache/pip


RUN rm -rf /var/cache/apk/

#-----------------安装 kubectl--------------------#
COPY kubectl /usr/local/bin/
RUN chmod +x /usr/local/bin/kubectl
# ------------------------------------------------#

#---------------安装 sonar-scanner-----------------#
COPY sonar-scanner /usr/lib/sonar-scanner
RUN ln -s /usr/lib/sonar-scanner/bin/sonar-scanner /usr/local/bin/sonar-scanner && chmod +x /usr/local/bin/sonar-scanner
ENV SONAR_RUNNER_HOME=/usr/lib/sonar-scanner
# ------------------------------------------------#

5.简单的脚本试cicd已经完成

我们提交代码进行测试：

jenkins可以和禅道一起用吗_jenkins_05


步骤基本： 代码下载-单元测试（调过）-代码扫描-构建镜像、推送到仓库-部署到k8s-用力测试-企业微信通知

然而 这样的写法比较复杂，可用性很低，让开发人员自己去写肯定没人愿意，下一篇我们需要把功能实现简化和标准化