下面这个阶段实在理解不了，可以将这个阶段去掉，如果公司不想这么用的话，大规模场景下使用也可以发现这种好处。
CI/CD流水线设计
总体目标：
我是一个用户，点开Jenkins之后输入版本分支，然后点流水线构建，第一个阶段下载代码，第二个阶段构建，生成了我们的包，第三个阶段进行代码扫描，第四个阶段是上传制品，此时这个制品到制品库里面去了，那么就到CD阶段了，也就是包在制品库了，可以进行去部署了。
传到制品库了，要将这个包下载下来，可以复制这个url的地址，现在就需要创建发布流水线，因为CI和CD都是不同的流水线，
这里包的版本号有几种格式
- 版本号1.1.1+commitid:拿到分支最后一次提交的ID，然后加上版本号，然后拼凑出最后的版本(这样是看不到历史版本的信息的)
- CI上传制品，将制品的下载地址信息存起来，存储到git上面，在上传制品这种生成这个文件，这个文件里面存储了包的一些配置信息，CD流水线在拿的时候就拿这个信息（所以有两种方式，一种是存储信息，一种是不存储信息，不存储就拼接url直接下载下来就可以了，直接输入版本号就行）
提交了ReleaseFile之后想自动触发，存放到git上面去了，这次做了一个变更，git识别到了，这是一个push动作，那么会自动触发CD流水线去部署。如果做自动化可以这样去搞，不做自动化可以将这个步骤去掉。
我们将CI和CD分成两条流水线作业。
- CI作业： 用户输入版本分支后下载代码，进行构建扫描最终将制品上传到制品仓库, 生成版本文件。（在上传制品之后，还得加一个阶段，生成一个文件，这个ReleaseFile文件里面存储的就是包的一些配置信息，CD流水线拿到这个文件就可以了）
- CD作业: 用户输入发布版本和选择要发布的主机IP后，下载制品，将制品和服务启动脚本cp到目标机器的发布目录， 远程执行启动脚本启动服务并进行健康检查。
CI 生成版本文件
在流水线最后一个步骤加上生成版本文件
- 获取gitlab上面的模板文件到本地（Gitlab下载文件的接口）
- 生成文件里面内容，将配置信息写入本地的一个文件当中（新版本文件，yaml文件的更改)
- 将这个文件传到gitlab（修改的是新版本文件）（gitlab上传文件的接口）
k8s里面的yaml文件都需要我们去更新，一般都是通过sed去修改yaml文件里面的内容，我们可以修改版本文件，这样就可以看到变更记录了。变更了啥信息都可以在git上面看得到。
最终效果如下：这个环境库里面存放着所有的发布的信息
发布信息的模板按照下面的去写
CD 获取版本文件（这样就可以看到变更的历史记录了）
1 从Gitlab下载版本文件
2 发布的时候只读取本地的版本文件
实践
先创建一个空项目，用来存储版本信息
新建一个模板文件叫release，根据实际业务想存储什么信息可以自己去定义。
buname: 业务名称
appname: 应用名称
version: 制品版本
artifact: 制品下载链接
现在要修改这个文件的内容，就是要先下载下来，然后上传。
现在创建token
这个token拿到了
现在找到gitlab的api
我这里的实现是用HttpRequest插件来实现的，没有使用curl
Repository files API | GitLab
https://docs.gitlab.com/ee/api/repository_files.html
现在要去下载这个文件，需要project ID和分支名称和文件名称。
pipeline {
agent any
stages {
stage('Hello') {
steps {
script {
//获取版本库文件内容 devops02-env/release.yaml
response = GetRepoFile(17,"release.yaml", "main")
println(response)
}
}
}
}
}
// 封装HTTP
def HttpReq(reqType, reqUrl,reqBody ){
def gitServer = "http://139.198.166.235:81/api/v4"
withCredentials([string(credentialsId: 'ecbcd399-da69-4802-8760-87a1c1ff58a1', variable: 'GITLABTOKEN')]) {
response = httpRequest acceptType: 'APPLICATION_JSON_UTF8',
consoleLogResponseBody: true,
contentType: 'APPLICATION_JSON_UTF8',
customHeaders: [[maskValue: false, name: 'PRIVATE-TOKEN', value: "${GITLABTOKEN}"]],
httpMode: "${reqType}",
url: "${gitServer}/${reqUrl}",
wrapAsMultipart: false,
requestBody: "${reqBody}"
}
return response
}
//获取文件内容
def GetRepoFile(projectId,filePath, branchName ){
//GET /projects/:id/repository/files/:file_path/raw
apiUrl = "/projects/${projectId}/repository/files/${filePath}/raw?ref=${branchName}"
response = HttpReq('GET', apiUrl, "")
return response.content
}结果如下：
Started by user admin
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on build-01 in /data/cicd/jenkinsagent/workspace/nexus/release-file
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Hello)
[Pipeline] script
[Pipeline] {
[Pipeline] withCredentials
Masking supported pattern matches of $GITLABTOKEN
[Pipeline] {
[Pipeline] httpRequest
Warning: A secret was passed to "httpRequest" using Groovy String interpolation, which is insecure.
Affected argument(s) used the following variable(s): [GITLABTOKEN]
See https://jenkins.io/redirect/groovy-string-interpolation for details.
HttpMethod: GET
URL: http://139.198.166.235:81/api/v4//projects/17/repository/files/release.yaml/raw?ref=main
Content-Type: application/json; charset=UTF-8
Accept: application/json
PRIVATE-TOKEN: ****
Sending request to url: http://139.198.166.235:81/api/v4//projects/17/repository/files/release.yaml/raw?ref=main
Response Code: HTTP/1.1 200 OK
Response:
buname: _NULL_
appname: _NULL_
version: _NULL_
artifact: _NULL_
Success: Status code 200 is in the accepted range: 100:399
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] echo
buname: _NULL_
appname: _NULL_
version: _NULL_
artifact: _NULL_
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
因为获取文件内容返回的是yaml格式的内容，以前返回的是json，现在用的yaml
//这里的分支版本可以通过branchName再做一个split就可以拿到，比如branchName:relaese-1.1.1
//可以在上传制品的时候返回artifactUrl信息
env.buName = "acmp"
env.appName = "myapp"
env.releaseVersion = "1.1.1"
env.artifactUrl = "http://139.198.166.235:8082/repository/devops-repo/acmp/acmp-myapp-service/1.1.1/acmp-myapp-service-1.1.1.jar"
pipeline {
agent any
stages {
stage('Hello') {
steps {
script {
//获取版本库文件内容 devops02-env/release.yaml
response = GetRepoFile(17,"release.yaml", "main")
//println(response)
yamlData = readYaml text: """${response}"""
yamlData.version = "${env.releaseVersion}"
yamlData.artifact = "${env.artifactUrl}"
yamlData.buname = "${env.buName}"
yamlData.appname = "${env.appName}"
println(yamlData.toString())
}
}
}
}
}
def HttpReq(reqType, reqUrl,reqBody ){
def gitServer = "http://139.198.166.235:81/api/v4"
withCredentials([string(credentialsId: 'ecbcd399-da69-4802-8760-87a1c1ff58a1', variable: 'GITLABTOKEN')]) {
response = httpRequest acceptType: 'APPLICATION_JSON_UTF8',
consoleLogResponseBody: true,
contentType: 'APPLICATION_JSON_UTF8',
customHeaders: [[maskValue: false, name: 'PRIVATE-TOKEN', value: "${GITLABTOKEN}"]],
httpMode: "${reqType}",
url: "${gitServer}/${reqUrl}",
wrapAsMultipart: false,
requestBody: "${reqBody}"
}
return response
}
//获取文件内容
def GetRepoFile(projectId,filePath, branchName ){
//GET /projects/:id/repository/files/:file_path/raw
apiUrl = "/projects/${projectId}/repository/files/${filePath}/raw?ref=${branchName}"
response = HttpReq('GET', apiUrl, "")
return response.content
}
Started by user admin
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on build-01 in /data/cicd/jenkinsagent/workspace/nexus/release-file
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Hello)
[Pipeline] script
[Pipeline] {
[Pipeline] withCredentials
Masking supported pattern matches of $GITLABTOKEN
[Pipeline] {
[Pipeline] httpRequest
Warning: A secret was passed to "httpRequest" using Groovy String interpolation, which is insecure.
Affected argument(s) used the following variable(s): [GITLABTOKEN]
See https://jenkins.io/redirect/groovy-string-interpolation for details.
HttpMethod: GET
URL: http://139.198.166.235:81/api/v4//projects/17/repository/files/release.yaml/raw?ref=main
Content-Type: application/json; charset=UTF-8
Accept: application/json
PRIVATE-TOKEN: ****
Sending request to url: http://139.198.166.235:81/api/v4//projects/17/repository/files/release.yaml/raw?ref=main
Response Code: HTTP/1.1 200 OK
Response:
buname: _NULL_
appname: _NULL_
version: _NULL_
artifact: _NULL_
Success: Status code 200 is in the accepted range: 100:399
[Pipeline] }
[Pipeline] // withCredentials
[Pipeline] readYaml
[Pipeline] echo
[buname:acmp, appname:myapp, version:1.1.1, artifact:http://139.198.166.235:8082/repository/devops-repo/acmp/acmp-myapp-service/1.1.1/acmp-myapp-service-1.1.1.jar]
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
现在就可以去替换文件里面内容，然后上传
env.buName = "acmp"
env.appName = "myapp"
env.releaseVersion = "1.1.1"
env.artifactUrl = "http://139.198.166.235:8082/repository/devops-repo/acmp/acmp-myapp-service/1.1.1/acmp-myapp-service-1.1.1.jar"
env.branchName = "release-1.1.1"
pipeline {
agent any
stages {
stage('Hello') {
steps {
script {
//下载版本库文件 devops02-env/release.yaml
response = GetRepoFile(17,"release.yaml", "main")
//println(response)
yamlData = readYaml text: """${response}"""
yamlData.version = "${env.releaseVersion}"
yamlData.artifact = "${env.artifactUrl}"
yamlData.buname = "${env.buName}"
yamlData.appname = "${env.appName}"
println(yamlData.toString())
sh "ls && rm -fr test.yaml"
writeYaml charset: 'UTF-8', data: yamlData, file: 'test.yaml'
newYaml = sh returnStdout: true, script: 'cat test.yaml'
println(newYaml)
//更新gitlab文件内容,转化为base64，在调用api上传的时候都是base64编码
base64Content = newYaml.bytes.encodeBase64().toString()
// 会有并行问题，同时更新报错
try {
UpdateRepoFile(17,"${env.appName}%2f${env.branchName}.yaml",base64Content, "main")
} catch(e){
CreateRepoFile(17,"${env.appName}%2f${env.branchName}.yaml",base64Content, "main")
}
}
}
}
}
}
def HttpReq(reqType, reqUrl,reqBody ){
def gitServer = "http://139.198.166.235:81/api/v4"
withCredentials([string(credentialsId: 'ecbcd399-da69-4802-8760-87a1c1ff58a1', variable: 'GITLABTOKEN')]) {
response = httpRequest acceptType: 'APPLICATION_JSON_UTF8',
consoleLogResponseBody: true,
contentType: 'APPLICATION_JSON_UTF8',
customHeaders: [[maskValue: false, name: 'PRIVATE-TOKEN', value: "${GITLABTOKEN}"]],
httpMode: "${reqType}",
url: "${gitServer}/${reqUrl}",
wrapAsMultipart: false,
requestBody: "${reqBody}"
}
return response
}
//获取文件内容
def GetRepoFile(projectId,filePath, branchName ){
//GET /projects/:id/repository/files/:file_path/raw
apiUrl = "/projects/${projectId}/repository/files/${filePath}/raw?ref=${branchName}"
response = HttpReq('GET', apiUrl, "")
return response.content
}
//更新文件内容
def UpdateRepoFile(projectId,filePath,fileContent, branchName){
apiUrl = "projects/${projectId}/repository/files/${filePath}"
reqBody = """{"branch": "${branchName}","encoding":"base64", "content": "${fileContent}", "commit_message": "update a new file"}"""
response = HttpReq('PUT',apiUrl,reqBody)
println(response)
}
//创建文件
def CreateRepoFile(projectId,filePath,fileContent, branchName){
apiUrl = "projects/${projectId}/repository/files/${filePath}"
reqBody = """{"branch": "${branchName}","encoding":"base64", "content": "${fileContent}", "commit_message": "update a new file"}"""
response = HttpReq('POST',apiUrl,reqBody)
println(response)
}
${env.appName}%2f${env.branchName}.yaml
上面意思是放在17号仓库， 仓库下面文件夹名字为${env.appName}，在该文件夹下面有生成一个文件{env.branchName}.yaml。这里使用的是%2f，本来是目录/文件这种格式，但是编码这里转换为了%2f，所以转化为编码要不然会失败。
(17,"${env.appName}%2f${env.branchName}.yaml",base64Content, "main")
仓库id+文件夹+文件+文件里面内容+分支名称
最后结果如下,可以看到符合预期，在版本信息管理库devops-env下面生成了对应项目的目录，并且目录下面包含制品的信息：
到时候CD的时候就可以拿下该文件进行发布了，拿下该文件，通过脚本对这个文件处理一下，拿到制品下载地址就可以下载下来了。
上面就是调用gitlab api去实现这个过程。
Gitops其实就是为环境单独创建了一个git仓库，无非就是在原有的CI的基础上面，加了一个步骤生成了一个文件，把项目的以及代码的信息全部都放在这个文件里面了（后期拿到这个文件能够获取里面的参数去部署），然后在这里面做了一个变更，此时自动触发到我们的环境里面，好处是每次的变更都可以看到。
在k8s里面，全部都是yaml文件，我们一般改的就是镜像，如果是helm的话，修改的是values.yaml文件，总之就是更新里面的内容，更新了之后去发布。
如果接受不了上面的步骤，那么就不要版本文件了，直接拿制品自己拼接就行了。
总结：所谓的gitops就是把部署描述文件，存放到git系统里面，ci的时候自动更新，cd可以通过仓库自动触发。
完整的代码如下
package org.devops
// 封装HTTP
def HttpReq(reqType, reqUrl,reqBody ){
def gitServer = "http://139.198.166.235:81/api/v4"
withCredentials([string(credentialsId: 'ecbcd399-da69-4802-8760-87a1c1ff58a1', variable: 'GITLABTOKEN')]) {
response = httpRequest acceptType: 'APPLICATION_JSON_UTF8',
consoleLogResponseBody: true,
contentType: 'APPLICATION_JSON_UTF8',
customHeaders: [[maskValue: false, name: 'PRIVATE-TOKEN', value: "${GITLABTOKEN}"]],
httpMode: "${reqType}",
url: "${gitServer}/${reqUrl}",
wrapAsMultipart: false,
requestBody: "${reqBody}"
}
return response
}
//获取文件内容
def GetRepoFile(projectId,filePath, branchName ){
//GET /projects/:id/repository/files/:file_path/raw
apiUrl = "/projects/${projectId}/repository/files/${filePath}/raw?ref=${branchName}"
response = HttpReq('GET', apiUrl, "")
return response.content
}
//更新文件内容
def UpdateRepoFile(projectId,filePath,fileContent, branchName){
apiUrl = "projects/${projectId}/repository/files/${filePath}"
reqBody = """{"branch": "${branchName}","encoding":"base64", "content": "${fileContent}", "commit_message": "update a new file"}"""
response = HttpReq('PUT',apiUrl,reqBody)
println(response)
}
//创建文件
def CreateRepoFile(projectId,filePath,fileContent, branchName){
apiUrl = "projects/${projectId}/repository/files/${filePath}"
reqBody = """{"branch": "${branchName}","encoding":"base64", "content": "${fileContent}", "commit_message": "update a new file"}"""
response = HttpReq('POST',apiUrl,reqBody)
println(response)
}
script {
//下载版本库文件 devops02-env/release.yaml
response = GetRepoFile(17,"release.yaml", "main")
//println(response)
yamlData = readYaml text: """${response}"""
yamlData.version = "${env.releaseVersion}"
yamlData.artifact = "${env.artifactUrl}"
yamlData.buname = "${env.buName}"
yamlData.appname = "${env.appName}"
println(yamlData.toString())
sh "ls && rm -fr test.yaml"
writeYaml charset: 'UTF-8', data: yamlData, file: 'test.yaml'
newYaml = sh returnStdout: true, script: 'cat test.yaml'
println(newYaml)
//更新gitlab文件内容,转化为base64，在调用api上传的时候都是base64编码
base64Content = newYaml.bytes.encodeBase64().toString()
// 会有并行问题，同时更新报错
try {
UpdateRepoFile(17,"${env.appName}%2f${env.branchName}.yaml",base64Content, "main")
} catch(e){
CreateRepoFile(17,"${env.appName}%2f${env.branchName}.yaml",base64Content, "main")
}
}
