文章目录
- 一. k8s集群修改config
- 1.1 备份当前k8s集群配置文件
- 1.2 删除当前k8s集群的apiserver的cert 和 key
- 1.3 生成新的apiserver的cert和key
- 1.4 刷新admin.conf
- 1.5 重启apiserver
- 1.6 刷新.kube/config
- 二. 安装kubectl
- 2.1 下载kubectl
- 2.2 配置kubectl
- 三. 使用kubernetes-client操作k8s集群
- 3.1 依赖
- 3.2 注意(可忽略)
- 3.3 创建 StatefulSet
- 3.4 运行shell命令
- 3.5 删除 StatefulSet
- 3.6 线上运行注意
一. k8s集群修改config
因为默认的是内网IP,复制出来后,后改为外面IP,报错
Unable to connect to the server: x509: certificate is valid for 10.0.1.27, 192.168.0.x, not x.x.x.x
原因从报错信息可以看出来:是证书对访问的IP做了校验,解决的方法就是要重新制作证书(不用动集群)。
1.1 备份当前k8s集群配置文件
cp -r /etc/kubernetes /etc/kubernetes.bak
1.2 删除当前k8s集群的apiserver的cert 和 key
rm -rf /etc/kubernetes/pki/apiserver.*
1.3 生成新的apiserver的cert和key
kubeadm init phase certs apiserver --apiserver-advertise-address ${Internal_IP} --apiserver-cert-extra-sans ${External_IP}
例如:
kubeadm init phase certs apiserver --apiserver-advertise-address 10.0.1.27 --apiserver-cert-extra-sans 122.51.103.44
1.4 刷新admin.conf
kubeadm alpha certs renew admin.conf
1.5 重启apiserver
kubectl -n kube-system delete pod -l component=kube-apiserver
1.6 刷新.kube/config
\cp /etc/kubernetes/admin.conf ~/.kube/config
注意:这里生成的
admin.conf
文件,server
填写的依然是master
的内网IP
,如果需要放在集群外的客户端上使用,将这个IP
改为master
的外网IP
即可。
二. 安装kubectl
2.1 下载kubectl
下载kubectl.exe
的可执行文件,我这里下载的是对应我们线上kubernetes
版本的kubectl
。如果想要下载其他版本,可以复制上面链接将版本号改对对应的即可。
将kubectl
添加到PATH
环境变量
在D
盘下创建kube
文件夹,将kubectl.exe
存放在kube
文件夹下。
将D:\kube\kubectl
添加到PATH
环境变量中
执行命令查看已安装的 kubectl
版本号
C:\Users\16872>kubectl version
Client Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.5", GitCommit:"5c99e2ac2ff9a3c549d9ca665e7bc05a3e18f07e", GitTreeState:"clean", BuildDate:"2021-12-16T08:38:33Z", GoVersion:"go1.16.12", Compiler:"gc", Platform:"windows/amd64"}
Unable to connect to the server: dial tcp 127.0.0.1:6443: connectex: No connection could be made because the target machine actively refused it.
2.2 配置kubectl
将kubectl config
文件存放在~/.kube/
下。 ~
代表当前的用户目录
创建~/.kube
文件夹,将master
上的config
文件拉取到本地, 存放在~/.kube/
文件夹下
该
config
文件支持公网IP连接(一
配置)
验证配置结果
PS C:\Users\16872> kubectl get nodes
NAME STATUS ROLES AGE VERSION
master Ready control-plane,master 4h7m v1.21.2
node1 Ready <none> 4h3m v1.21.2
node2 Ready <none> 4h3m v1.21.2
PS C:\Users\16872> kubectl get pods -n kube-system
NAME READY STATUS RESTARTS AGE
calico-kube-controllers-69cccc84d-jcc4f 1/1 Running 3 4h1m
calico-node-6qvhg 1/1 Running 1 4h1m
calico-node-jpqbc 1/1 Running 1 4h1m
calico-node-xg558 1/1 Running 2 4h1m
coredns-59d64cd4d4-cb6hg 1/1 Running 2 4h7m
coredns-59d64cd4d4-w9nt4 1/1 Running 2 4h7m
etcd-master 1/1 Running 1 4h7m
ingress-nginx-controller-qkw2c 1/1 Running 0 39m
ingress-nginx-defaultbackend-c49866855-dlst8 1/1 Running 0 39m
kube-apiserver-master 1/1 Running 1 4h7m
kube-controller-manager-master 1/1 Running 1 4h7m
kube-proxy-4jtzf 1/1 Running 2 4h3m
kube-proxy-6gbxz 1/1 Running 1 4h7m
kube-proxy-ljjct 1/1 Running 1 4h3m
kube-scheduler-master 1/1 Running 1 4h7m
三. 使用kubernetes-client操作k8s集群
3.1 依赖
<!--kubernetes-client-->
<dependency>
<groupId>io.kubernetes</groupId>
<artifactId>client-java</artifactId>
<version>16.0.0</version>
</dependency>
版本选择参考 https://github.com/kubernetes-client/java/wiki/2.-Versioning-and-Compatibility
3.2 注意(可忽略)
编辑配置添加-Djdk.tls.client.protocols=TLSv1.2
,
否则报错
io.kubernetes.client.openapi.ApiException: javax.net.ssl.SSLHandshakeException: extension (5) should not be presented in certificate_request
3.3 创建 StatefulSet
@Override
@SneakyThrows
public Boolean createCpsGo() {
ApiClient client = Config.defaultClient();
Configuration.setDefaultApiClient(client);
// new一个statefulSet
V1StatefulSet body = new V1StatefulSet()
.metadata(new V1ObjectMeta().name("pod-go"))
.spec(new V1StatefulSetSpec()
.serviceName("pod-go")
.replicas(3)
.selector(new V1LabelSelector().matchLabels(Map.of("app", "pod-go")))
.template(new V1PodTemplateSpec()
.metadata(new V1ObjectMeta().labels(Map.of("app", "pod-go")))
.spec(new V1PodSpec().containers(List.of(
new V1Container()
.name("pod-go")
.image("registry.cn-shanghai.aliyuncs.com/wanfei/centos7-golang-sh:v1")
.imagePullPolicy("IfNotPresent")
.stdin(true)
.tty(true)
)))
)
);
// 运行statefulSet
AppsV1Api appsV1Api = new AppsV1Api();
appsV1Api.createNamespacedStatefulSet("default", body, null, null, null, null);
return true;
}
创建成功
[root@master jenkins]# kubectl get StatefulSet | grep pod-go
pod-go 3/3 17s
3.4 运行shell命令
例如:生成文件,压缩文件
@Override
@SneakyThrows
public Boolean createFile() {
// 生成 pod-go-0.exe 文件命令
String shExeCmd = "sh build.sh /app/test/ pod-go-0";
// 压缩 pod-go-0.exe 为 pod-go-0.zip
String shZipCmd = "zip /app/test/pod-go-0.zip /app/test/pod-go-0.exe";
ApiClient client = Config.defaultClient();
Configuration.setDefaultApiClient(client);
// 执行kubectl exec在指定pod内运行命令
String shExeStr = execForStr(shExeCmd);
log.info("编译exe文件返回: {}", shExeStr);
String shZipStr = execForStr(shZipCmd);
log.info("压缩exe文件返回: {}", shZipStr);
return true;
}
/**
* 执行kubectl exec
* @param command 命令
*/
@SneakyThrows
private static String execForStr(String command) {
Exec exec = new Exec();
Process proc = exec.exec("default", "pod-go-0", ArrayUtil.toArray(List.of("/bin/sh", "-c", command), String.class), "pod-go", true, true);
return RuntimeUtil.getResult(proc);
}
编译压缩成功
[root@pod-go-0 test]# ls
main.go pdf1.syso pod-go-0.exe pod-go-0.zip
3.5 删除 StatefulSet
@Override
@SneakyThrows
public Boolean deleteCpsGo() {
ApiClient client = Config.defaultClient();
Configuration.setDefaultApiClient(client);
AppsV1Api appsV1Api = new AppsV1Api();
appsV1Api.deleteNamespacedStatefulSet("pod-go", "default", null, null, null, null, null, null);
return true;
}
3.6 线上运行注意
部署到线上后发现报错
进入pod
执行
[root@pod-java-67cdc944f8-jknpq app]# kubectl get pods
Error from server (Forbidden): pods is forbidden: User "system:serviceaccount:default:default" cannot list resource "pods" in API group "" in the namespace "default"
创建最高权限的角色
cat <<EOF> client-java-sa.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: client-java-sa
namespace: default
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: client-java-sa-clusterrolebinding
subjects:
- kind: ServiceAccount
name: client-java-sa
namespace: default
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io
EOF
部署
java
应用的时候使用client-java-sa
这个ServiceAccount
,否则会使用默认的default
这个ServiceAccount
添加serviceAccountName: client-java-sa
部署
kubectl apply -f client-java-sa.yaml
参考 https://blog.51cto.com/zq2599/4907211