在生产环境中业务迁移至Kubernetes环境都需要提前规划机房kubernetes集群部署
- 基本步骤：
- 机房环境搭建
- 基础服务搭建
- 系统迁移
- 数据库迁移
- 测试及联调
- 使用服务及版本
- Pod地址规划
- 端口使用统计
- 业务迁移
Nginx+Tomcat+NFS实现动静分离
- 实现步骤：
- Centos 基础环境镜像制作
#准备安装包
[root@K8s-ansible centos]#ls
Dockerfile build-command.sh filebeat-7.12.1-x86_64.rpm
#准备环境初始化脚本
[root@K8s-ansible centos]#cat Dockerfile
#自定义Centos 基础镜像
FROM centos:7.9.2009
MAINTAINER mooreyxia mooreyxia@gmail.com
ADD filebeat-7.12.1-x86_64.rpm /tmp
RUN yum install -y /tmp/filebeat-7.12.1-x86_64.rpm vim wget tree lrzsz gcc gcc-c++ automake pcre pcre-devel zlib zlib-devel openssl openssl-devel iproute net-tools iotop && rm -rf /etc/localtime /tmp/filebeat-7.12.1-x86_64.rpm && ln -snf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime && useradd nginx -u 2088
#构建镜像并上传到harbor
[root@K8s-ansible centos]#cat build-command.sh
#!/bin/bash
docker build -t K8s-harbor01.mooreyxia.com/baseimages/centos-base:7.9.2009 .
docker push K8s-harbor01.mooreyxia.com/baseimages/centos-base:7.9.2009
#/usr/local/bin/nerdctl build -t K8s-harbor01.mooreyxia.com/baseimages/centos-base:7.9.2009 .
#/usr/local/bin/nerdctl push K8s-harbor01.mooreyxia.com/baseimages/centos-base:7.9.2009
[root@K8s-ansible centos]#bash build-command.sh
...
Complete!
Removing intermediate container 58b517f0b116
---> 2a553a8d53c1
Successfully built 2a553a8d53c1
Successfully tagged K8s-harbor01.mooreyxia.com/baseimages/centos-base:7.9.2009
The push refers to repository [K8s-harbor01.mooreyxia.com/baseimages/centos-base]
3e6fa9b75f89: Pushed
cf71274b159a: Pushed
174f56854903: Layer already exists
7.9.2009: digest: sha256:946a768695c6fd3570559569abba02a4530ee6a96f0b50bdc7a1cf6dd9c44749 size: 954- Nginx 基础镜像制作
#准备服务安装包
[root@K8s-ansible nginx-base]#ls
Dockerfile build-command.sh nginx-1.22.0.tar.gz
#准备环境初始化文件
[root@K8s-ansible nginx-base]#vim Dockerfile
[root@K8s-ansible nginx-base]#cat Dockerfile
#Nginx Base Image
FROM K8s-harbor01.mooreyxia.com/baseimages/centos-base:7.9.2009
MAINTAINER mooreyxia mooreyxia@gmail.com
RUN yum install -y vim wget tree lrzsz gcc gcc-c++ automake pcre pcre-devel zlib zlib-devel openssl openssl-devel iproute net-tools iotop
ADD nginx-1.22.0.tar.gz /usr/local/src/
RUN cd /usr/local/src/nginx-1.22.0 && ./configure && make && make install && ln -sv /usr/local/nginx/sbin/nginx /usr/sbin/nginx &&rm -rf /usr/local/src/nginx-1.22.0.tar.gz
#构建镜像并上传到harbor
[root@K8s-ansible nginx-base]#vim build-command.sh
[root@K8s-ansible nginx-base]#cat build-command.sh
#!/bin/bash
docker build -t K8s-harbor01.mooreyxia.com/pub-images/nginx-base:v1.22.0 .
sleep 1
docker push K8s-harbor01.mooreyxia.com/pub-images/nginx-base:v1.22.0
[root@K8s-ansible nginx-base]#bash build-command.sh
...
Successfully tagged K8s-harbor01.mooreyxia.com/pub-images/nginx-base:v1.22.0
The push refers to repository [K8s-harbor01.mooreyxia.com/pub-images/nginx-base]
74ac29189cb9: Pushed
65bd2886841a: Pushed
4236d10e4e6b: Pushed
3e6fa9b75f89: Mounted from demo/tomcat-app1
cf71274b159a: Mounted from demo/tomcat-app1
174f56854903: Mounted from demo/tomcat-app1
v1.22.0: digest: sha256:7f026afc340b237f41fc4a86289af0abc65c5da705ee5a8e9a267b077c95febb size: 1588- Nginx业务镜像制作
#准备构建镜像的文件
[root@K8s-ansible nginx]#ls
Dockerfile app1.tar.gz build-command.sh index.html nginx.conf webapp
#nginx配置文件
[root@K8s-ansible nginx]#cat nginx.conf
user nginx nginx;
worker_processes auto;
#error_log logs/error.log;
#error_log logs/error.log notice;
#error_log logs/error.log info;
#pid logs/nginx.pid;
daemon off; #开启前台运行
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
#log_format main '$remote_addr - $remote_user [$time_local] "$request" '
# '$status $body_bytes_sent "$http_referer" '
# '"$http_user_agent" "$http_x_forwarded_for"';
#access_log logs/access.log main;
sendfile on;
#tcp_nopush on;
#keepalive_timeout 0;
keepalive_timeout 65;
#gzip on;
upstream tomcat_webserver { #服务调度到pod
server mooreyxia-tomcat-app1-service.mooreyxia.svc.mooreyxia.local:80;
}
server {
listen 80;
server_name localhost;
#charset koi8-r;
#access_log logs/host.access.log main;
location / {
root html;
index index.html index.htm;
}
location /webapp {
root html;
index index.html index.htm;
}
location /myapp {
proxy_pass http://tomcat_webserver; #转发到服务
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
}
...
}
-------------------------查找域名-------------------------------------------
[root@K8s-ansible nginx-base]#kubectl exec -it net-test1 -n myserver bash
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
[root@net-test1 /]# cat /etc/resolv.conf
search myserver.svc.mooreyxia.local svc.mooreyxia.local mooreyxia.local mooreyxia.org mooreyxia.com
nameserver 10.100.0.2
options ndots:5
#测试域名是否正确
[root@net-test1 /]# ping mooreyxia-tomcat-app1-service.mooreyxia.svc.mooreyxia.local
PING mooreyxia-tomcat-app1-service.mooreyxia.svc.mooreyxia.local (10.100.168.87) 56(84) bytes of data.
-------------------------查找域名-------------------------------------------
#准备业务文件
[root@K8s-ansible nginx]#cat index.html
nginx web1 mooreyxia n70 v1
[root@K8s-ansible nginx]#cat webapp/index.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Devops</title>
</head>
<body>
<h1>mooreyxia devops v11111111</h1>
</body>
</html>
#构建文档
[root@K8s-ansible nginx]#cat Dockerfile
#Nginx 1.22.0
FROM K8s-harbor01.mooreyxia.com/pub-images/nginx-base:v1.22.0
ADD nginx.conf /usr/local/nginx/conf/nginx.conf #配置文件
ADD app1.tar.gz /usr/local/nginx/html/webapp/ #业务文件 - 及其不推荐存放自动生成文件的业务代码，有需要就挂载到其他存储目录，否则会使得镜像过分增大，影响编译加载效率
ADD index.html /usr/local/nginx/html/index.html #首页
#静态资源挂载路径
RUN mkdir -p /usr/local/nginx/html/webapp/static /usr/local/nginx/html/webapp/images
EXPOSE 80 443
CMD ["nginx"]
#构建镜像并上传到harbor
[root@K8s-ansible nginx]#cat build-command.sh
#!/bin/bash
TAG=$1
docker build -t K8s-harbor01.mooreyxia.com/demo/nginx-web1:${TAG} .
echo "镜像构建完成，即将上传到harbor"
sleep 1
docker push K8s-harbor01.mooreyxia.com/demo/nginx-web1:${TAG}
echo "镜像上传到harbor完成"
[root@K8s-ansible nginx]#bash build-command.sh v1
...
v1: digest: sha256:dbf2d235debb861f7a251eadc54b71fc6d23b74b2c4000c53d33651872f2305a size: 2417
镜像上传到harbor完成- Nginx业务镜像测试，在kubernetes环境运行nginx
#测试镜像
#由于nginx中的用于转发的service需要在kubernetes集群内使用，创建pod测试
[root@K8s-ansible nginx]#cat nginx2.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
labels:
app: mooreyxia-nginx-deployment-label
name: mooreyxia-nginx-deployment
namespace: mooreyxia
spec:
replicas: 1
selector:
matchLabels:
app: mooreyxia-nginx-selector
template:
metadata:
labels:
app: mooreyxia-nginx-selector
spec:
containers:
- name: mooreyxia-nginx-container
image: K8s-harbor01.mooreyxia.com/demo/nginx-web1:v1
#command: ["/apps/tomcat/bin/run_tomcat.sh"]
#imagePullPolicy: IfNotPresent
imagePullPolicy: Always
ports:
- containerPort: 80
protocol: TCP
name: http
- containerPort: 443
protocol: TCP
name: https
env:
- name: "password"
value: "123456"
- name: "age"
value: "20"
#resources:
# limits:
# cpu: 500m
# memory: 512Mi
# requests:
# cpu: 500m
# memory: 256Mi
volumeMounts:
- name: mooreyxia-images
mountPath: /usr/local/nginx/html/webapp/images
readOnly: false
- name: mooreyxia-static
mountPath: /usr/local/nginx/html/webapp/static
readOnly: false
volumes:
- name: mooreyxia-images
nfs:
server: 192.168.11.203
path: /data/k8sdata/mooreyxia/images
- name: mooreyxia-static
nfs:
server: 192.168.11.203
path: /data/k8sdata/mooreyxia/static
#nodeSelector:
# group: mooreyxia
---
kind: Service
apiVersion: v1
metadata:
labels:
app: mooreyxia-nginx-service-label
name: mooreyxia-nginx-service
namespace: mooreyxia
spec:
type: NodePort
ports:
- name: http
port: 80
protocol: TCP
targetPort: 80
nodePort: 30090
- name: https
port: 443
protocol: TCP
targetPort: 443
nodePort: 30091
selector:
app: mooreyxia-nginx-selector
#创建并运行pod
[root@K8s-ansible nginx]#kubectl apply -f nginx2.yaml
deployment.apps/mooreyxia-nginx-deployment created
service/mooreyxia-nginx-service created
#查看运行状况
[root@K8s-ansible nginx]#kubectl describe pod mooreyxia-nginx-deployment-76669f8678-rsmwh -n mooreyxia
Name: mooreyxia-nginx-deployment-76669f8678-rsmwh
Namespace: mooreyxia
Priority: 0
Service Account: default
Node: 192.168.11.216/192.168.11.216
Start Time: Sun, 09 Apr 2023 08:29:23 +0000
Labels: app=mooreyxia-nginx-selector
pod-template-hash=76669f8678
Annotations: <none>
Status: Running
IP: 10.200.128.172
IPs:
IP: 10.200.128.172
Controlled By: ReplicaSet/mooreyxia-nginx-deployment-76669f8678
Containers:
mooreyxia-nginx-container:
Container ID: containerd://78026afe0814c53466c7db6649fb9fed3b2b65b28443e2fdc1922d1ca4d96b81
Image: K8s-harbor01.mooreyxia.com/demo/nginx-web1:v1
Image ID: K8s-harbor01.mooreyxia.com/demo/nginx-web1@sha256:dbf2d235debb861f7a251eadc54b71fc6d23b74b2c4000c53d33651872f2305a
Ports: 80/TCP, 443/TCP
Host Ports: 0/TCP, 0/TCP
State: Running
Started: Sun, 09 Apr 2023 08:29:57 +0000
Ready: True
Restart Count: 0
Environment:
password: 123456
age: 20
Mounts:
/usr/local/nginx/html/webapp/images from mooreyxia-images (rw)
/usr/local/nginx/html/webapp/static from mooreyxia-static (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-dvzbg (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
mooreyxia-images:
Type: NFS (an NFS mount that lasts the lifetime of a pod)
Server: 192.168.11.203
Path: /data/k8sdata/mooreyxia/images
ReadOnly: false
mooreyxia-static:
Type: NFS (an NFS mount that lasts the lifetime of a pod)
Server: 192.168.11.203
Path: /data/k8sdata/mooreyxia/static
ReadOnly: false
kube-api-access-dvzbg:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 70s default-scheduler Successfully assigned mooreyxia/mooreyxia-nginx-deployment-76669f8678-rsmwh to 192.168.11.216
Normal Pulling 69s kubelet Pulling image "K8s-harbor01.mooreyxia.com/demo/nginx-web1:v1"
Normal Pulled 37s kubelet Successfully pulled image "K8s-harbor01.mooreyxia.com/demo/nginx-web1:v1" in 31.548427872s (31.548473114s including waiting)
Normal Created 37s kubelet Created container mooreyxia-nginx-container
Normal Started 34s kubelet Started container mooreyxia-nginx-container
#测试访问nginx-pod的30090端口
http://NodeIP:30090/
- 在kubernetes中nginx+tomcat实现动静分离
#确认Pod可以访问后加入负载均衡器
[root@K8s-haproxy01 ~]#cat /etc/haproxy/haproxy.cfg
...
listen myserver-80
bind 192.168.11.242:80
mode tcp
server K8s-master01 192.168.11.211:30090 check inter 3000 fall 2 rise 5
server K8s-master02 192.168.11.212:30090 check inter 3000 fall 2 rise 5
server K8s-master03 192.168.11.213:30090 check inter 3000 fall 2 rise 5
#如果有配置https的话加443端口
listen myserver-443
bind 192.168.11.242:443
mode tcp
server K8s-master01 192.168.11.211:30091 check inter 3000 fall 2 rise 5
server K8s-master02 192.168.11.212:30091 check inter 3000 fall 2 rise 5
server K8s-master03 192.168.11.213:30091 check inter 3000 fall 2 rise 5
#重启haproxy
[root@K8s-haproxy01 ~]#systemctl restart haproxy
#测试访问负载均衡地址
http://192.168.11.242/
#确认pod内nginx是否有调度到tomcat业务
http://192.168.11.242/myapp/
#之后可以将负载均衡地址解析到防火墙就可以对外使用了
此处省略
- JDK基础镜像制作
#准备安装包
[root@K8s-ansible jdk-1.8.212]#ls
Dockerfile build-command.sh jdk-8u212-linux-x64.tar.gz profile
#准备环境变量脚本
[root@K8s-ansible jdk-1.8.212]#cat profile
# /etc/profile
# System wide environment and startup programs, for login setup
# Functions and aliases go in /etc/bashrc
# It's NOT a good idea to change this file unless you know what you
# are doing. It's much better to create a custom.sh shell script in
# /etc/profile.d/ to make custom changes to your environment, as this
# will prevent the need for merging in future updates.
pathmunge () {
case ":${PATH}:" in
*:"$1":*)
;;
*)
if [ "$2" = "after" ] ; then
PATH=$PATH:$1
else
PATH=$1:$PATH
fi
esac
}
if [ -x /usr/bin/id ]; then
if [ -z "$EUID" ]; then
# ksh workaround
EUID=`/usr/bin/id -u`
UID=`/usr/bin/id -ru`
fi
USER="`/usr/bin/id -un`"
LOGNAME=$USER
MAIL="/var/spool/mail/$USER"
fi
# Path manipulation
if [ "$EUID" = "0" ]; then
pathmunge /usr/sbin
pathmunge /usr/local/sbin
else
pathmunge /usr/local/sbin after
pathmunge /usr/sbin after
fi
HOSTNAME=`/usr/bin/hostname 2>/dev/null`
HISTSIZE=1000
if [ "$HISTCONTROL" = "ignorespace" ] ; then
export HISTCONTROL=ignoreboth
else
export HISTCONTROL=ignoredups
fi
export PATH USER LOGNAME MAIL HOSTNAME HISTSIZE HISTCONTROL
# By default, we want umask to get set. This sets it for login shell
# Current threshold for system reserved uid/gids is 200
# You could check uidgid reservation validity in
# /usr/share/doc/setup-*/uidgid file
if [ $UID -gt 199 ] && [ "`/usr/bin/id -gn`" = "`/usr/bin/id -un`" ]; then
umask 002
else
umask 022
fi
for i in /etc/profile.d/*.sh /etc/profile.d/sh.local ; do
if [ -r "$i" ]; then
if [ "${-#*i}" != "$-" ]; then
. "$i"
else
. "$i" >/dev/null
fi
fi
done
unset i
unset -f pathmunge
export LANG=en_US.UTF-8
export HISTTIMEFORMAT="%F %T `whoami` "
export JAVA_HOME=/usr/local/jdk
export TOMCAT_HOME=/apps/tomcat
export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$TOMCAT_HOME/bin:$PATH
export CLASSPATH=.$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib:$JAVA_HOME/lib/tools.jar
#准备环境初始化脚本
[root@K8s-ansible jdk-1.8.212]#cat Dockerfile
#JDK Base Image
FROM K8s-harbor01.mooreyxia.com/baseimages/centos-base:7.9.2009
#FROM centos:7.9.2009
MAINTAINER mooreyxia mooreyxia@gmail.com
ADD jdk-8u212-linux-x64.tar.gz /usr/local/src/
RUN ln -sv /usr/local/src/jdk1.8.0_212 /usr/local/jdk
ADD profile /etc/profile
ENV JAVA_HOME /usr/local/jdk
ENV JRE_HOME $JAVA_HOME/jre
ENV CLASSPATH $JAVA_HOME/lib/:$JRE_HOME/lib/
ENV PATH $PATH:$JAVA_HOME/bin
#构建镜像并上传到harbor
[root@K8s-ansible jdk-1.8.212]#cat build-command.sh
#!/bin/bash
docker build -t K8s-harbor01.mooreyxia.com/pub-images/jdk-base:v8.212 .
sleep 1
docker push K8s-harbor01.mooreyxia.com/pub-images/jdk-base:v8.212 #pub-images需要在harbor上提前建立
#nerdctl build -t K8s-harbor01.mooreyxia.com/pub-images/jdk-base:v8.212 .
#nerdctl push K8s-harbor01.mooreyxia.com/pub-images/jdk-base:v8.212
[root@K8s-ansible jdk-1.8.212]#bash build-command.sh- tomcat基础镜像制作
#准备安装包及环境初始化文件
[root@K8s-ansible tomcat-base-8.5.43]#ls
Dockerfile apache-tomcat-8.5.43.tar.gz build-command.sh
[root@K8s-ansible tomcat-base-8.5.43]#vim Dockerfile
[root@K8s-ansible tomcat-base-8.5.43]#cat Dockerfile
#Tomcat 8.5.43基础镜像
FROM K8s-harbor01.mooreyxia.com/pub-images/jdk-base:v8.212
MAINTAINER mooreyxia mooreyxia@gmail.com
RUN mkdir /apps /data/tomcat/webapps /data/tomcat/logs -pv
ADD apache-tomcat-8.5.43.tar.gz /apps
RUN useradd tomcat -u 2050 && ln -sv /apps/apache-tomcat-8.5.43 /apps/tomcat && chown -R tomcat.tomcat /apps /data -R
#生成镜像并上传到harbor
[root@K8s-ansible tomcat-base-8.5.43]#vim build-command.sh
[root@K8s-ansible tomcat-base-8.5.43]#cat build-command.sh
#!/bin/bash
docker build -t K8s-harbor01.mooreyxia.com/pub-images/tomcat-base:v8.5.43 .
sleep 3
docker push K8s-harbor01.mooreyxia.com/pub-images/tomcat-base:v8.5.43
#nerdctl build -t K8s-harbor01.mooreyxia.com/pub-images/tomcat-base:v8.5.43 .
#nerdctl push K8s-harbor01.mooreyxia.com/pub-images/tomcat-base:v8.5.43
[root@K8s-ansible tomcat-base-8.5.43]#bash build-command.sh - tomcat业务镜像app1制作
[root@K8s-ansible tomcat-app1]#ls
Dockerfile app1.tar.gz build-command.sh catalina.sh filebeat-7.5.1-x86_64.rpm filebeat.yml index.html myapp run_tomcat.sh server.xml
[root@K8s-ansible tomcat-app1]#cat myapp/index.html
tomcat app1 for linux mooreyxia
[root@K8s-ansible tomcat-app1]#cp myapp/app1.tar.gz .
[root@K8s-ansible tomcat-app1]#mv app1.tar.gz /tmp/
[root@K8s-ansible tomcat-app1]#cd /tmp/
#确定了目录结构不要轻易更改，否则资源访问路径可能会发生变化
[root@K8s-ansible tmp]#tar xvf app1.tar.gz
./
./index.html
#日志收集 - 暂时不用
[root@K8s-ansible tomcat-app1]#cat filebeat.yml
filebeat.inputs:
- type: log
enabled: true
paths:
- /apps/tomcat/logs/catalina.out
fields:
type: tomcat-catalina
- type: log
enabled: true
paths:
- /apps/tomcat/logs/localhost_access_log.*.txt
fields:
type: tomcat-accesslog
filebeat.config.modules:
path: ${path.config}/modules.d/*.yml
reload.enabled: false
setup.template.settings:
index.number_of_shards: 1
setup.kibana:
output.kafka:
hosts: ["172.31.4.101:9092"]
required_acks: 1
topic: "magedu-n56-app1"
compression: gzip
max_message_bytes: 1000000
#output.redis:
# hosts: ["172.31.2.105:6379"]
# key: "k8s-magedu-app1"
# db: 1
# timeout: 5
# password: "123456"
#tomcat运行命令脚本 - 去其他环境复制一份
[root@K8s-ansible tomcat-app1]#ll catalina.sh
-rwxr-xr-x 1 root root 23611 Apr 9 02:59 catalina.sh*
#准备环境初始化文件
[root@K8s-ansible tomcat-app1]#cat Dockerfile
#tomcat web1
FROM K8s-harbor01.mooreyxia.com/pub-images/tomcat-base:v8.5.43
ADD catalina.sh /apps/tomcat/bin/catalina.sh
ADD server.xml /apps/tomcat/conf/server.xml
#ADD myapp/* /data/tomcat/webapps/myapp/
ADD app1.tar.gz /data/tomcat/webapps/myapp/
ADD run_tomcat.sh /apps/tomcat/bin/run_tomcat.sh
#ADD filebeat.yml /etc/filebeat/filebeat.yml
RUN chown -R nginx.nginx /data/ /apps/
#ADD filebeat-7.5.1-x86_64.rpm /tmp/
#RUN cd /tmp && yum localinstall -y filebeat-7.5.1-amd64.deb
EXPOSE 8080 8443
CMD ["/apps/tomcat/bin/run_tomcat.sh"]
#开始构建
[root@K8s-ansible tomcat-app1]#cat build-command.sh
#!/bin/bash
TAG=$1
docker build -t K8s-harbor01.mooreyxia.com/demo/tomcat-app1:${TAG} .
sleep 3
docker push K8s-harbor01.mooreyxia.com/demo/tomcat-app1:${TAG}
[root@K8s-ansible tomcat-app1]#bash build-command.sh v1
...
#测试镜像
http://192.168.11.205:8080/myapp/
- 在kubernetes环境运行tomcat
#准备Kubernetes对象控制脚本
[root@K8s-ansible tomcat-app1]#cat tomcat-app1.yaml
kind: Deployment
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
metadata:
labels:
app: mooreyxia-tomcat-app1-deployment-label
name: mooreyxia-tomcat-app1-deployment
namespace: mooreyxia
spec:
replicas: 1
selector:
matchLabels:
app: mooreyxia-tomcat-app1-selector
template:
metadata:
labels:
app: mooreyxia-tomcat-app1-selector
spec:
containers:
- name: mooreyxia-tomcat-app1-container
image: K8s-harbor01.mooreyxia.com/demo/tomcat-app1:v1 #业务读写已提前打入镜像
#command: ["/apps/tomcat/bin/run_tomcat.sh"]
imagePullPolicy: IfNotPresent
#imagePullPolicy: Always
ports:
- containerPort: 8080
protocol: TCP
name: http
env:
- name: "password"
value: "123456"
- name: "age"
value: "18"
#resources:
# limits:
# cpu: 1
# memory: "512Mi"
# requests:
# cpu: 500m
# memory: "512Mi"
volumeMounts:
- name: mooreyxia-images
mountPath: /usr/local/nginx/html/webapp/images #读
readOnly: false
- name: mooreyxia-static
mountPath: /usr/local/nginx/html/webapp/static #写
readOnly: false
volumes:
- name: mooreyxia-images
nfs:
server: 192.168.11.203
path: /data/k8sdata/mooreyxia/images
- name: mooreyxia-static
nfs:
server: 192.168.11.203
path: /data/k8sdata/mooreyxia/static
# nodeSelector:
# project: mooreyxia
# app: tomcat
---
kind: Service
apiVersion: v1
metadata:
labels:
app: mooreyxia-tomcat-app1-service-label
name: mooreyxia-tomcat-app1-service
namespace: mooreyxia
spec:
type: NodePort #开放测试用，测试完毕后注释改用cluster-ip
ports:
- name: http
port: 80
protocol: TCP
targetPort: 8080
nodePort: 30092
selector:
app: mooreyxia-tomcat-app1-selector
#确认NFS服务
[root@K8s-ansible tomcat-app1]#showmount -e 192.168.11.203
Export list for 192.168.11.203:
/data/volumes *
/data/k8sdata *
#创建pod并运行
[root@K8s-ansible tomcat-app1]#kubectl create namespace mooreyxia
namespace/mooreyxia created
[root@K8s-ansible tomcat-app1]#kubectl apply -f tomcat-app1.yaml
deployment.apps/mooreyxia-tomcat-app1-deployment created
service/mooreyxia-tomcat-app1-service created
[root@K8s-ansible tomcat-app1]#kubectl get svc -n mooreyxia
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
mooreyxia-tomcat-app1-service NodePort 10.100.168.87 <none> 80:30092/TCP 6m26s
#查看运行情况
[root@K8s-ansible tomcat-app1]#kubectl describe pod mooreyxia-tomcat-app1-deployment-6448dfbc76-sgwq8 -n mooreyxia
Name: mooreyxia-tomcat-app1-deployment-6448dfbc76-sgwq8
Namespace: mooreyxia
Priority: 0
Service Account: default
Node: 192.168.11.215/192.168.11.215
Start Time: Sun, 09 Apr 2023 07:16:03 +0000
Labels: app=mooreyxia-tomcat-app1-selector
pod-template-hash=6448dfbc76
Annotations: <none>
Status: Running
IP: 10.200.67.32
IPs:
IP: 10.200.67.32
Controlled By: ReplicaSet/mooreyxia-tomcat-app1-deployment-6448dfbc76
Containers:
mooreyxia-tomcat-app1-container:
Container ID: containerd://58b6b444a17f568f71b529ba1eb2cca8d9bc198cedaf74069cd9599fd1412bdb
Image: K8s-harbor01.mooreyxia.com/demo/tomcat-app1:v1
Image ID: K8s-harbor01.mooreyxia.com/demo/tomcat-app1@sha256:825d210c5ad9052c8d608d87f67749ff41343563e6585e22fd8ca99640207a8b
Port: 8080/TCP
Host Port: 0/TCP
State: Running
Started: Sun, 09 Apr 2023 07:17:10 +0000
Ready: True
Restart Count: 0
Environment:
password: 123456
age: 18
Mounts:
/usr/local/nginx/html/webapp/images from mooreyxia-images (rw)
/usr/local/nginx/html/webapp/static from mooreyxia-static (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-hkwcr (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
mooreyxia-images:
Type: NFS (an NFS mount that lasts the lifetime of a pod)
Server: 192.168.11.203
Path: /data/k8sdata/mooreyxia/images
ReadOnly: false
mooreyxia-static:
Type: NFS (an NFS mount that lasts the lifetime of a pod)
Server: 192.168.11.203
Path: /data/k8sdata/mooreyxia/static
ReadOnly: false
kube-api-access-hkwcr:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 98s default-scheduler Successfully assigned mooreyxia/mooreyxia-tomcat-app1-deployment-6448dfbc76-sgwq8 to 192.168.11.215
Normal Pulling 96s kubelet Pulling image "K8s-harbor01.mooreyxia.com/demo/tomcat-app1:v1"
Normal Pulled 31s kubelet Successfully pulled image "K8s-harbor01.mooreyxia.com/demo/tomcat-app1:v1" in 1m4.963821559s (1m4.963854218s including waiting)
Normal Created 31s kubelet Created container mooreyxia-tomcat-app1-container
Normal Started 31s kubelet Started container mooreyxia-tomcat-app1-container
#测试访问任意node节点的30092端口
NodeIP:30092/myapp
- 基于NFS实现数据共享
#NFS准备资源文件
[root@K8s-haproxy01 ~]#mkdir -p /data/k8sdata/mooreyxia/images
[root@K8s-haproxy01 ~]#mkdir -p /data/k8sdata/mooreyxia/static
[root@K8s-haproxy01 ~]#tree /data/k8sdata/mooreyxia
/data/k8sdata/mooreyxia
├── images
└── static
2 directories, 0 files
[root@K8s-haproxy01 ~]#vim /etc/exports
[root@K8s-haproxy01 ~]#cat /etc/exports
# /etc/exports: the access control list for filesystems which may be exported
# to NFS clients. See exports(5).
#
# Example for NFSv2 and NFSv3:
# /srv/homes hostname1(rw,sync,no_subtree_check) hostname2(ro,sync,no_subtree_check)
#
# Example for NFSv4:
# /srv/nfs4 gss/krb5i(rw,sync,fsid=0,crossmnt,no_subtree_check)
# /srv/nfs4/homes gss/krb5i(rw,sync,no_subtree_check)
#
/data/k8sdata *(rw,no_root_squash)
/data/volumes *(rw,no_root_squash)
#设置后重新载入NFS资源
[root@K8s-haproxy01 ~]#exportfs -avs
exportfs: /etc/exports [2]: Neither 'subtree_check' or 'no_subtree_check' specified for export "*:/data/k8sdata".
Assuming default behaviour ('no_subtree_check').
NOTE: this default has changed since nfs-utils version 1.0.x
exportfs: /etc/exports [3]: Neither 'subtree_check' or 'no_subtree_check' specified for export "*:/data/volumes".
Assuming default behaviour ('no_subtree_check').
NOTE: this default has changed since nfs-utils version 1.0.x
exporting *:/data/volumes
exporting *:/data/k8sdata- 在后端服务生成数据并访问验证
#进去pod-tomcat挂载目录下载数据
[root@K8s-ansible ~]#kubectl get pod -n mooreyxia
NAME READY STATUS RESTARTS AGE
mooreyxia-nginx-deployment-76669f8678-rsmwh 1/1 Running 0 21m
mooreyxia-tomcat-app1-deployment-6448dfbc76-sgwq8 1/1 Running 0 95m
[root@K8s-ansible ~]#kubectl exec -it mooreyxia-tomcat-app1-deployment-6448dfbc76-sgwq8 bash -n mooreyxia
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
[root@mooreyxia-tomcat-app1-deployment-6448dfbc76-sgwq8 /]# df
Filesystem 1K-blocks Used Available Use% Mounted on
overlay 59969748 9212676 48174188 17% /
tmpfs 65536 0 65536 0% /dev
/dev/mapper/ubuntu--vg-ubuntu--lv 59969748 9212676 48174188 17% /etc/hosts
shm 65536 0 65536 0% /dev/shm
tmpfs 1715928 12 1715916 1% /run/secrets/kubernetes.io/serviceaccount
192.168.11.203:/data/k8sdata/mooreyxia/images 59969792 5516288 51869440 10% /usr/local/nginx/html/webapp/images
192.168.11.203:/data/k8sdata/mooreyxia/static 59969792 5516288 51869440 10% /usr/local/nginx/html/webapp/static
tmpfs 1011564 0 1011564 0% /proc/acpi
tmpfs 1011564 0 1011564 0% /proc/scsi
tmpfs 1011564 0 1011564 0% /sys/firmware
[root@mooreyxia-tomcat-app1-deployment-6448dfbc76-sgwq8 /]# cd /usr/local/nginx/html/webapp/images
[root@mooreyxia-tomcat-app1-deployment-6448dfbc76-sgwq8 images]# wget https://s2.51cto.com/oss/202210/01/e105cfecae44666d619c86ade0254bcf.jpg?x-oss-process=image/format,webp/ignore-error,1
--2023-04-09 16:54:29-- https://s2.51cto.com/oss/202210/01/e105cfecae44666d619c86ade0254bcf.jpg?x-oss-process=image/format,webp/ignore-error,1
Resolving s2.51cto.com (s2.51cto.com)... 222.184.83.75, 222.184.83.76
Connecting to s2.51cto.com (s2.51cto.com)|222.184.83.75|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 7136 (7.0K) [image/webp]
Saving to: 'e105cfecae44666d619c86ade0254bcf.jpg?x-oss-process=image%2Fformat,webp%2Fignore-error,1'
100%[=========================================================================================================================================>] 7,136 --.-K/s in 0s
2023-04-09 16:54:31 (215 MB/s) - 'e105cfecae44666d619c86ade0254bcf.jpg?x-oss-process=image%2Fformat,webp%2Fignore-error,1' saved [7136/7136]
[root@mooreyxia-tomcat-app1-deployment-6448dfbc76-sgwq8 images]# mv e105cfecae44666d619c86ade0254bcf.jpg\?x-oss-process\=image%2Fformat\,webp%2Fignore-error\,1 mooreyxia.jpg
[root@mooreyxia-tomcat-app1-deployment-6448dfbc76-sgwq8 images]# ls
mooreyxia.jpg
#确认是否存放到NFS
[root@K8s-haproxy01 ~]#ll /data/k8sdata/mooreyxia/images
total 16
drwxr-xr-x 2 root root 4096 Apr 9 08:54 ./
drwxr-xr-x 4 root root 4096 Apr 9 06:41 ../
-rw-r--r-- 1 root root 7136 Oct 1 2022 mooreyxia.jpg
#测试访问
http://192.168.11.242/myapp/images/mooreyxia.jpg我是moore，大家一起加油！！！
