部署Loki和Promtail到Kubernetes(K8s)的过程涉及创建ConfigMap、Secret(如果需要)、以及Deployment或StatefulSet资源。以下是一个简化的部署步骤和相关YAML配置文件示例:
步骤1:创建Loki的ConfigMap
首先,创建一个ConfigMap来存放Loki的配置文件。假设你已经有了一个名为loki-local-config.yaml
的Loki配置文件,内容如下:
# loki-local-config.yaml
auth_enabled: false # 关闭认证,生产环境建议开启并使用Secret存储凭证
server:
http_listen_port: 3100
grpc_listen_port: 9095
chunk_store_config:
max_look_back_period: 1h
schema_config:
configs:
- from: 2020-01-01
store: boltdb-shipper
object_store: filesystem
schema: v11
index:
prefix: index_
period: 168h
limits_config:
reject_old_samples: true
reject_old_samples_max_age: 168h
ingester:
lifecycler:
ring:
kvstore:
store: inmemory
compactor:
working_directory: /loki/boltdb-shipper-working-dir
shared_store: filesystem
table_manager:
retention_deletes_enabled: true
retention_period: 7d
创建ConfigMap:
kubectl create configmap loki-config --from-file=loki-local-config.yaml
步骤2:创建Loki的PersistentVolumeClaim (PVC)
(根据你的存储需求创建持久化存储资源,如果是本地存储或者动态PV,可以跳过这一步)
# pvc-loki.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: loki-pvc
spec:
accessModes:
- ReadWriteOnce
storageClassName: your-storage-class # 替换为你的存储类别
resources:
requests:
storage: 10Gi # 配置所需存储大小
kubectl apply -f pvc-loki.yaml
步骤3:部署Loki
创建Loki的Deployment或StatefulSet资源:
# deployment-loki.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: loki
spec:
replicas: 1
selector:
matchLabels:
app: loki
template:
metadata:
labels:
app: loki
spec:
containers:
- name: loki
image: grafana/loki:latest
ports:
- name: loki-http
containerPort: 3100
- name: loki-grpc
containerPort: 9095
volumeMounts:
- name: config-volume
mountPath: /etc/loki
- name: loki-storage
mountPath: /loki
volumes:
- name: config-volume
configMap:
name: loki-config
- name: loki-storage
persistentVolumeClaim:
claimName: loki-pvc
kubectl apply -f deployment-loki.yaml
步骤4:部署Promtail
创建Promtail的DaemonSet资源,并配置它从Kubernetes Pods中抓取日志:
# daemonset-promtail.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: promtail
spec:
selector:
matchLabels:
app: promtail
template:
metadata:
labels:
app: promtail
spec:
serviceAccountName: promtail # 需要有一个service account并赋予适当权限
hostNetwork: true # 为了访问宿主机日志
containers:
- name: promtail
image: grafana/promtail:vX.Y.Z # 替换为合适的Promtail版本
args:
- -config.file=/etc/promtail/promtail.yaml
volumeMounts:
- name: config-volume
mountPath: /etc/promtail/
- name: varlog
mountPath: /var/log
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
volumes:
- name: config-volume
configMap:
name: promtail-config
- name: varlog
hostPath:
path: /var/log
- name: varlibdockercontainers
hostPath:
path: /var/lib/docker/containers
---
# promtail-config.yaml (放在ConfigMap里的内容)
server:
http_listen_port: 9080
scrape_configs:
- job_name: kubernetes-pods
kubernetes_sd_configs:
- role: pod
relabel_configs:
# 配置Promtail只抓取有特定注解的Pod日志
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
action: keep
regex: true
- source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
target_label: __path__
replacement: /var/log/pods/$${host}/$${pod_name}/$${container_name}/stdout.log
kubectl apply -f daemonset-promtail.yaml
kubectl create configmap promtail-config --from-file=promtail-config.yaml
注意事项:
- 上述配置仅为基础示例,实际部署时需要根据自身Kubernetes集群的具体情况进行调整,比如存储配置、网络策略、安全设置等。
- Promtail的配置文件
promtail-config.yaml
中定义了如何从Kubernetes Pods中抓取日志,可以根据实际需要调整relabel_configs
和scrape_configs
部分。 - 实际部署时,可能需要创建Service资源以方便内外部访问Loki服务。
- 生产环境中,建议启用身份验证和加密传输,此时需要创建Secret来存储Loki的凭据,并在配置文件中引用这些凭据。