部署Loki和Promtail到Kubernetes(K8s)的过程涉及创建ConfigMap、Secret(如果需要)、以及Deployment或StatefulSet资源。以下是一个简化的部署步骤和相关YAML配置文件示例:

步骤1:创建Loki的ConfigMap

首先,创建一个ConfigMap来存放Loki的配置文件。假设你已经有了一个名为loki-local-config.yaml的Loki配置文件,内容如下:

# loki-local-config.yaml
auth_enabled: false # 关闭认证,生产环境建议开启并使用Secret存储凭证
server:
  http_listen_port: 3100
  grpc_listen_port: 9095
chunk_store_config:
  max_look_back_period: 1h
schema_config:
  configs:
  - from: 2020-01-01
    store: boltdb-shipper
    object_store: filesystem
    schema: v11
    index:
      prefix: index_
      period: 168h
limits_config:
  reject_old_samples: true
  reject_old_samples_max_age: 168h
ingester:
  lifecycler:
    ring:
      kvstore:
        store: inmemory
compactor:
  working_directory: /loki/boltdb-shipper-working-dir
  shared_store: filesystem
table_manager:
  retention_deletes_enabled: true
  retention_period: 7d

创建ConfigMap:

kubectl create configmap loki-config --from-file=loki-local-config.yaml

步骤2:创建Loki的PersistentVolumeClaim (PVC)

(根据你的存储需求创建持久化存储资源,如果是本地存储或者动态PV,可以跳过这一步)

# pvc-loki.yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: loki-pvc
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: your-storage-class # 替换为你的存储类别
  resources:
    requests:
      storage: 10Gi # 配置所需存储大小
kubectl apply -f pvc-loki.yaml

步骤3:部署Loki

创建Loki的Deployment或StatefulSet资源:

# deployment-loki.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: loki
spec:
  replicas: 1
  selector:
    matchLabels:
      app: loki
  template:
    metadata:
      labels:
        app: loki
    spec:
      containers:
      - name: loki
        image: grafana/loki:latest
        ports:
        - name: loki-http
          containerPort: 3100
        - name: loki-grpc
          containerPort: 9095
        volumeMounts:
        - name: config-volume
          mountPath: /etc/loki
        - name: loki-storage
          mountPath: /loki
      volumes:
      - name: config-volume
        configMap:
          name: loki-config
      - name: loki-storage
        persistentVolumeClaim:
          claimName: loki-pvc
kubectl apply -f deployment-loki.yaml

步骤4:部署Promtail

创建Promtail的DaemonSet资源,并配置它从Kubernetes Pods中抓取日志:

# daemonset-promtail.yaml
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: promtail
spec:
  selector:
    matchLabels:
      app: promtail
  template:
    metadata:
      labels:
        app: promtail
    spec:
      serviceAccountName: promtail # 需要有一个service account并赋予适当权限
      hostNetwork: true # 为了访问宿主机日志
      containers:
      - name: promtail
        image: grafana/promtail:vX.Y.Z # 替换为合适的Promtail版本
        args:
        - -config.file=/etc/promtail/promtail.yaml
        volumeMounts:
        - name: config-volume
          mountPath: /etc/promtail/
        - name: varlog
          mountPath: /var/log
        - name: varlibdockercontainers
          mountPath: /var/lib/docker/containers
      volumes:
      - name: config-volume
        configMap:
          name: promtail-config
      - name: varlog
        hostPath:
          path: /var/log
      - name: varlibdockercontainers
        hostPath:
          path: /var/lib/docker/containers
---
# promtail-config.yaml (放在ConfigMap里的内容)
server:
  http_listen_port: 9080
scrape_configs:
- job_name: kubernetes-pods
  kubernetes_sd_configs:
  - role: pod
  relabel_configs:
  # 配置Promtail只抓取有特定注解的Pod日志
  - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
    action: keep
    regex: true
  - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
    target_label: __path__
    replacement: /var/log/pods/$${host}/$${pod_name}/$${container_name}/stdout.log
kubectl apply -f daemonset-promtail.yaml
kubectl create configmap promtail-config --from-file=promtail-config.yaml

注意事项:

  • 上述配置仅为基础示例,实际部署时需要根据自身Kubernetes集群的具体情况进行调整,比如存储配置、网络策略、安全设置等。
  • Promtail的配置文件promtail-config.yaml中定义了如何从Kubernetes Pods中抓取日志,可以根据实际需要调整relabel_configsscrape_configs部分。
  • 实际部署时,可能需要创建Service资源以方便内外部访问Loki服务。
  • 生产环境中,建议启用身份验证和加密传输,此时需要创建Secret来存储Loki的凭据,并在配置文件中引用这些凭据。