前言
- loki 是轻量、易用的日志聚合系统。如果你的k8s集群规模并不大,推荐使用grafana+loki的方案来做微服务日志的采集;
Loki组成
- loki架构很简单,主要由3部分组成:
- loki:服务端,负责存储日志和处理查询;
- promtail:采集端,负责采集日志发送给loki;
- grafana:负责采集日志的展示;
promtail.yml
kubectl apply -f promtail.yml
注意: 使用前,请删掉promtail.yml中的中文注释
---
apiVersion: v1
kind: ConfigMap
metadata:
name: promtail-configmap
data:
promtail-config.yml: |
server:
http_listen_port: 9080
grpc_listen_port: 0
positions:
filename: /tmp/positions.yaml
clients:
- url: https://{YOUR_LOKI_ENDPOINT}/loki/api/v1/push #输入自己的loki地址
scrape_configs:
- job_name: containers
static_configs:
- targets:
- localhost
labels:
log_from: static_pods
__path__: /var/log/pods/*/*/*.log
pipeline_stages:
- docker: {}
- match:
selector: '{log_from="static_pods"}'
stages:
- regex:
source: filename
expression: "(?:pods)/(?P<namespace>\\S+?)_(?P<pod>\\S+)-\\S+?-\\S+?\\S+?/(?P<container>\\S+?)/"
- labels:
namespace:
pod:
container:
- match:
selector: '{namespace!~"(default|kube-system|kube-systemen)"}' #输入自己的命名空间
action: drop
drop_counter_reason: no_use
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: promtail
namespace: kube-systemen
spec:
selector:
matchLabels:
name: promtail
template:
metadata:
labels:
name: promtail
spec:
nodeSelector:
promtail: promtail
containers:
- name: promtail
image: grafana/promtail:2.4.2
imagePullPolicy: Always
args:
- -config.file=/mnt/config/promtail-config.yml
- -client.external-labels=hostip=$(NODE_IP),hostname=$(NODE_NAME),env=STM
ports:
- containerPort: 9080
volumeMounts:
- mountPath: /opt/docker/containers
name: containers-volume
- mountPath: /var/lib/docker/containers
name: containers-volumes
- mountPath: /var/log/pods
name: pods-volume
- mountPath: /mnt/config
name: config-volume
env:
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: 'spec.nodeName'
- name: NODE_IP
valueFrom:
fieldRef:
fieldPath: 'spec.nodeIP'
volumes:
- name: containers-volume
hostPath:
path: /opt/docker/containers
- name: containers-volumes
hostPath:
path: /var/lib/docker/containers
- name: pods-volume
hostPath:
path: /var/log/pods
- name: config-volume
configMap:
name: promtail-configmap
items:
- key: promtail-config.yml
path: promtail-config.yml
tolerations:
- key: node-role.kubernetes.io/master
operator: Exists
errect: NoSchedule
官方文档:https://grafana.com/docs/loki/latest/send-data/promtail/installation/