在容器化环境中，工作负载的瞬时特性使得日志管理面临独特挑战：Pod生命周期短暂、日志分散存储、故障定位困难。构建统一的日志汇聚与分析平台，能够实现全链路追踪、异常检测及容量规划，是保障云原生应用可观测性的关键环节。

日志采集架构选型

针对Kubernetes场景，业界主要存在三种技术路线：

方案	实现方式	适用场景	资源开销
节点级代理	DaemonSet部署日志采集器	中小规模集群	低（每节点单实例）
容器级伴生	Sidecar模式注入采集容器	多租户隔离场景	高（随Pod数量线性增长）
应用内嵌	SDK直写日志后端	超高吞吐量场景	依赖业务改造

节点级代理方案因部署简洁、资源可控，成为多数生产环境的首选。以下演示基于该模式搭建完整的EFK技术栈。

Elasticsearch单实例部署

创建专用命名空间并部署搜索引擎：

apiVersion: v1
kind: Namespace
metadata:
  name: logging-stack

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: es-storage
  namespace: logging-stack
spec:
  storageClassName: "nfs-client"
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 20Gi

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: es-master
  namespace: logging-stack
spec:
  replicas: 1
  selector:
    matchLabels:
      app: elasticsearch
  template:
    metadata:
      labels:
        app: elasticsearch
    spec:
      containers:
      - name: es-node
        image: docker.elastic.co/elasticsearch/elasticsearch:7.17.22
        env:
        - name: "discovery.type"
          value: "single-node"
        - name: "cluster.name"
          value: "k8s-logs"
        - name: "bootstrap.memory_lock"
          value: "false"
        - name: "ES_JAVA_OPTS"
          value: "-Xms512m -Xmx512m"
        resources:
          limits:
            cpu: "2"
            memory: 4Gi
          requests:
            cpu: "500m"
            memory: 1Gi
        ports:
        - containerPort: 9200
          name: rest-api
        volumeMounts:
        - name: data-vol
          mountPath: /usr/share/elasticsearch/data
      volumes:
      - name: data-vol
        persistentVolumeClaim:
          claimName: es-storage

---
apiVersion: v1
kind: Service
metadata:
  name: es-endpoint
  namespace: logging-stack
spec:
  ports:
  - port: 9200
    targetPort: 9200
  selector:
    app: elasticsearch

Kibana可视化层配置

部署日志查询与仪表盘界面：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: kibana-ui
  namespace: logging-stack
spec:
  replicas: 1
  selector:
    matchLabels:
      app: kibana
  template:
    metadata:
      labels:
        app: kibana
    spec:
      containers:
      - name: web-server
        image: docker.elastic.co/kibana/kibana:7.17.22
        env:
        - name: ELASTICSEARCH_HOSTS
          value: '["http://es-endpoint.logging-stack.svc:9200"]'
        - name: I18N_LOCALE
          value: "zh-CN"
        - name: SERVER_PUBLICBASEURL
          value: "http://kibana.logging-stack.svc:5601"
        resources:
          limits:
            cpu: "1"
            memory: 2Gi
          requests:
            cpu: "250m"
            memory: 512Mi
        ports:
        - containerPort: 5601
          name: http-port

---
apiVersion: v1
kind: Service
metadata:
  name: kibana-svc
  namespace: logging-stack
spec:
  type: NodePort
  ports:
  - port: 5601
    targetPort: 5601
    nodePort: 30601
  selector:
    app: kibana

Filebeat采集代理部署

通过DaemonSet实现全节点覆盖的日志抓取：

apiVersion: v1
kind: ConfigMap
metadata:
  name: fb-main-conf
  namespace: logging-stack
data:
  filebeat.yml: |
    filebeat.inputs:
    - type: filestream
      id: container-logs
      paths:
        - /var/log/containers/*.log
      parsers:
        - container:
            stream: all
            format: auto
      processors:
        - add_kubernetes_metadata:
            host: ${NODE_NAME}
            matchers:
            - logs_path:
                logs_path: "/var/log/containers/"
    
    output.elasticsearch:
      hosts: ["es-endpoint.logging-stack.svc:9200"]
      index: "container-logs-%{+yyyy.MM.dd}"
    
    setup.ilm.enabled: false
    setup.template.name: "container-logs"
    setup.template.pattern: "container-logs-*"
    setup.template.settings:
      index.number_of_shards: 3
      index.number_of_replicas: 0

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: log-collector-role
rules:
- apiGroups: [""]
  resources: ["namespaces", "pods", "nodes"]
  verbs: ["get", "list", "watch"]

---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: log-collector-binding
subjects:
- kind: ServiceAccount
  name: log-agent
  namespace: logging-stack
roleRef:
  kind: ClusterRole
  name: log-collector-role
  apiGroup: rbac.authorization.k8s.io

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: log-agent
  namespace: logging-stack

---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: log-agent
  namespace: logging-stack
spec:
  selector:
    matchLabels:
      app: log-shipper
  template:
    metadata:
      labels:
        app: log-shipper
    spec:
      serviceAccountName: log-agent
      terminationGracePeriodSeconds: 60
      tolerations:
      - operator: Exists
        effect: NoSchedule
      containers:
      - name: shipper
        image: docker.elastic.co/beats/filebeat:7.17.22
        args: ["-c", "/etc/filebeat/filebeat.yml", "-e"]
        securityContext:
          runAsUser: 0
          privileged: true
        resources:
          limits:
            memory: 256Mi
            cpu: 500m
          requests:
            memory: 128Mi
            cpu: 100m
        env:
        - name: NODE_NAME
          valueFrom:
            fieldRef:
              fieldPath: spec.nodeName
        volumeMounts:
        - name: config-vol
          mountPath: /etc/filebeat
          readOnly: true
        - name: container-logs
          mountPath: /var/log/containers
          readOnly: true
        - name: pod-logs
          mountPath: /var/log/pods
          readOnly: true
      volumes:
      - name: config-vol
        configMap:
          name: fb-main-conf
          defaultMode: 0400
      - name: container-logs
        hostPath:
          path: /var/log/containers
      - name: pod-logs
        hostPath:
          path: /var/log/pods

部署完成后，通过NodePort访问Kibana界面，创建索引模式container-logs-*即可检索全集群容器标准输出日志。该架构支持动态扩缩容，新增节点自动纳入采集范围，无需人工干预。