小趙同學
0%

神策logagent收集Kubernetes-Pod日志

发表于 更新于

神策logagent收集Kubernetes Pod日志

项目背景：

神策logagent需要收集到java程序输出的日志，用于分析用户行为等

想到两种方案

  1. pod指定部署到某台宿主机上，日志目录挂载到宿主机，宿主机上启动logagent程序。
  2. sidecar的形式

目前使用第一种方案，比较稳妥，后续技术扎实后会替换成第二种，更稳定高效。

部署logagent

官方部署文档

安装Jdk
1
yum install -y java-1.8.0-openjdk.x86_64
下载LogAgent部署包和解压
1
2
wget http://download.sensorsdata.cn/release/logagent/logagent_20190605.tgz
tar zxvf logagent_20190605.tgz
编辑配置文件
1
2
3
4
5
6
7
8
9
vim logagent/logagent.conf

path=/var/log/veercms      #本地路径
logagent_id=veerservice_2_241    #id
pattern=cms.veerservice.log*     #log名称格式
project=production
pid_file=/data/vcg/logagent/logagent.pid
host=172.16.249.136    #server端地址
port=8106
启动
1
nohup bin/logagent >/dev/null 2>&1 &
查看日志
1
tail -f /data/vcg/logagent/log/logagent.log

配置节点标签

两种方式实现：

  1. 控制台上添加标签

    image-20190619142748161

  2. 命令行

    1
    2
    3
    4
    5
    #例如给 cn-beijing.i-2ze7nm52ylwxd8j4d0lq 这个node添加标签：
    kubectl label nodes cn-beijing.i-2ze7nm52ylwxd8j4d0lq test-key=test-value

    #查看标签：
    kubectl get node --show-labels

配置k8s yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
---
apiVersion: apps/v1beta2
kind: Deployment
metadata:
  name: com-veer-veerservice
  labels:
    app: com-veer-veerservice
  namespace: default
spec:
  replicas: 6
  selector:
    matchLabels:
      app: com-veer-veerservice
  template:
    metadata:
      labels:
        app: com-veer-veerservice
    spec:
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
      imagePullSecrets:
        - name: registry-vpc.cn-beijing.aliyuncs.com   ## 保密字典中的仓储认证
      affinity:
        nodeAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - preference: {}
              weight: 100
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: vcgapp
                    operator: In
                    values:
                      - veerlogagent     ## node节点亲和性
      containers:
        - name: com-veer-veerservice
          image: 'registry-vpc.cn-beijing.aliyuncs.com/vcg/com-veer-veerservice:pro-20190610183629'   ## 镜像地址
          env:
            - name: aliyun_logs_com-veer-veerservice     ## 日志服务
              value: stdout
          imagePullPolicy: Always
          volumeMounts:
            - mountPath: /var/log/veerservice_log   # 容器内路径
              name: volume-veerservice
          resources:
            limits:
              cpu: '1'
              memory: 4Gi
            requests:
              cpu: 500m
              memory: 1000Mi
          livenessProbe:        ## 存活检测
            initialDelaySeconds: 80
            periodSeconds: 10
            timeoutSeconds: 1
            successThreshold: 1
            failureThreshold: 3
            tcpSocket:
              port: 9101
          readinessProbe:         ## 就绪检测
            initialDelaySeconds: 80
            periodSeconds: 10
            timeoutSeconds: 1
            successThreshold: 1
            failureThreshold: 3
            tcpSocket:
              port: 9101
      volumes:
        - hostPath:
            path: /var/log/veerservice  # 主机路径
            type: ''
          name: volume-veerservice
---
apiVersion: v1
kind: Service
metadata:
  name: com-veer-veerservice
  labels:
    app: com-veer-veerservice
  namespace: default
spec:
  selector:
    app: com-veer-veerservice
  ports:
    - port: 80
      protocol: TCP
      targetPort: 9101
  type: ClusterIP