Graylog分布式日志管理集群搭建指南
Graylog是一种开源的日志聚合、分析与展示工具，具有低成本和高性能的特点。
Graylog相较于ELK：
Graylog需要将日志源收集到graylog-server进行处理，并使用Elasticsearch存储数据。同时，Graylog自带Web界面，功能类似于Kibana。
集群架构参考：
官方文档链接：Graylog官方文档
部署节点规划：
| IP | 作用 |
|---|---|
| 192.168.122.71 | Graylog、MongoDB、Nginx |
| 192.168.122.72 | Graylog、MongoDB |
| 192.168.122.73 | Graylog、MongoDB |
| 192.168.122.74 | Elasticsearch |
| 192.168.122.75 | Elasticsearch |
| 192.168.122.76 | Elasticsearch |
前置准备：
sudo systemctl status firewalld
# 确保防火墙已关闭：Active: inactive (dead)
cat /etc/selinux/config
# 确保SELinux已关闭：SELINUX=disabled
java -version
# 确保JDK版本为1.8或更高
MongoDB集群配置：
MongoDB用于存储Graylog的配置数据（如用户信息和流配置），而非日志数据本身。
# 安装MongoDB
sudo yum install -y mongodb-org
# 创建目录
mkdir -p /work/mongo/{data,log}
touch /work/mongo/log/mongod.log
# 配置mongod.conf
systemLog:
destination: file
logAppend: true
path: /work/mongo/log/mongod.log
storage:
dbPath: /work/mongo/data
journal:
enabled: true
net:
port: 27017
bindIp: 0.0.0.0
replication:
replSetName: rs0
初始化MongoDB集群：
mongo
rs.initiate({
_id: "rs0",
members: [{ _id: 0, host: "192.168.122.71:27017" }]
})
rs.add("192.168.122.72")
rs.add("192.168.122.73")
rs.status()
use graylog
db.createUser({
user: "graylog",
pwd: "password",
roles: ["readWrite", "dbAdmin"]
})
Elasticsearch集群部署：
Graylog 3.0要求Elasticsearch版本不低于5.6.13且不支持7.x。
# 下载并解压
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.4.3.tar.gz
tar -zxf elasticsearch-6.4.3.tar.gz
# 编辑配置文件
cluster.name: graylog
node.name: graylogXX
node.master: true
node.data: true
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
path.data: /work/elasticsearch-6.4.3/data
path.logs: /work/elasticsearch-6.4.3/logs
network.host: 0.0.0.0
http.port: 9200
discovery.zen.ping.unicast.hosts: ["192.168.122.74:9300", "192.168.122.75:9300", "192.168.122.76:9300"]
# 启动服务
bin/elasticsearch -d
Graylog集群安装：
# 安装Graylog
sudo rpm -Uvh https://packages.graylog2.org/repo/packages/graylog-3.0-repository_latest.rpm
# 生成密码密钥
pwgen -N 1 -s 96
echo -n admin | sha256sum
# 编辑配置文件
is_master = true
password_secret = YOUR_GENERATED_SECRET
root_password_sha2 = YOUR_SHA256_PASSWORD_HASH
root_timezone = Asia/Shanghai
http_bind_address = 0.0.0.0:9000
elasticsearch_hosts = http://192.168.122.74:9200,http://192.168.122.75:9200,http://192.168.122.76:9200
mongodb_uri = mongodb://graylog:password@192.168.122.71:27017,192.168.122.72:27017,192.168.122.73:27017/graylog
# 启动服务
sudo systemctl enable graylog-server.service
sudo systemctl start graylog-server.service
Nginx负载均衡配置：
upstream graylog_servers {
least_conn;
server 192.168.122.71:9000 max_fails=3 fail_timeout=30s;
server 192.168.122.72:9000 max_fails=3 fail_timeout=30s;
server 192.168.122.73:9000 max_fails=3 fail_timeout=30s;
}
server {
listen 9100;
server_name 192.168.122.71;
location / {
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Graylog-Server-URL http://$server_name/;
proxy_pass http://graylog_servers;
}
}
通过Logstash转发日志到Graylog：
Graylog使用GELF协议接收日志消息，需在Logstash中配置输出插件。
output {
gelf {
host => "graylog_ip_address"
port => 12201
}
stdout { codec => rubydebug }
}
