Harbor生产环境高可用搭建指南：DNS集群与SSL证书篇
整体架构概览
整个Harbor高可用方案包含以下层次：
- 数据层：PostgreSQL（一主一从一选举节点）+ Redis（一主两从）
- 存储层：MinIO 4节点分布式集群
- 接入层：2节点HAProxy配合Keepalived实现多VIP
- 应用层：双节点Harbor部署
- DNS层：内部主从DNS用于服务发现
第一阶段：内部DNS集群搭建
建议部署内部DNS（亦可用hosts替代）。在节点104和105上部署Bind9，配置主从同步，将域名（如 harbor.zz520.online）解析至VIP_Harbor，并将所有节点的DNS指向VIP_DNS。
部署DNS主服务器（节点104）
环境准备
关闭防火墙：
systemctl disable ufw --now
解决systemd-resolved占用53端口问题，创建脚本fix_port_53.sh并执行：
#!/bin/bash
# 释放53端口（Ubuntu systemd-resolved占用处理）
echo "检查53端口..."
ss -tulpn | grep :53
systemctl stop systemd-resolved
systemctl disable systemd-resolved
# 备份配置
cp /etc/systemd/resolved.conf /etc/systemd/resolved.conf.bak
# 写入新配置
cat > /etc/systemd/resolved.conf << 'RESOLVED'
[Resolve]
DNS=223.5.5.5 114.114.114.114
FallbackDNS=8.8.8.8
LLMNR=no
MulticastDNS=no
DNSSEC=no
DNSOverTLS=no
Cache=no
DNSStubListener=no
RESOLVED
# 重建resolv.conf
rm -f /etc/resolv.conf
cat > /etc/resolv.conf << 'RESOLVCONF'
nameserver 223.5.5.5
nameserver 114.114.114.114
RESOLVCONF
systemctl daemon-reload
sleep 1
ss -tulpn | grep :53
echo "53端口释放完毕"
主DNS部署脚本
创建deploy_bind_master.sh：
#!/bin/bash
apt update && apt install -y bind9 bind9-utils dnsutils
cd /etc/bind
cp named.conf.options named.conf.options.bak
cp named.conf.local named.conf.local.bak
# 主配置文件
cat > named.conf.options << 'OPTIONS'
options {
directory "/var/cache/bind";
allow-query { localhost; 10.0.0.0/24; };
recursion yes;
forwarders { 223.5.5.5; 114.114.114.114; };
dnssec-validation auto;
listen-on-v6 { any; };
};
OPTIONS
# 区域配置
cat > named.conf.local << 'ZONE'
zone "zz520.online" {
type master;
file "/etc/bind/db.zz520.online";
allow-transfer { 10.0.0.105; };
also-notify { 10.0.0.105; };
};
ZONE
# 区域数据库文件
cat > db.zz520.online << 'DB'
$TTL 604800
@ IN SOA ns1.zz520.online. admin.zz520.online. (
2
604800
86400
2419200
604800 )
@ IN NS ns1.zz520.online.
@ IN NS ns2.zz520.online.
ns1 IN A 10.0.0.104
ns2 IN A 10.0.0.105
harbor IN A 10.0.0.204
DB
chown root:bind /etc/bind/db.zz520.online
chmod 644 /etc/bind/db.zz520.online
named-checkconf
if [ $? -ne 0 ]; then
echo "配置检查失败"
exit 1
fi
systemctl daemon-reload
systemctl restart named
systemctl enable named
echo "主DNS部署完成"
部署DNS从服务器（节点105）
同样需要关闭防火墙并释放53端口。创建deploy_bind_slave.sh：
#!/bin/bash
apt update && apt install -y bind9 bind9-utils dnsutils
cd /etc/bind
cp named.conf.options named.conf.options.bak
cp named.conf.local named.conf.local.bak
cat > named.conf.options << 'OPTIONS'
options {
directory "/var/cache/bind";
allow-query { localhost; 10.0.0.0/24; };
recursion yes;
forwarders { 223.5.5.5; 114.114.114.114; };
dnssec-validation auto;
listen-on-v6 { any; };
};
OPTIONS
cat > named.conf.local << 'ZONE'
zone "zz520.online" {
type slave;
masters { 10.0.0.104; };
file "/var/cache/bind/db.zz520.online";
allow-notify { 10.0.0.104; };
};
ZONE
named-checkconf
if [ $? -ne 0 ]; then
echo "配置检查失败"
exit 1
fi
systemctl daemon-reload
systemctl restart named
systemctl enable named
echo "从DNS部署完成"
sleep 3
ls -la /var/cache/bind/ | grep db.zz520
DNS记录管理工具
在主节点创建manage_bind_records.sh，用于增删查DNS记录：
#!/bin/bash
ZONE_FILE="/etc/bind/db.zz520.online"
ZONE_NAME="zz520.online"
increment_serial() {
local serial=$(grep -Eo '^[[:space:]]*[0-9]+[[:space:]]*;[[:space:]]*Serial' "$ZONE_FILE" | awk '{print $1}')
local new_serial=$((serial + 1))
sed -i "s/^[[:space:]]*${serial}[[:space:]]*;[[:space:]]*Serial/ ${new_serial} ; Serial/" "$ZONE_FILE"
echo "序列号：$serial -> $new_serial"
}
add_record() {
read -p "主机名（不含域名）: " HOST
read -p "IP地址: " IP
if grep -q "^${HOST}[[:space:]]" "$ZONE_FILE"; then
read -p "记录已存在，继续? (y/n): " confirm
[[ "$confirm" != "y" ]] && return
fi
echo "${HOST} IN A ${IP}" >> "$ZONE_FILE"
echo "记录已添加"
increment_serial
named-checkzone "$ZONE_NAME" "$ZONE_FILE" && rndc reload
}
view_records() {
echo "--- DNS记录 ---"
printf "%-20s %s\n" "Host" "IP"
echo "--------------------------------"
grep 'IN[[:space:]]*A' "$ZONE_FILE" | awk '{printf "%-20s %s\n", $1, $NF}'
}
while true; do
echo "1. 查看记录"
echo "2. 添加A记录"
echo "3. 退出"
read -p "选择: " ch
case $ch in
1) view_records;;
2) add_record;;
3) exit;;
*) echo "无效";;
esac
done
第二阶段：Keepalived高可用配置
系统优化（节点104/105均执行）
创建optimize_ubuntu_for_harbor_ha.sh：
#!/bin/bash
set -e
TARGET_INTERFACE="ens33"
SYSCTL_FILE="/etc/sysctl.d/99-harbor-ha.conf"
# 内核参数
cat > "$SYSCTL_FILE" << 'EOF'
net.ipv4.ip_nonlocal_bind = 1
net.ipv4.ip_forward = 1
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.default.arp_ignore = 1
net.ipv4.conf.ens33.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.conf.default.arp_announce = 2
net.ipv4.conf.ens33.arp_announce = 2
net.core.somaxconn = 65535
net.core.netdev_max_backlog = 65535
net.ipv4.tcp_max_syn_backlog = 65535
net.ipv4.ip_local_port_range = 1024 65535
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 30
EOF
# 资源限制
cat > /etc/security/limits.d/99-harbor-ha.conf << 'EOF'
root soft nofile 1048576
root hard nofile 1048576
root soft nproc 1048576
root hard nproc 1048576
* soft nofile 1048576
* hard nofile 1048576
* soft nproc 1048576
* hard nproc 1048576
EOF
sysctl --system
echo "优化完成，建议重启"
read -p "立即重启? (y/n): " reboot_confirm
[[ "$reboot_confirm" == "y" ]] && reboot
安装Keepalived
apt update && apt install -y keepalived
部署健康检查脚本
创建deploy_health_checks.sh在/etc/keepalived/scripts/目录下生成各服务检查脚本：
#!/bin/bash
set -e
SCRIPT_DIR="/etc/keepalived/scripts"
mkdir -p "$SCRIPT_DIR"
cd "$SCRIPT_DIR"
# HAProxy检查
cat > chk_haproxy.sh << 'EOF'
#!/bin/bash
pgrep haproxy &>/dev/null
exit $?
EOF
# 端口检查
cat > chk_port.sh << 'EOF'
#!/bin/bash
ss -tulpn | grep -q ":${1}" &>/dev/null
exit $?
EOF
# DNS检查
cat > chk_dns.sh << 'EOF'
#!/bin/bash
pgrep named &>/dev/null
exit $?
EOF
# MinIO检查
cat > chk_minio.sh << 'EOF'
#!/bin/bash
LOCAL_IP="10.0.0.104" # 手动修改为本机IP
ss -tulpn | grep -q ":9000" &>/dev/null
exit $?
EOF
# PostgreSQL角色检查
cat > chk_pgsql_repmgr.sh << 'EOF'
#!/bin/bash
PG_USER="postgres"
REPMGR_BIN="/usr/local/bin/repmgr"
REPMGR_CONF="/apps/repmgr/etc/repmgr.conf"
ROLE=$(su - "$PG_USER" -c "$REPMGR_BIN -f $REPMGR_CONF node status 2>/dev/null" | grep 'Role' | awk '{print $NF}')
if [[ -z "$ROLE" ]]; then
SQL="SELECT pg_is_in_recovery();"
IN_REC=$(su - "$PG_USER" -c "psql -t -c \"$SQL\" 2>/dev/null" | xargs)
[[ "$IN_REC" == "f" ]] && ROLE="primary" || ROLE="standby"
fi
[[ "$ROLE" == "primary" ]] && exit 0 || exit 1
EOF
# Redis检查
cat > chk_redis.sh << 'EOF'
#!/bin/bash
pgrep redis-server &>/dev/null
exit $?
EOF
chmod +x *.sh
chown -R root:root "$SCRIPT_DIR"
echo "健康检查脚本部署完毕，请修改chk_minio.sh中的LOCAL_IP"
生成Keepalived配置
创建交互式脚本setup_keepalived_conf.sh：
#!/bin/bash
set -e
CONF_FILE="/etc/keepalived/keepalived.conf"
IFACE="ens33"
NET_PREFIX="10.0.0"
echo "选择节点类型："
select NODE in "ha1 (104 Master)" "ha2 (105 Backup)"; do
case $REPLY in
1) ROUTER_ID="LVS_HA1"; STATE="MASTER"; PRIO=150; break;;
2) ROUTER_ID="LVS_HA2"; STATE="BACKUP"; PRIO=100; break;;
*) echo "无效选择";;
esac
done
[[ -f "$CONF_FILE" ]] && cp "$CONF_FILE" "${CONF_FILE}.bak.$(date +%s)"
cat > "$CONF_FILE" << EOF
global_defs {
router_id $ROUTER_ID
script_user root
enable_script_security
}
vrrp_script chk_haproxy {
script "/etc/keepalived/scripts/chk_haproxy.sh"
interval 2; weight -20; fall 2; rise 2
}
vrrp_script chk_dns {
script "/etc/keepalived/scripts/chk_dns.sh"
interval 2; weight -20
}
vrrp_script chk_minio {
script "/etc/keepalived/scripts/chk_minio.sh"
interval 3; weight -20
}
vrrp_script chk_harbor {
script "/etc/keepalived/scripts/chk_port.sh 443"
interval 3; weight -20
}
vrrp_script chk_pgsql {
script "/etc/keepalived/scripts/chk_pgsql_repmgr.sh"
interval 2; weight -30; fall 1; rise 2
}
vrrp_script chk_redis {
script "/etc/keepalived/scripts/chk_redis.sh"
interval 2; weight -20
}
vrrp_instance VI_DNS {
state $STATE; interface $IFACE; virtual_router_id 51; priority $PRIO; advert_int 1
authentication { auth_type PASS; auth_pass dnsvip1; }
virtual_ipaddress { ${NET_PREFIX}.205/32 dev $IFACE label ${IFACE}:dns; }
track_script { chk_dns; chk_haproxy; }
}
vrrp_instance VI_MINIO {
state $STATE; interface $IFACE; virtual_router_id 54; priority $PRIO; advert_int 1
authentication { auth_type PASS; auth_pass minivip1; }
virtual_ipaddress { ${NET_PREFIX}.203/32 dev $IFACE label ${IFACE}:minio; }
track_script { chk_minio; }
}
vrrp_instance VI_HARBOR {
state $STATE; interface $IFACE; virtual_router_id 55; priority $PRIO; advert_int 1
authentication { auth_type PASS; auth_pass harborvip1; }
virtual_ipaddress { ${NET_PREFIX}.204/32 dev $IFACE label ${IFACE}:harbor; }
track_script { chk_harbor; chk_haproxy; }
}
vrrp_instance VI_PG {
state BACKUP; interface $IFACE; virtual_router_id 52; priority 100; advert_int 1; nopreempt
authentication { auth_type PASS; auth_pass pgvip1; }
virtual_ipaddress { ${NET_PREFIX}.201/32 dev $IFACE label ${IFACE}:pg; }
track_script { chk_pgsql; }
}
vrrp_instance VI_REDIS {
state BACKUP; interface $IFACE; virtual_router_id 53; priority 100; advert_int 1; nopreempt
authentication { auth_type PASS; auth_pass redisvip1; }
virtual_ipaddress { ${NET_PREFIX}.202/32 dev $IFACE label ${IFACE}:redis; }
track_script { chk_redis; }
}
EOF
echo "配置生成完毕：$CONF_FILE"
启动与测试
systemctl enable keepalived --now
在主节点查看VIP：ip addr show ens33，应看到5个VIP。模拟故障：在主节点停止keepalived，VIP应漂移至从节点。恢复主节点后，无状态VIP自动迁回，有状态VIP（PG、Redis）保持不动。
第三阶段：SSL证书申请
配置DNSPod API密钥
在DNSPod后台获取API Token后，执行安全初始化脚本setup-secrets.sh：
#!/bin/bash
set -euo pipefail
SECRET_DIR="/etc/acme-secrets"
mkdir -p "$SECRET_DIR"
chmod 700 "$SECRET_DIR"
read -p "DNSPod ID: " dp_id
read -s -p "DNSPod Key: " dp_key
echo
echo "$dp_id" > "$SECRET_DIR/dp_id"
echo "$dp_key" > "$SECRET_DIR/dp_key"
chmod 600 "$SECRET_DIR/dp_id" "$SECRET_DIR/dp_key"
chown root:root "$SECRET_DIR/dp_id" "$SECRET_DIR/dp_key"
echo "密钥已保存"
申请通配符证书
创建acme_tencent_issue.sh：
#!/bin/bash
set -euo pipefail
MAIN_DOMAIN="zz520.online"
ACCOUNT_EMAIL="admin@${MAIN_DOMAIN}"
SECRET_DIR="/etc/acme-secrets"
TARGET_DIR="/etc/ssl/acme/${MAIN_DOMAIN}"
# 读取密钥
DP_Id=$(cat "$SECRET_DIR/dp_id" | tr -d '[:space:]')
DP_Key=$(cat "$SECRET_DIR/dp_key" | tr -d '[:space:]')
export DP_Id DP_Key
# 安装acme.sh
ACME_BIN="$HOME/.acme.sh/acme.sh"
[[ ! -f "$ACME_BIN" ]] && curl https://get.acme.sh | sh
# 设置CA和申请证书
"$ACME_BIN" --set-default-ca --server letsencrypt
"$ACME_BIN" --issue --dns dns_dp -d "$MAIN_DOMAIN" -d "*.${MAIN_DOMAIN}" --keylength ec-256
# 部署
mkdir -p "$TARGET_DIR"
"$ACME_BIN" --install-cert -d "$MAIN_DOMAIN" -d "*.${MAIN_DOMAIN}" --ecc \
--key-file "$TARGET_DIR/privkey.pem" \
--fullchain-file "$TARGET_DIR/fullchain.pem"
chmod 700 "$TARGET_DIR"
chmod 600 "$TARGET_DIR/privkey.pem"
chmod 644 "$TARGET_DIR/fullchain.pem"
chown -R root:root "$TARGET_DIR"
echo "证书路径："
echo " Full Chain: $TARGET_DIR/fullchain.pem"
echo " Private Key: $TARGET_DIR/privkey.pem"
附：Vim粘贴缩进修复：在~/.vimrc中添加：
augroup pastetoggle
autocmd!
au InsertEnter * set paste
au InsertLeave * set nopaste
augroup END
