Linux高可用集群搭建指南：Keepalived+Nginx双机热备配置与故障切换

服务器高可用集群是保障业务连续性的基础设施。Keepalived基于VRRP协议实现虚拟IP漂移，配合Nginx负载均衡可实现双机热备架构，主节点故障时备节点在秒级内接管虚拟IP，业务感知中断时间控制在3秒以内。

高可用集群架构设计

采用Active-Standby模式部署两台服务器，通过VRRP协议维护一个虚拟IP（VIP）。客户端请求访问VIP，由主节点的Nginx处理后转发到后端应用服务器。主节点宕机时，备节点检测到VRRP心跳超时，自动将VIP漂移到备节点，实现故障自动切换。

# 网络拓扑
# Master: 192.168.1.10 (eth0)
# Backup: 192.168.1.11 (eth0)
# VIP:    192.168.1.100

两台服务器需在同一二层网络内，VRRP协议通过组播地址224.0.0.18交换心跳包。如果跨网段部署，需配置unicast_peer使用单播模式。

Keepalived安装与VRRP配置

在Ubuntu/Debian系统上安装Keepalived：

apt update && apt install -y keepalived nginx

# 启用IP转发（Nginx作为反向代理需要）
echo "net.ipv4.ip_forward = 1" >> /etc/sysctl.conf
sysctl -p

Master节点Keepalived配置文件/etc/keepalived/keepalived.conf：

global_defs {
    router_id NGINX_MASTER
    vrrp_skip_check_adv_addr
    vrrp_strict
    vrrp_garp_interval 0
    vrrp_gna_interval 0
}

vrrp_script check_nginx {
    script "/etc/keepalived/check_nginx.sh"
    interval 2
    weight -20
    fall 2
    rise 1
}

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1

    authentication {
        auth_type PASS
        auth_pass MyStr0ngP@ss
    }

    virtual_ipaddress {
        192.168.1.100/24
    }

    track_script {
        check_nginx
    }

    notify_master "/etc/keepalived/notify.sh master"
    notify_backup "/etc/keepalived/notify.sh backup"
    notify_fault "/etc/keepalived/notify.sh fault"
}

Backup节点配置基本相同，将state改为BACKUP，priority设为90。weight -20表示脚本检查失败时优先级降低20，确保Master故障后Backup的优先级高于Master。

Nginx健康检查脚本与负载均衡配置

Keepalived本身不检测Nginx状态，需要通过vrrp_script调用外部脚本。创建/etc/keepalived/check_nginx.sh：

#!/bin/bash
# 检查Nginx进程是否存活
if ! pidof nginx > /dev/null 2>&1; then
    # 尝试重启Nginx
    systemctl restart nginx
    sleep 2
    if ! pidof nginx > /dev/null 2>&1; then
        echo "Nginx is down" >&2
        exit 1
    fi
fi
exit 0

Nginx负载均衡配置/etc/nginx/conf.d/upstream.conf：

upstream backend {
    # 后端应用服务器
    server 192.168.1.20:8080 weight=3 max_fails=3 fail_timeout=30s;
    server 192.168.1.21:8080 weight=2 max_fails=3 fail_timeout=30s;
    server 192.168.1.22:8080 weight=1 max_fails=3 fail_timeout=30s backup;

    keepalive 32;
    keepalive_timeout 60s;
}

server {
    listen 80;
    server_name 192.168.1.100;

    location / {
        proxy_pass http://backend;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_connect_timeout 5s;
        proxy_read_timeout 60s;
        proxy_next_upstream error timeout http_502 http_503 http_504;
    }

    location /health {
        access_log off;
        return 200 "ok\n";
    }
}

proxy_next_upstream配置在某个后端节点返回502/503/504时自动切换到下一个节点，配合max_fails和fail_timeout实现自动剔除故障节点。

故障切换测试与验证

部署完成后需要进行故障切换测试，验证切换时间和数据一致性：

# 1. 正常状态确认VIP在Master上
ip addr show eth0 | grep 192.168.1.100

# 2. 模拟Master Nginx宕机
ssh root@192.168.1.10 "systemctl stop nginx"

# 3. 观察Backup节点日志，确认VIP漂移
journalctl -u keepalived -f
# 预期输出：Entering MASTER STATE

# 4. 在Backup节点确认VIP已接管
ip addr show eth0 | grep 192.168.1.100

# 5. 持续curl VIP验证服务连续性
while true; do curl -s -o /dev/null -w "%{http_code} %{time_total}s\n" http://192.168.1.100/; sleep 0.5; done

正常情况下切换过程在1-3秒内完成。curl持续请求会看到少量502或超时后恢复200，中断时间取决于VRRP advert_int间隔和脚本检测间隔。

脑裂问题诊断与防护

脑裂（Split-Brain）是指主备节点同时持有VIP的状态，通常由网络分区引起。防护措施：

# 1. 配置多条VRRP心跳链路
vrrp_instance VI_1 {
    interface eth0
    lvs_sync_daemon_interface eth1  # 使用独立网卡做心跳
}

# 2. 通过fencing强制关闭异常节点
# notify_fault脚本中调用IPMI强制关机
#!/bin/bash
# /etc/keepalived/notify.sh
case "$1" in
    fault)
        # 通过IPMI强制重启对端
        ipmitool -H 192.168.1.10-bmc -U admin -P pass chassis power cycle
        ;;
esac

# 3. 使用arping检测VIP冲突
arping -c 3 -I eth0 192.168.1.100
# 如果有多个MAC地址响应，说明存在脑裂

生产环境建议配置监控告警，当检测到两个节点同时处于MASTER状态时立即告警。可通过Zabbix或Prometheus监控keepalived进程状态和VIP归属。