Nginx Ingress Controller在K8s中的流量治理与灰度发布实战

Kubernetes流量入口的治理挑战

Kubernetes集群对外暴露服务时，Ingress Controller是流量入口的核心组件。Nginx Ingress Controller基于Nginx反向代理，将外部HTTP/HTTPS流量路由到集群内Service，是K8s网站运维中使用最广泛的Ingress实现。流量治理不仅涉及基础路由转发，还涵盖灰度发布、限流熔断、TLS终止、跨域配置等生产级需求，直接决定业务上线质量和系统稳定性。

Nginx Ingress Controller部署与基础路由配置

# Helm部署
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm install ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress-nginx --create-namespace \
  --set controller.replicaCount=2 \
  --set controller.service.type=LoadBalancer

# 基础Ingress路由
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-app
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - app.example.com
    secretName: app-tls-secret
  rules:
  - host: app.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-app-svc
            port:
              number: 80

基于Canary的灰度发布流量切分

Nginx Ingress Controller通过Canary注解实现灰度发布，支持按权重、按Cookie、按请求头三种切分策略：

# 主版本Ingress（v1）
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-app-stable
spec:
  ingressClassName: nginx
  rules:
  - host: app.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-app-v1
            port:
              number: 80

# 金丝雀版本Ingress（v2），按权重切分20%流量
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-app-canary
  annotations:
    nginx.ingress.kubernetes.io/canary: "true"
    nginx.ingress.kubernetes.io/canary-weight: "20"
spec:
  ingressClassName: nginx
  rules:
  - host: app.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-app-v2
            port:
              number: 80

按Cookie灰度——仅特定用户进入v2：

annotations:
  nginx.ingress.kubernetes.io/canary: "true"
  nginx.ingress.kubernetes.io/canary-by-cookie: "canary"
  nginx.ingress.kubernetes.io/canary-by-cookie-value: "true"

按请求头灰度——移动端流量走v2：

annotations:
  nginx.ingress.kubernetes.io/canary: "true"
  nginx.ingress.kubernetes.io/canary-by-header: "X-Canary"
  nginx.ingress.kubernetes.io/canary-by-header-value: "mobile"

限流与熔断配置实战

annotations:
  nginx.ingress.kubernetes.io/limit-connections: "50"
  nginx.ingress.kubernetes.io/limit-rps: "100"
  nginx.ingress.kubernetes.io/limit-rpm: "3000"
  nginx.ingress.kubernetes.io/limit-status-code: "429"

被动健康检查配置：

annotations:
  nginx.ingress.kubernetes.io/proxy-connect-timeout: "5"
  nginx.ingress.kubernetes.io/proxy-read-timeout: "60"
  nginx.ingress.kubernetes.io/proxy-send-timeout: "60"
  nginx.ingress.kubernetes.io/proxy-next-upstream: "error timeout http_502 http_503"
  nginx.ingress.kubernetes.io/proxy-next-upstream-timeout: "10"
  nginx.ingress.kubernetes.io/proxy-next-upstream-tries: "3"

TLS证书自动化管理与跨域配置

使用cert-manager自动签发和续期证书：

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: admin@example.com
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
    - http01:
        ingress:
          class: nginx

CORS跨域配置：

annotations:
  nginx.ingress.kubernetes.io/enable-cors: "true"
  nginx.ingress.kubernetes.io/cors-allow-origin: "https://frontend.example.com"
  nginx.ingress.kubernetes.io/cors-allow-methods: "GET, POST, PUT, DELETE, OPTIONS"
  nginx.ingress.kubernetes.io/cors-allow-headers: "Authorization,Content-Type"
  nginx.ingress.kubernetes.io/cors-max-age: "86400"

自定义Nginx配置与性能调优

apiVersion: v1
kind: ConfigMap
metadata:
  name: ingress-nginx-controller
  namespace: ingress-nginx
data:
  max-worker-connections: "65535"
  keep-alive: "75"
  keep-alive-requests: "10000"
  proxy-body-size: "50m"
  enable-brotli: "true"
  brotli-level: "4"
  upstream-keepalive-connections: "320"
  upstream-keepalive-timeout: "60"

upstream keepalive连接池对高并发场景至关重要。启用后Nginx与后端Pod之间维持长连接池，QPS提升可达30%-50%。

生产环境故障排查与监控

Nginx Ingress Controller暴露Prometheus指标端口（10254/metrics），关键指标包括请求总量、内存占用、请求延迟分布。配置告警规则：5xx错误率超过5%触发P1告警，P99延迟超过2秒触发P2告警。高QPS场景建议关闭access log以降低IO压力，改用采样日志或仅记录异常请求。