Docker自动化部署进阶：CI/CD流水线与监控告警体系实战

Docker自动化部署配合CI/CD流水线是现代DevOps实践的基础设施。本文从GitLab CI/CD流水线配置、Docker多阶段构建到Prometheus+Grafana监控告警体系，完整拆解生产环境的自动化运维链路。

GitLab CI/CD流水线配置与多阶段构建

CI/CD流水线的核心是定义清晰的构建-测试-部署阶段。以下是一个完整的.gitlab-ci.yml配置，覆盖代码检查、单元测试、镜像构建和滚动部署：

# .gitlab-ci.yml
stages:
  - lint
  - test
  - build
  - deploy

variables:
  IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA
  KUBECONFIG: /etc/kube/config

lint:
  stage: lint
  image: node:20-alpine
  script:
    - npm ci
    - npx eslint src/ --max-warnings 0
    - npx prettier --check src/
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"

test:
  stage: test
  image: node:20-alpine
  services:
    - postgres:16-alpine
  variables:
    POSTGRES_DB: testdb
    POSTGRES_USER: testuser
    POSTGRES_PASSWORD: testpass
  script:
    - npm ci
    - npm run test:unit -- --coverage
    - npm run test:integration
  coverage: '/Lines.*?(d+.%)/'
  artifacts:
    reports:
      coverage_report:
        coverage_format: cobertura
        path: coverage/cobertura-coverage.xml

build:
  stage: build
  image: docker:24
  services:
    - docker:24-dind
  script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - docker build -t $IMAGE_TAG -f Dockerfile.prod .
    - docker push $IMAGE_TAG
    - docker tag $IMAGE_TAG $CI_REGISTRY_IMAGE:latest
    - docker push $CI_REGISTRY_IMAGE:latest
  rules:
    - if: $CI_COMMIT_BRANCH == "main"

Dockerfile采用多阶段构建，最终镜像体积控制在50MB以内：

# Dockerfile.prod
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM node:20-alpine AS runtime
WORKDIR /app
RUN addgroup -g 1001 appgroup && \
    adduser -u 1001 -G appgroup -s /bin/sh -D appuser
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/package.json ./
USER appuser
EXPOSE 3000
HEALTHCHECK --interval=30s --timeout=3s --retries=3 \
  CMD wget --quiet --tries=1 --spider http://localhost:3000/health || exit 1
CMD ["node", "dist/server.js"]

Kubernetes容器编排中的滚动更新配置

部署阶段通过Kubernetes实现滚动更新和自动回滚。Deployment配置中需重点设置更新策略和健康检查：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
spec:
  replicas: 6
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 2
      maxUnavailable: 0
  selector:
    matchLabels:
      app: web-app
  template:
    spec:
      containers:
      - name: web-app
        image: registry.example.com/web-app:latest
        ports:
        - containerPort: 3000
        resources:
          requests:
            cpu: 200m
            memory: 256Mi
          limits:
            cpu: 500m
            memory: 512Mi
        livenessProbe:
          httpGet:
            path: /health
            port: 3000
          initialDelaySeconds: 10
          periodSeconds: 15
          failureThreshold: 3
        readinessProbe:
          httpGet:
            path: /ready
            port: 3000
          initialDelaySeconds: 5
          periodSeconds: 10
          failureThreshold: 2
        lifecycle:
          preStop:
            exec:
              command: ["sh", "-c", "sleep 10 && node dist/shutdown.js"]

maxUnavailable: 0确保更新过程中始终有足够副本可用。preStop钩子给应用10秒优雅关闭时间，避免请求中断。readinessProbe失败时Pod从Service Endpoints移除，新请求不再路由到该Pod。

混沌工程：故障注入与系统韧性验证

混沌工程通过主动注入故障验证系统韧性。使用Chaos Mesh进行网络延迟和Pod故障注入：

# 网络延迟注入(模拟200ms延迟)
apiVersion: chaos-mesh.org/v1alpha1
kind: NetworkChaos
metadata:
  name: web-app-delay
spec:
  action: delay
  mode: all
  selector:
    namespaces:
      - production
    labelSelectors:
      app: web-app
  delay:
    latency: "200ms"
    correlation: "100"
    jitter: "50ms"
  duration: "60s"
  scheduler:
    cron: "@every 1h"

---
# Pod随机杀死注入
apiVersion: chaos-mesh.org/v1alpha1
kind: PodChaos
metadata:
  name: web-app-kill
spec:
  action: pod-kill
  mode: one
  selector:
    namespaces:
      - production
    labelSelectors:
      app: web-app
  scheduler:
    cron: "@every 5m"

混沌实验前需定义稳态假设(Steady State Hypothesis)，例如”API P99延迟<500ms"和"错误率<0.1%"。实验期间通过监控大盘观察指标变化，任何稳态假设被打破都暴露了系统脆弱点。

故障应急响应流程与日志分析

SRE稳定性工程要求建立标准化的故障应急响应流程。生产事故发生时，按以下步骤执行：

# 1. 快速止血：回滚到上一稳定版本
kubectl rollout undo deployment/web-app -n production

# 2. 保留现场：导出故障Pod日志
kubectl logs web-app-xxx -n production --previous > /tmp/crash.log
kubectl describe pod web-app-xxx -n production > /tmp/pod-desc.txt

# 3. 检查事件
kubectl get events -n production --sort-by='.lastTimestamp' | tail -50

# 4. 集群资源检查
kubectl top nodes
kubectl top pods -n production --sort-by=cpu
kubectl get pods -n production -o wide | grep -v Running

日志收集采用EFK(Elasticsearch+Fluent Bit+Kibana)方案。Fluent Bit以DaemonSet方式运行在每个节点，采集容器stdout/stderr日志：

# Fluent Bit配置
[INPUT]
    Name tail
    Path /var/log/containers/*.log
    Parser docker
    Tag kube.*
    Refresh_Interval 5

[FILTER]
    Name kubernetes
    Match kube.*
    Kube_URL https://kubernetes.default.svc:443
    Merge_Log On

[OUTPUT]
    Name es
    Match *
    Host elasticsearch.logging
    Port 9200
    Index k8s-logs
    Type _doc

告警规则配置需区分Warning和Critical级别。Critical级别(如服务完全不可用、错误率>5%)触发PagerDuty电话告警，Warning级别(如CPU>80%、磁盘>85%)发送钉钉/飞书通知。每个告警必须包含runbook链接，指明排查步骤和责任人。