Kubernetes HPA自定义指标自动扩缩容：Prometheus Adapter实战配置

Kubernetes默认HPA的局限性

Kubernetes原生的Horizontal Pod Autoscaler默认只支持CPU和内存指标做扩缩容。对于Web应用来说，CPU使用率40%不代表用户请求没在排队——请求延迟、队列深度、QPS这些业务指标才是更准确的伸缩信号。Prometheus Adapter把Prometheus采集的自定义指标暴露给Kubernetes Metrics API，HPA就能基于真实业务负载做决策。

组件架构与数据流

完整链路：应用暴露Metrics → Prometheus采集 → Prometheus Adapter转换 → Kubernetes Metrics API → HPA Controller决策。

前提条件：已部署Kubernetes集群（v1.28+）、Prometheus Operator（kube-prometheus-stack）、应用已接入Prometheus监控。

部署Prometheus Adapter

Helm安装：

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

helm install prometheus-adapter prometheus-community/prometheus-adapter   --namespace monitoring   --create-namespace   --set prometheus.url=http://prometheus-operated.monitoring.svc:9090   --set prometheus.port=9090   --set metricsRelistInterval=30s   --set rules.default=false

验证Metrics API注册成功：

kubectl get --raw "/apis/metrics.k8s.io/v1beta1" | jq .
kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1" | jq .

自定义指标发现规则配置

Prometheus Adapter通过ConfigMap定义指标发现规则。这是最关键的配置环节——规则写错，HPA拿不到数据。

apiVersion: v1
kind: ConfigMap
metadata:
  name: prometheus-adapter-custom-rules
  namespace: monitoring
data:
  custom-rules.yaml: |
    rules:
    # HTTP请求QPS指标
    - seriesQuery: 'http_requests_total{namespace!="",pod!=""}'
      resources:
        overrides:
          namespace: {resource: "namespace"}
          pod: {resource: "pod"}
      name:
        matches: "http_requests_total"
        as: "http_requests_per_second"
      metricsQuery: 'sum(rate(<<.Series>>{<<.LabelMatchers>>}[2m])) by (<<.GroupBy>>)'

    # 请求延迟P99指标
    - seriesQuery: 'http_request_duration_seconds_bucket{namespace!="",pod!=""}'
      resources:
        overrides:
          namespace: {resource: "namespace"}
          pod: {resource: "pod"}
      name:
        matches: "http_request_duration_seconds_bucket"
        as: "http_request_duration_p99_seconds"
      metricsQuery: 'histogram_quantile(0.99, sum(rate(<<.Series>>{<<.LabelMatchers>>}[2m])) by (<<.GroupBy>>, le))'

    # 消息队列深度
    - seriesQuery: 'rabbitmq_queue_messages{namespace!="",queue!=""}'
      resources:
        overrides:
          namespace: {resource: "namespace"}
      name:
        matches: "rabbitmq_queue_messages"
        as: "rabbitmq_queue_messages_count"
      metricsQuery: 'avg(<<.Series>>{<<.LabelMatchers>>}) by (<<.GroupBy>>)'

更新Adapter引用此ConfigMap：

kubectl create configmap prometheus-adapter-custom-rules   --from-file=custom-rules.yaml=custom-rules.yaml   -n monitoring

# 重启Adapter加载新规则
kubectl rollout restart deployment prometheus-adapter -n monitoring

HPA策略配置与冷启动防护

基于QPS和延迟的HPA配置：

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-api-hpa
  namespace: production
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-api
  minReplicas: 3
  maxReplicas: 50
  metrics:
  - type: Pods
    pods:
      metric:
        name: http_requests_per_second
      target:
        type: AverageValue
        averageValue: "1000"
  - type: Pods
    pods:
      metric:
        name: http_request_duration_p99_seconds
      target:
        type: AverageValue
        averageValue: "0.5"
  behavior:
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
      - type: Percent
        value: 25
        periodSeconds: 60
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
      - type: Percent
        value: 100
        periodSeconds: 30
      - type: Pods
        value: 4
        periodSeconds: 30
      selectPolicy: Max

behavior字段是冷启动防护的关键：stabilizationWindowSeconds: 300避免缩容抖动，selectPolicy: Max确保扩容取最激进策略。

常见排错与验证

验证自定义指标是否被API识别：

kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1/namespaces/production/pods/*/http_requests_per_second" | jq .

HPA事件排查：

kubectl describe hpa web-api-hpa -n production

如果出现the HPA was unable to compute the replica count，检查PromQL是否能直接在Prometheus中返回数据，以及Adapter规则中的LabelMatchers是否与Pod标签匹配。

调整adapter日志级别进行调试：

kubectl edit deployment prometheus-adapter -n monitoring
# 在container args中添加 --v=6

自定义指标驱动的HPA让扩缩容决策回归业务本质，不再依赖间接的CPU指标做猜测。正确配置Adapter规则后，系统对流量波峰的响应速度能从分钟级提升到秒级。