引言
部署是软件交付的最后一公里,也是最关键的环节。不当的部署策略可能导致服务中断、用户流失。本文将系统介绍各种零停机部署策略,并提供生产级实现方案。
部署策略对比
| 策略 | 停机时间 | 回滚速度 | 资源消耗 | 复杂度 | 适用场景 |
|---|---|---|---|---|---|
| 重建部署 | 有 | 慢 | 低 | 低 | 开发环境 |
| 滚动更新 | 无 | 中 | 中 | 低 | 大多数场景 |
| 蓝绿部署 | 无 | 快 | 高 | 中 | 关键业务 |
| 金丝雀发布 | 无 | 快 | 中 | 高 | 高风险变更 |
| A/B测试 | 无 | 快 | 高 | 高 | 功能验证 |
滚动更新(Rolling Update)
Kubernetes原生滚动更新
apiVersion: apps/v1
kind: Deployment
metadata:
name: web-app
spec:
replicas: 10
strategy:
type: RollingUpdate
rollingUpdate:
# 最大不可用Pod数(可以是数字或百分比)
maxUnavailable: 1 # 或 10%
# 最大超出Pod数
maxSurge: 2 # 或 20%
selector:
matchLabels:
app: web-app
template:
metadata:
labels:
app: web-app
spec:
containers:
- name: web-app
image: myapp:v1.0.0
ports:
- containerPort: 8080
# 就绪探针:确保只有健康的Pod接收流量
readinessProbe:
httpGet:
path: /health
port: 8080
initialDelaySeconds: 5
periodSeconds: 5
failureThreshold: 3
# 存活探针:检测死锁或hang
livenessProbe:
httpGet:
path: /health
port: 8080
initialDelaySeconds: 15
periodSeconds: 10
# 启动探针:慢启动应用
startupProbe:
httpGet:
path: /health
port: 8080
failureThreshold: 30
periodSeconds: 2
# 执行滚动更新
kubectl set image deployment/web-app web-app=myapp:v2.0.0
# 查看更新状态
kubectl rollout status deployment/web-app
# 回滚到上一版本
kubectl rollout undo deployment/web-app
# 查看历史版本
kubectl rollout history deployment/web-app
# 回滚到特定版本
kubectl rollout undo deployment/web-app --to-revision=3
优雅终止配置
spec:
terminationGracePeriodSeconds: 60 # 优雅终止时间
containers:
- name: web-app
lifecycle:
preStop:
exec:
command:
- /bin/sh
- -c
- |
# 等待负载均衡器更新
sleep 15
# 执行清理操作
/app/cleanup.sh
蓝绿部署(Blue-Green)
基本原理
初始状态:
Blue (v1) ← 生产流量
Green (v2) ← 待机
部署过程:
1. 部署Green版本(v2)
2. 测试Green版本
3. 切换流量到Green
4. Blue变为待机(可回滚)
回滚:
立即切换回Blue
Kubernetes实现
# blue-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: web-app-blue
spec:
replicas: 5
selector:
matchLabels:
app: web-app
version: blue
template:
metadata:
labels:
app: web-app
version: blue
spec:
containers:
- name: web-app
image: myapp:v1.0.0
---
# green-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: web-app-green
spec:
replicas: 5
selector:
matchLabels:
app: web-app
version: green
template:
metadata:
labels:
app: web-app
version: green
spec:
containers:
- name: web-app
image: myapp:v2.0.0
---
# service.yaml
apiVersion: v1
kind: Service
metadata:
name: web-app
spec:
selector:
app: web-app
version: blue # 指向当前活跃版本
ports:
- port: 80
targetPort: 8080
#!/bin/bash
# deploy-blue-green.sh
# 1. 部署Green版本
kubectl apply -f green-deployment.yaml
# 2. 等待Green就绪
kubectl rollout status deployment/web-app-green
# 3. 测试Green版本
GREEN_IP=$(kubectl get pods -l version=green -o jsonpath='{.items[0].status.podIP}')
curl -f http://$GREEN_IP:8080/health
if [ $? -ne 0 ]; then
echo "Green版本测试失败,中止部署"
exit 1
fi
# 4. 切换流量到Green
kubectl patch service web-app -p '{"spec":{"selector":{"version":"green"}}}'
# 5. 等待一段时间观察
sleep 60
# 6. 如果一切正常,删除Blue版本
kubectl delete deployment web-app-blue
自动化蓝绿部署脚本
#!/usr/bin/env python3
import subprocess
import time
import sys
class BlueGreenDeployer:
def __init__(self, app_name: str, namespace: str = "default"):
self.app_name = app_name
self.namespace = namespace
def deploy(self, new_image: str, target_color: str):
"""执行蓝绿部署"""
current_color = self.get_current_color()
print(f"当前活跃版本: {current_color}")
print(f"部署新版本到: {target_color}")
# 1. 部署新版本
self.apply_deployment(target_color, new_image)
# 2. 等待就绪
if not self.wait_for_ready(target_color):
print("部署失败,新版本未就绪")
return False
# 3. 健康检查
if not self.health_check(target_color):
print("健康检查失败,中止部署")
self.rollback_deployment(target_color)
return False
# 4. 切换流量
self.switch_traffic(target_color)
print(f"流量已切换到 {target_color}")
# 5. 观察期
print("进入观察期(60秒)...")
time.sleep(60)
# 6. 验证
if not self.verify_deployment():
print("部署后验证失败,回滚")
self.switch_traffic(current_color)
return False
# 7. 清理旧版本
self.cleanup_deployment(current_color)
print("部署成功!")
return True
def get_current_color(self) -> str:
"""获取当前活跃的颜色"""
result = subprocess.run(
["kubectl", "get", "service", self.app_name,
"-o", "jsonpath={.spec.selector.version}",
"-n", self.namespace],
capture_output=True, text=True
)
return result.stdout.strip()
def apply_deployment(self, color: str, image: str):
"""应用部署配置"""
manifest = f"""
apiVersion: apps/v1
kind: Deployment
metadata:
name: {self.app_name}-{color}
spec:
replicas: 5
selector:
matchLabels:
app: {self.app_name}
version: {color}
template:
metadata:
labels:
app: {self.app_name}
version: {color}
spec:
containers:
- name: {self.app_name}
image: {image}
"""
subprocess.run(
["kubectl", "apply", "-f", "-", "-n", self.namespace],
input=manifest, text=True
)
def wait_for_ready(self, color: str, timeout: int = 300) -> bool:
"""等待部署就绪"""
result = subprocess.run(
["kubectl", "rollout", "status",
f"deployment/{self.app_name}-{color}",
"-n", self.namespace,
f"--timeout={timeout}s"],
capture_output=True
)
return result.returncode == 0
def health_check(self, color: str) -> bool:
"""执行健康检查"""
result = subprocess.run(
["kubectl", "get", "pods",
"-l", f"version={color}",
"-o", "jsonpath={.items[0].status.podIP}",
"-n", self.namespace],
capture_output=True, text=True
)
pod_ip = result.stdout.strip()
# 测试健康端点
import requests
try:
response = requests.get(f"http://{pod_ip}:8080/health", timeout=5)
return response.status_code == 200
except:
return False
def switch_traffic(self, color: str):
"""切换流量"""
subprocess.run(
["kubectl", "patch", "service", self.app_name,
"-p", f'{{"spec":{{"selector":{{"version":"{color}"}}}}}}',
"-n", self.namespace]
)
def verify_deployment(self) -> bool:
"""验证部署"""
# 检查错误率、延迟等指标
# 这里简化为检查Pod状态
result = subprocess.run(
["kubectl", "get", "pods",
"-l", f"app={self.app_name}",
"-o", "jsonpath={.items[*].status.phase}",
"-n", self.namespace],
capture_output=True, text=True
)
phases = result.stdout.split()
return all(phase == "Running" for phase in phases)
def rollback_deployment(self, color: str):
"""回滚部署"""
subprocess.run(
["kubectl", "delete", "deployment",
f"{self.app_name}-{color}",
"-n", self.namespace]
)
def cleanup_deployment(self, color: str):
"""清理旧部署"""
self.rollback_deployment(color)
# 使用示例
deployer = BlueGreenDeployer("web-app")
success = deployer.deploy("myapp:v2.0.0", "green")
sys.exit(0 if success else 1)
金丝雀发布(Canary)
Argo Rollouts实现
# canary-rollout.yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
name: web-app
spec:
replicas: 10
strategy:
canary:
# 金丝雀步骤
steps:
# 部署1个Pod(10%流量)
- setWeight: 10
- pause: {duration: 5m} # 观察5分钟
# 增加到20%
- setWeight: 20
- pause: {duration: 5m}
# 增加到50%
- setWeight: 50
- pause: {duration: 10m}
# 全量发布
- setWeight: 100
# 分析模板(自动化验证)
analysis:
templates:
- templateName: success-rate
startingStep: 1 # 从第二步开始分析
# 流量管理
canaryService: web-app-canary # 金丝雀服务
stableService: web-app-stable # 稳定版服务
# 流量路由(使用Istio)
trafficRouting:
istio:
virtualServices:
- name: web-app-virtualservice
routes:
- primary
selector:
matchLabels:
app: web-app
template:
metadata:
labels:
app: web-app
spec:
containers:
- name: web-app
image: myapp:v2.0.0
分析模板(自动回滚)
# analysis-template.yaml
apiVersion: argoproj.io/v1alpha1
kind: AnalysisTemplate
metadata:
name: success-rate
spec:
args:
- name: service-name
metrics:
- name: success-rate
interval: 1m
successCondition: result[0] >= 0.95 # 成功率>=95%
failureLimit: 3
provider:
prometheus:
address: http://prometheus:9090
query: |
sum(rate(
http_requests_total{service="{{args.service-name}}", status!~"5.."}[2m]
))
/
sum(rate(
http_requests_total{service="{{args.service-name}}"}[2m]
))
- name: latency-p99
interval: 1m
successCondition: result[0] <= 0.5 # P99延迟<=500ms
failureLimit: 3
provider:
prometheus:
address: http://prometheus:9090
query: |
histogram_quantile(0.99,
sum(rate(
http_request_duration_seconds_bucket{service="{{args.service-name}}"}[2m]
)) by (le)
)
Flagger实现(渐进式交付)
# flagger-canary.yaml
apiVersion: flagger.app/v1beta1
kind: Canary
metadata:
name: web-app
spec:
targetRef:
apiVersion: apps/v1
kind: Deployment
name: web-app
service:
port: 80
targetPort: 8080
gateways:
- public-gateway
hosts:
- app.example.com
analysis:
interval: 1m
threshold: 5 # 连续5次失败则回滚
maxWeight: 50 # 最大金丝雀权重
stepWeight: 10 # 每步增加10%
metrics:
- name: request-success-rate
thresholdRange:
min: 99
interval: 1m
- name: request-duration
thresholdRange:
max: 500
interval: 1m
webhooks:
# 负载测试
- name: load-test
url: http://flagger-loadtester.test/
timeout: 5m
metadata:
cmd: "hey -z 1m -q 10 -c 2 http://web-app-canary.test/"
# 通知
- name: slack-notification
type: event
url: http://flagger-slack.notifications/
metadata:
channel: "#deployments"
渐进式交付(Progressive Delivery)
完整流程图
代码合并
↓
CI流水线(构建+测试)
↓
部署到Staging
↓
自动化测试(单元+集成+E2E)
↓
部署到Production(Canary 10%)
↓
自动化分析(指标监控)
↓
├─ 通过 → 增加流量(20% → 50% → 100%)
└─ 失败 → 自动回滚
↓
全量发布完成
GitOps集成
# argocd-application.yaml
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: web-app
spec:
project: default
source:
repoURL: https://github.com/org/web-app-manifests.git
targetRevision: HEAD
path: production
destination:
server: https://kubernetes.default.svc
namespace: production
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
# 渐进式同步
retry:
limit: 5
backoff:
duration: 5s
factor: 2
maxDuration: 3m
数据库迁移与部署
安全迁移策略
-- 阶段1:添加新列(向后兼容)
ALTER TABLE users ADD COLUMN email_verified BOOLEAN DEFAULT FALSE;
-- 阶段2:部署新代码(同时写入新旧列)
-- 应用代码:INSERT时同时写入新旧列
-- 阶段3:回填历史数据
UPDATE users SET email_verified = (email IS NOT NULL)
WHERE email_verified IS NULL;
-- 阶段4:部署新代码(仅使用新列)
-- 应用代码:只读写新列
-- 阶段5:删除旧列(可选)
-- ALTER TABLE users DROP COLUMN old_email_status;
总结
部署策略选择指南:
滚动更新:
- 适用:大多数无状态服务
- 优点:简单、资源效率高
- 缺点:回滚慢、可能版本混合
蓝绿部署:
- 适用:关键业务、需要快速回滚
- 优点:回滚即时、可充分测试
- 缺点:资源消耗翻倍
金丝雀发布:
- 适用:高风险变更、需要验证
- 优点:风险可控、数据驱动
- 缺点:复杂度高、需要监控支持
渐进式交付:
- 适用:持续部署、成熟团队
- 优点:完全自动化、风险最小
- 缺点:需要完整的CI/CD和监控体系
延伸阅读
- Kubernetes Deployments
- Argo Rollouts
- Flagger Documentation
- Martin Fowler: Canary Release
- Progressive Delivery
继续阅读
探索更多技术文章
浏览归档,发现更多关于系统设计、工具链和工程实践的内容。