feat(ci): Production部署接入健康检查+自动回滚框架 #325

Merged
xiaoxia merged 3 commits from feat/production-healthcheck into develop 2026-07-15 00:34:23 +08:00
Owner

变更内容

1. 新增 scripts/ci_production_healthcheck.sh(Production专用健康检查+自动回滚)

  • 公网视角健康检查:API /health + Web首页 + API /docs + 登录API(期望401/422),超时180秒
  • 自动回滚机制:部署前记录当前运行的镜像版本,健康检查失败时通过SSH重新部署旧版本
  • 回滚后验证:回滚完成后重新检查健康状态,确认服务恢复
  • 飞书通知:success / failure / rollback 三种状态卡片通知

2. 升级 scripts/deploy_notify.py

  • 新增 --env 参数支持 staging / production 两种环境
  • 卡片显示环境标识和对应访问链接
  • 版本号支持 tag 显示(v0.1.xxx 格式)

3. 修改 ci-cd.yml deploy-production 阶段

  • 新增 Checkout 步骤获取脚本
  • 新增 Health check + auto rollback 步骤(部署后自动检查,不健康自动回滚)
  • 成功通知由健康检查脚本发出(更详细的卡片)
  • 失败兜底通知升级为 deploy_notify.py 卡片格式,且检测标记文件避免重复通知
  • timeout 从 20 分钟调整为 25 分钟(预留健康检查+回滚时间)

设计说明

  • Production 采用 SSH 直连部署模式,与 Staging 的 Watchtower 模式不同
  • 回滚方式:通过 SSH 在生产服务器上重新拉取旧版本镜像并重启容器(不执行 migration 回滚)
  • 注意:migration 不可回滚(新版本可能新增字段/表),回滚后旧代码仍可兼容(需确保 migration 是增量的、向前兼容的)

关联任务

  • 生产环境部署升级:健康检查+回滚方案从Staging同步到Production
  • 部署通知从占位改为实际可用的飞书卡片通知
## 变更内容 ### 1. 新增 `scripts/ci_production_healthcheck.sh`(Production专用健康检查+自动回滚) - **公网视角健康检查**:API /health + Web首页 + API /docs + 登录API(期望401/422),超时180秒 - **自动回滚机制**:部署前记录当前运行的镜像版本,健康检查失败时通过SSH重新部署旧版本 - **回滚后验证**:回滚完成后重新检查健康状态,确认服务恢复 - **飞书通知**:success / failure / rollback 三种状态卡片通知 ### 2. 升级 `scripts/deploy_notify.py` - 新增 `--env` 参数支持 staging / production 两种环境 - 卡片显示环境标识和对应访问链接 - 版本号支持 tag 显示(v0.1.xxx 格式) ### 3. 修改 `ci-cd.yml` deploy-production 阶段 - 新增 Checkout 步骤获取脚本 - 新增 Health check + auto rollback 步骤(部署后自动检查,不健康自动回滚) - 成功通知由健康检查脚本发出(更详细的卡片) - 失败兜底通知升级为 deploy_notify.py 卡片格式,且检测标记文件避免重复通知 - timeout 从 20 分钟调整为 25 分钟(预留健康检查+回滚时间) ### 设计说明 - Production 采用 SSH 直连部署模式,与 Staging 的 Watchtower 模式不同 - 回滚方式:通过 SSH 在生产服务器上重新拉取旧版本镜像并重启容器(不执行 migration 回滚) - 注意:migration 不可回滚(新版本可能新增字段/表),回滚后旧代码仍可兼容(需确保 migration 是增量的、向前兼容的) ### 关联任务 - 生产环境部署升级:健康检查+回滚方案从Staging同步到Production - 部署通知从占位改为实际可用的飞书卡片通知
auto-approve-bot approved these changes 2026-07-14 20:02:00 +08:00
auto-approve-bot left a comment
Collaborator

CI全绿,自动审批通过。

CI全绿,自动审批通过。
xiaoxia closed this pull request 2026-07-14 21:34:13 +08:00
xiaoxia deleted branch feat/production-healthcheck 2026-07-14 21:34:13 +08:00
xiaoxia reopened this pull request 2026-07-14 21:34:56 +08:00
xiaoxia force-pushed feat/production-healthcheck from 1704a75084 to f36d6e5c40 2026-07-14 22:51:09 +08:00 Compare
xiaoxia force-pushed feat/production-healthcheck from f36d6e5c40 to 107c4457df 2026-07-15 00:18:37 +08:00 Compare
xiaoxia added 3 commits 2026-07-15 00:31:16 +08:00
feat(ci): deploy-production接入健康检查+自动回滚
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Successful in 1m56s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 2m0s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 2m27s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m21s
CI/CD Pipeline / Validate Code Quality And Tests (push) Successful in 1m54s
CI/CD Pipeline / Frontend Lint (push) Successful in 1m57s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Unit Tests (push) Successful in 2m2s
CI/CD Pipeline / Integration Tests (push) Successful in 1m12s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 2m6s
CI/CD Pipeline / Build Staging API Image (push) Successful in 7m4s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 23m13s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 12m19s
CI/CD Pipeline / Staging E2E Tests (push) Successful in 2m9s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 3m14s
62ab361940
xiaoxia force-pushed feat/production-healthcheck from 107c4457df to 62ab361940 2026-07-15 00:31:16 +08:00 Compare
xiaoxia merged commit 62ab361940 into develop 2026-07-15 00:34:23 +08:00
Sign in to join this conversation.