fix: Worker health check 扫描全部 /proc 进程修复 unhealthy 误判 #1825

Merged
xiaoxia merged 1 commits from fix/worker-healthcheck-ci-e2e into develop 2026-09-09 19:09:15 +08:00
Owner

问题

Worker 容器内无 pgrep/ps 工具,旧 health check 使用 grep -q celery /proc/1/cmdline 检查 PID 1 的 cmdline,但 PID 1 是 entrypoint-worker.sh(bash 脚本),不含 celery 关键字,导致 health check 永远返回 unhealthy。

同时 deploy 脚本中 sh -c 嵌套引号导致 $pid 变量被外层 shell 提前展开为空。

修复

改用 grep -lq celery /proc/[0-9]*/cmdline 扫描容器内所有进程的 cmdline,找到任意 celery 进程即判定 healthy。无需变量、无嵌套 shell、无 pgrep 依赖。

影响文件

  • infra/docker/compose.yml
  • infra/docker/deploy-staging-registry.sh
  • infra/docker/deploy-production-registry.sh
## 问题 Worker 容器内无 pgrep/ps 工具,旧 health check 使用 `grep -q celery /proc/1/cmdline` 检查 PID 1 的 cmdline,但 PID 1 是 `entrypoint-worker.sh`(bash 脚本),不含 celery 关键字,导致 health check 永远返回 unhealthy。 同时 deploy 脚本中 `sh -c` 嵌套引号导致 `$pid` 变量被外层 shell 提前展开为空。 ## 修复 改用 `grep -lq celery /proc/[0-9]*/cmdline` 扫描容器内所有进程的 cmdline,找到任意 celery 进程即判定 healthy。无需变量、无嵌套 shell、无 pgrep 依赖。 ## 影响文件 - `infra/docker/compose.yml` - `infra/docker/deploy-staging-registry.sh` - `infra/docker/deploy-production-registry.sh`

🚀 预览环境已部署

项目 详情
PR号 #1825
预览链接 https://pr-1825.preview.xiaoxiajianji.com
API环境 staging

💡 预览环境使用 staging API 数据,请勿在预览环境中操作重要数据。

🔄 每次提交新代码后预览环境会自动更新。

🗑️ PR 关闭或合并后,预览环境会自动清理。

🚀 **预览环境已部署** | 项目 | 详情 | |------|------| | PR号 | #1825 | | 预览链接 | [https://pr-1825.preview.xiaoxiajianji.com](https://pr-1825.preview.xiaoxiajianji.com) | | API环境 | staging | > 💡 预览环境使用 staging API 数据,请勿在预览环境中操作重要数据。 > > 🔄 每次提交新代码后预览环境会自动更新。 > > 🗑️ PR 关闭或合并后,预览环境会自动清理。
xiaoxia force-pushed fix/worker-healthcheck-ci-e2e from f2ab0c87a2 to ec5cc4c206 2026-09-09 18:27:54 +08:00 Compare
xiaoxia added 1 commit 2026-09-09 18:42:25 +08:00
fix: Worker health check 扫描全部 /proc 进程修复 unhealthy 误判
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m1s
AI Code Review / AI Code Review (pull_request) Successful in 6m36s
PR Automation / Auto Approve on CI Green (pull_request) Failing after 13m19s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been skipped
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 1s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 15s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 16s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 11s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 30s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m16s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 1m58s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 2m53s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 6m33s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 12m20s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 6s
54defa4429
问题:Worker 容器内无 pgrep/ps 工具,旧 health check 使用
'grep -q celery /proc/1/cmdline' 检查 PID 1 的 cmdline,
但 PID 1 是 entrypoint-worker.sh(bash 脚本),不含 celery 关键字,
导致 health check 永远返回 unhealthy。

同时部署脚本中 sh -c 嵌套引号导致 $pid 变量被外层 shell 提前展开为空。

修复:改用 grep -lq celery /proc/[0-9]*/cmdline 扫描所有进程 cmdline,
找到任意 celery 进程即判定 healthy。无需变量、无嵌套 shell、无 pgrep 依赖。

影响文件:
- infra/docker/compose.yml
- infra/docker/deploy-staging-registry.sh
- infra/docker/deploy-production-registry.sh
xiaoxia force-pushed fix/worker-healthcheck-ci-e2e from ec5cc4c206 to 54defa4429 2026-09-09 18:42:25 +08:00 Compare
xiaoxia merged commit 1bd0f0c1ae into develop 2026-09-09 19:09:15 +08:00

🗑️ 预览环境已清理

PR #1825 已关闭或合并,对应的预览环境已被清理。

如有需要,可以重新打开 PR 来重新生成预览环境。

🗑️ **预览环境已清理** PR #1825 已关闭或合并,对应的预览环境已被清理。 > 如有需要,可以重新打开 PR 来重新生成预览环境。
Sign in to join this conversation.