xiaoxia/xiaoxia-saas

fix(voice-clone): 处理中卡死兜底 — worker重启/Celery消息丢失后processing永久卡住 #2050

Merged

auto-approve-bot merged 6 commits from fix/voice-clone-stuck-processing into develop

2026-09-26 15:31:08 +08:00

Author	SHA1	Message	Date
saas-backend	1ff2ab1c1f	test(voice-clone): 放宽 model 身份断言，兼容 CI 测试加载顺序 CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped Details CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s Details CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 1s Details CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped Details CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped Details CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped Details CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped Details CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped Details CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped Details CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped Details CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped Details CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped Details CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped Details CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 38s Details CI/CD Pipeline / PR Build API Image (pull_request) Successful in 41s Details CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m53s Details CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 2m28s Details CI/CD Pipeline / Validate - Style (pull_request) Successful in 2m44s Details PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m4s Details Preview Deploy / Deploy Preview Environment (pull_request) Successful in 4m39s Details CI/CD Pipeline / Validate - Security (pull_request) Successful in 6m19s Details AI Code Review / AI Code Review (pull_request) Successful in 6m35s Details CI/CD Pipeline / Unit Tests (pull_request) Successful in 7m3s Details CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped Details CI/CD Pipeline / Deploy Production (pull_request) Has been skipped Details CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped Details CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped Details CI/CD Pipeline / CI Gate (pull_request) Successful in 1s Details PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 4m37s Details ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 31s Details Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 37s Details	2026-09-26 15:23:22 +08:00
saas-backend	08a52c7bba	test(voice-clone): 为 cleanup_stale_processing 补单测，满足增量覆盖率门槛 CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 0s Details CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped Details CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped Details CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s Details CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped Details CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped Details CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped Details CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped Details CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped Details CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped Details CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped Details CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped Details CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 52s Details CI/CD Pipeline / PR Build API Image (pull_request) Successful in 54s Details Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m19s Details CI/CD Pipeline / Unit Tests (pull_request) Failing after 1m46s Details CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 2m16s Details CI/CD Pipeline / Validate - Style (pull_request) Successful in 2m30s Details PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m6s Details CI/CD Pipeline / Integration Tests (pull_request) Successful in 3m10s Details CI/CD Pipeline / Validate - Security (pull_request) Successful in 6m18s Details CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped Details CI/CD Pipeline / Deploy Production (pull_request) Has been skipped Details CI/CD Pipeline / CI Gate (pull_request) Failing after 1s Details CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped Details PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 3m15s Details CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped Details AI Code Review / AI Code Review (pull_request) Successful in 6m44s Details 新增 5 个用例覆盖： - 无卡死记录不 commit - 单条超时记录标 failed + 错误信息含分钟数 - 自定义 timeout 反映在错误信息中 - 多条记录批量清理且只 commit 一次 - query 链路包含 status/updated_at 双过滤条件使用 sys.modules 预注入假模型类，避免 SQLAlchemy/DB 依赖。	2026-09-26 15:13:26 +08:00
CI Bot	c4150311a6	style: auto-format with black + isort + ruff + prettier [skip ci-format-check] CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped Details CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 0s Details CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2s Details CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped Details CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped Details CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped Details CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped Details CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped Details CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped Details CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped Details CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped Details CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped Details CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 50s Details CI/CD Pipeline / PR Build API Image (pull_request) Successful in 51s Details Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m15s Details CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m43s Details CI/CD Pipeline / Unit Tests (pull_request) Failing after 2m6s Details CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 2m14s Details CI/CD Pipeline / Validate - Style (pull_request) Successful in 2m30s Details PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m9s Details AI Code Review / AI Code Review (pull_request) Successful in 6m38s Details CI/CD Pipeline / Validate - Security (pull_request) Successful in 6m34s Details CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped Details CI/CD Pipeline / CI Gate (pull_request) Failing after 1s Details CI/CD Pipeline / Deploy Production (pull_request) Has been skipped Details CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped Details CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped Details PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 3m32s Details	2026-09-26 06:55:34 +00:00
saas-backend	ce1bd8e7c7	test(voice-clone): 修复 _resolve_task 跨 Celery 版本鲁棒性 CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped Details CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2s Details CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped Details CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s Details CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped Details CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped Details CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped Details CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped Details CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped Details CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped Details CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped Details CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped Details CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 53s Details CI/CD Pipeline / PR Build API Image (pull_request) Successful in 56s Details Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m28s Details CI/CD Pipeline / Integration Tests (pull_request) Successful in 2m18s Details CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 2m24s Details CI/CD Pipeline / Unit Tests (pull_request) Failing after 2m24s Details PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m11s Details CI/CD Pipeline / Validate - Style (pull_request) Has been cancelled Details CI/CD Pipeline / Validate - Security (pull_request) Has been cancelled Details CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled Details CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled Details CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled Details CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled Details CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled Details CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled Details CI/CD Pipeline / CI Gate (pull_request) Has been cancelled Details AI Code Review / AI Code Review (pull_request) Has been cancelled Details PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been cancelled Details - _get_current_object() 加 try/except 兜底，无 active app context 时不抛 AttributeError - _resolve_task 统一返回 (callable, mock_self, real_task) 三元组 - timeout 重试用例复用 real_task patch retry，不再二次调用无保护的 _get_current_object() - mock_self 判断改为 is not None（语义更清晰，兼容 MagicMock 真值）	2026-09-26 14:51:57 +08:00
CI Bot	73068eb22c	style: auto-format with black + isort + ruff + prettier [skip ci-format-check]	2026-09-26 14:51:57 +08:00
CI Bot	a33532113b	fix(voice-clone): 处理中卡死兜底 — worker重启/Celery消息丢失后processing记录永久卡住问题根因: - 声音克隆走 API提交CosyVoice→Celery worker轮询5分钟→mark ready/failed - 当worker容器重启、进程OOM或Celeryprefetch消息丢失时,已进入processing状态的voice_clone_profile没有兜底恢复机制 - 对比ingest链路有cleanup_stale_ingest_jobs beat任务+worker_ready启动恢复,voice_clone链路缺失 - 结果: 用户看到"克隆处理中，请稍候..."永久转圈,刷新也不变修复: 1. SQLAlchemyVoiceCloneProfileRepository新增cleanup_stale_processing方法: 扫描updated_at超过10分钟的processing记录(正常克隆<5分钟),标记failed并提示用户重试 2. _startup.py新增recover_stale_voice_clones_on_startup+worker_ready信号: worker启动时一次性扫描恢复,用户重启/部署后立即解锁卡死任务 3. cleanup.py新增scheduled_cleanup_stale_voice_clones beat任务: 每5分钟巡检兜底,防止运行期OOM/消息丢失导致的新卡死 4. celery_app.py beat_schedule注册新任务 5. voice_clone.py任务入口加received日志,便于日志排查	2026-09-26 14:51:57 +08:00