xiaoxia/xiaoxia-saas
1
0
Fork 0
Code Issues 16 Pull Requests 12 Actions 34.2k Packages Projects Releases 7 Wiki Activity

fix(worker): 修复 worker 镜像漏 COPY apps/api/app 目录导致转码队列不消费（Staging E2E 阻塞） #1850

Merged
xiaoxia merged 1 commits from fix/worker-dockerfile-missing-lipsync-tasks into develop 2026-09-10 22:56:39 +08:00
Conversation 2 Commits 1 Files Changed 1
+5 -3
xiaoxia commented 2026-09-10 22:28:43 +08:00
Owner
Copy Link
Copy Source

问题

Staging E2E Tests 连续多日失败（Run #48630 起），core-upload 和 core-generation 两个用例持续超时：

  • expect(...).toBe("ready") 收到 "processing"
  • expect(...).toMatch(/...ready$/) 收到 "video/mp4:processing"

上传素材后，ingest_job 一直卡在 pending，资产卡在 processing 超过 30s。

根因

PR #1844（f73837c7，对口型 TTS 异步化）在 apps/worker/worker_app/celery_app.py 的 celery_app.conf.imports 新增了：

"apps.api.app.tasks.lipsync_tts",

但 infra/docker/worker.Dockerfile 没有同步更新：

  1. 漏 COPY 业务目录：worker 镜像只 COPY 了 apps/api/app/config.py 和 apps/api/app/core/，没有 COPY apps/api/app/tasks/（含 lipsync_tts.py）和 apps/api/app/services/（lipsync_tts 依赖的 lipsync_service/mediakit_client）。启动时 ModuleNotFoundError: No module named 'apps.api.app.tasks'，celery worker 启动失败或任务注册不完整，transcode 队列没人消费，ingest 任务永久 pending。
  2. PYTHONPATH 不一致：API 容器 PYTHONPATH=/app:/app/apps/api（from app.core.celery_app import ... 通过 /app/apps/api/app/core/celery_app.py 解析），但 worker 只有 /app:/app/packages，即使补 COPY 了 tasks/，from app.core.celery_app 也找不到 app 包。

修复

  1. 将 COPY apps/api/app/config.py + COPY apps/api/app/core/ 合并为 COPY apps/api/app/ /app/apps/api/app/，把整个 API app 目录（1.2M）复制进 worker 镜像，避免后续新增 task 再漏 COPY；
  2. PYTHONPATH 追加 /app/apps/api，对齐 API 容器配置，from app.* 导入路径在 worker 内可解析。

验证

本地模拟容器 PYTHONPATH + 文件结构后，apps.api.app.tasks.lipsync_tts 及所有 worker_app.tasks.* 模块 import 成功。

影响面

  • 仅影响 worker 镜像构建层缓存（COPY apps/api/app/ 一层），镜像增大约 1.2M；
  • 不改动任何业务代码；
  • CI 需重新构建 worker 镜像 + 跑 Staging E2E 验证。
## 问题 Staging E2E Tests 连续多日失败（Run #48630 起），core-upload 和 core-generation 两个用例持续超时： - `expect(...).toBe("ready")` 收到 `"processing"` - `expect(...).toMatch(/...ready$/)` 收到 `"video/mp4:processing"` 上传素材后，ingest_job 一直卡在 `pending`，资产卡在 `processing` 超过 30s。 ## 根因 PR #1844（f73837c7，对口型 TTS 异步化）在 `apps/worker/worker_app/celery_app.py` 的 `celery_app.conf.imports` 新增了： ```python "apps.api.app.tasks.lipsync_tts", ``` 但 `infra/docker/worker.Dockerfile` 没有同步更新： 1. **漏 COPY 业务目录**：worker 镜像只 COPY 了 `apps/api/app/config.py` 和 `apps/api/app/core/`，没有 COPY `apps/api/app/tasks/`（含 lipsync_tts.py）和 `apps/api/app/services/`（lipsync_tts 依赖的 lipsync_service/mediakit_client）。启动时 `ModuleNotFoundError: No module named 'apps.api.app.tasks'`，celery worker 启动失败或任务注册不完整，transcode 队列没人消费，ingest 任务永久 pending。 2. **PYTHONPATH 不一致**：API 容器 `PYTHONPATH=/app:/app/apps/api`（`from app.core.celery_app import ...` 通过 `/app/apps/api/app/core/celery_app.py` 解析），但 worker 只有 `/app:/app/packages`，即使补 COPY 了 tasks/，`from app.core.celery_app` 也找不到 `app` 包。 ## 修复 1. 将 `COPY apps/api/app/config.py` + `COPY apps/api/app/core/` 合并为 `COPY apps/api/app/ /app/apps/api/app/`，把整个 API app 目录（1.2M）复制进 worker 镜像，避免后续新增 task 再漏 COPY； 2. `PYTHONPATH` 追加 `/app/apps/api`，对齐 API 容器配置，`from app.*` 导入路径在 worker 内可解析。 ## 验证 本地模拟容器 PYTHONPATH + 文件结构后，`apps.api.app.tasks.lipsync_tts` 及所有 worker_app.tasks.* 模块 import 成功。 ## 影响面 - 仅影响 worker 镜像构建层缓存（COPY apps/api/app/ 一层），镜像增大约 1.2M； - 不改动任何业务代码； - CI 需重新构建 worker 镜像 + 跑 Staging E2E 验证。
xiaoxia added 1 commit 2026-09-10 22:28:43 +08:00
fix(worker): 修复 worker 镜像漏 COPY apps/api/app 目录导致 lipsync_tts task 无法导入、转码队列不消费
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2s
Details
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
Details
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 30s
Details
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 30s
Details
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m31s
Details
CI/CD Pipeline / Validate - Style (pull_request) Successful in 1m52s
Details
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m31s
Details
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m50s
Details
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 3m44s
Details
AI Code Review / AI Code Review (pull_request) Successful in 6m39s
Details
CI/CD Pipeline / Unit Tests (pull_request) Successful in 9m8s
Details
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 10m33s
Details
CI/CD Pipeline / Validate - Security (pull_request) Successful in 21m22s
Details
CI/CD Pipeline / CI Gate (pull_request) Successful in 3s
Details
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
Details
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 7s
Details
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 1m25s
Details
CI/CD Pipeline / Deploy Production (pull_request) Failing after 136h33m41s
Details
CI/CD Pipeline / Build Production API Image (pull_request) Failing after 136h33m44s
Details
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 136h54m58s
Details
CI/CD Pipeline / Build Staging Web Image (pull_request) Failing after 136h55m7s
Details
CI/CD Pipeline / Frontend Lint (pull_request) Failing after 136h54m51s
Details
CI/CD Pipeline / Build Staging Worker Image (pull_request) Failing after 136h54m54s
Details
CI/CD Pipeline / Build Production Web Image (pull_request) Failing after 136h33m31s
Details
CI/CD Pipeline / ACR Image Cleanup (pull_request) Failing after 136h54m44s
Details
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Failing after 136h54m49s
Details
CI/CD Pipeline / Canary Release to Production (pull_request) Failing after 136h33m28s
Details
CI/CD Pipeline / Build Production Worker Image (pull_request) Failing after 136h33m31s
Details
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 136h54m44s
Details
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Failing after 136h54m48s
Details
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Failing after 136h54m49s
Details
CI/CD Pipeline / PR Build Web Image (pull_request) Failing after 136h54m50s
Details
CI/CD Pipeline / Build Staging API Image (pull_request) Failing after 136h54m54s
Details
CI/CD Pipeline / Frontend Unit Tests (pull_request) Failing after 136h54m51s
Details
CI/CD Pipeline / Check push changed paths (pull_request) Failing after 136h54m56s
Details
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Failing after 137h29m54s
Details
0a369a607b 
PR #1844 在 worker celery_app.conf.imports 新增了 "apps.api.app.tasks.lipsync_tts"，
但 worker.Dockerfile 只 COPY 了 config.py 和 core/，没 COPY tasks/ 和 services/，
且 PYTHONPATH 缺少 /app/apps/api（API 容器用 from app.xxx 解析的必需前缀）。

导致 worker 启动时 celery import lipsync_tts 失败，transcode 队列没有正常注册，
素材上传后卡在 processing 状态超时，Staging E2E 的 core-upload 和 core-generation
两个用例持续失败（素材永远到不了 ready）。

修复：
1. 将单独 COPY config.py/core/ 改为 COPY 整个 apps/api/app/ 目录（1.2M），
   避免后续新增 task 再次漏 COPY；
2. PYTHONPATH 追加 /app/apps/api，对齐 API 容器配置，让 from app.core.celery_app、
   from app.services.* 等导入路径在 worker 中也能解析。
gitea-actions bot commented 2026-09-10 22:31:14 +08:00
Copy Link
Copy Source

🚀 预览环境已部署

项目 详情
PR号 #1850
预览链接 https://pr-1850.preview.xiaoxiajianji.com
API环境 staging

💡 预览环境使用 staging API 数据，请勿在预览环境中操作重要数据。

🔄 每次提交新代码后预览环境会自动更新。

🗑️ PR 关闭或合并后，预览环境会自动清理。

🚀 **预览环境已部署** | 项目 | 详情 | |------|------| | PR号 | #1850 | | 预览链接 | [https://pr-1850.preview.xiaoxiajianji.com](https://pr-1850.preview.xiaoxiajianji.com) | | API环境 | staging | > 💡 预览环境使用 staging API 数据，请勿在预览环境中操作重要数据。 > > 🔄 每次提交新代码后预览环境会自动更新。 > > 🗑️ PR 关闭或合并后，预览环境会自动清理。
xiaoxia merged commit 582f73c2f2 into develop 2026-09-10 22:56:39 +08:00
xiaoxia deleted branch fix/worker-dockerfile-missing-lipsync-tasks 2026-09-10 22:56:40 +08:00
gitea-actions bot commented 2026-09-10 22:58:01 +08:00
Copy Link
Copy Source

🗑️ 预览环境已清理

PR #1850 已关闭或合并，对应的预览环境已被清理。

如有需要，可以重新打开 PR 来重新生成预览环境。

🗑️ **预览环境已清理** PR #1850 已关闭或合并，对应的预览环境已被清理。 > 如有需要，可以重新打开 PR 来重新生成预览环境。
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
architecture
架构设计
 backend
后端相关
 blocked
阻塞中
 bug
Bug修复
 ci-bug
CI本身的bug
 ci-optimization
CI优化类任务
 ci-pipeline
CI流水线/workflow相关
 cleanup
代码清理
 devops
运维/部署/CI
 documentation
文档相关
 feature
新功能
 frontend
前端相关
 P0
阻塞性，必须立即处理
 P1
高优先级
 P2
中优先级
 P3
低优先级/后考虑
 question
需要讨论确认
 ready
PR CI通过，可合并
 refactor
代码重构
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
auto-approve-bot frontend-dev xiaoxia
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: xiaoxia/xiaoxia-saas#1850
Delete Branch "fix/worker-dockerfile-missing-lipsync-tasks"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?