xiaoxia/xiaoxia-saas
1
0
Fork 0
Code Issues 16 Pull Requests 12 Actions 34.2k Packages Projects Releases 7 Wiki Activity

feat(gpu): #1978 MuseTalk GPU Worker 反向轮询对接（后端API + Worker脚本） #1979

Merged
auto-approve-bot merged 1 commits from feat/gpu-musetalk-worker into develop 2026-09-18 19:59:51 +08:00
Conversation 2 Commits 1 Files Changed 19
+1538 -1
xiaoxia commented 2026-09-18 19:40:41 +08:00
Owner
Copy Link
Copy Source

背景

阿里云CPU服务器无GPU无法跑MuseTalk。用户有一台RTX2060 6G本地电脑（192.168.0.193）部署MuseTalk，采用反向轮询模式（Worker主动拉任务→处理→回传），无需内网穿透。

后端API（/api/v1/gpu 前缀，GPU_WORKER_TOKEN 长期Token鉴权）

方法 路径 用途
POST /gpu/register Worker 注册/心跳（hostname/gpu_name/free_vram/capabilities）
GET /gpu/lipsync/poll?worker_id=xxx 拉取 pending 任务，原子 CLAIM；无任务返回 204；返回视频/音频预签名下载URL + 结果预签名PUT URL
POST /gpu/lipsync/result multipart 上报结果（支持代传文件到OSS或先自PUT后无文件上报）
GET /gpu/lipsync/status/{task_id} 业务侧查询任务状态

数据层

  • 新表 gpu_lipsync_tasks：id/status(pending/processing/done/failed)/video_url/audio_url/result_url/result_duration/worker_id/attempt/error_msg/created_at/started_at/finished_at/last_heartbeat_at/lipsync_job_id/user_id/project_id
  • 新表 gpu_workers：worker_id/hostname/gpu_name/free_vram_mb/capabilities/last_heartbeat_at
  • alembic 迁移：081_add_gpu_lipsync_tasks

可靠性

  • 原子认领（UPDATE WHERE status=pending）防并发
  • processing 任务超时（默认300s，可配 GPU_TASK_TIMEOUT_SECONDS）自动回退 pending 重试
  • 最多重试 3 次（MAX_ATTEMPTS），耗尽标记 failed
  • 失败可配置本地重试（TASK_MAX_RETRY）
  • 输入/输出都用预签名URL，Worker不需要OSS凭证

Worker脚本（deploy/gpu_worker/，零项目代码依赖）

  • gpu_worker.py：register→5s轮询→下载视频/音频→POST本地MuseTalk:7861/inference→multipart回传结果
  • requirements.txt：仅 requests>=2.31
  • xiaoxia-gpu-worker.service：systemd 开机自启
  • .env.example + README.md：Linux/Windows部署+联调+故障排查

配置

  • GPU_WORKER_TOKEN：长期API Token（staging/production必填，development空值放行本地调试）
  • GPU_TASK_TIMEOUT_SECONDS：任务超时，默认300s
  • 已在.env.example、deploy/configs/.env.{staging,production}、scripts/render_env.sh、ci-pipeline.yml 中添加

测试

  • 单元测试 10 个全绿：创建/轮询空/原子认领/并发单胜/成功上报/失败重试/耗尽失败/超时回退/注册更新/按lipsync_job查询
  • 业务侧集成（在lipsync_service里切到GPU路径）留作下一步，当前先交付API+Worker基础设施，可以先手动调GpuLipsyncService.create_task做端到端联调

待运维配置

  1. 生成随机 GPU_WORKER_TOKEN 写入 Gitea Actions Secrets + staging/production .env
  2. staging部署后给 RTX2060 电脑部署 Worker，.env 配 staging API + 同一个Token
  3. Worker启动后观察日志，手动创建测试任务验证端到端流程
## 背景 阿里云CPU服务器无GPU无法跑MuseTalk。用户有一台RTX2060 6G本地电脑（192.168.0.193）部署MuseTalk，采用反向轮询模式（Worker主动拉任务→处理→回传），无需内网穿透。 ## 后端API（/api/v1/gpu 前缀，GPU_WORKER_TOKEN 长期Token鉴权） | 方法 | 路径 | 用途 | |---|---|---| | POST | /gpu/register | Worker 注册/心跳（hostname/gpu_name/free_vram/capabilities） | | GET | /gpu/lipsync/poll?worker_id=xxx | 拉取 pending 任务，原子 CLAIM；无任务返回 204；返回视频/音频预签名下载URL + 结果预签名PUT URL | | POST | /gpu/lipsync/result | multipart 上报结果（支持代传文件到OSS或先自PUT后无文件上报） | | GET | /gpu/lipsync/status/{task_id} | 业务侧查询任务状态 | ## 数据层 - 新表 `gpu_lipsync_tasks`：id/status(pending/processing/done/failed)/video_url/audio_url/result_url/result_duration/worker_id/attempt/error_msg/created_at/started_at/finished_at/last_heartbeat_at/lipsync_job_id/user_id/project_id - 新表 `gpu_workers`：worker_id/hostname/gpu_name/free_vram_mb/capabilities/last_heartbeat_at - alembic 迁移：`081_add_gpu_lipsync_tasks` ## 可靠性 - 原子认领（UPDATE WHERE status=pending）防并发 - processing 任务超时（默认300s，可配 GPU_TASK_TIMEOUT_SECONDS）自动回退 pending 重试 - 最多重试 3 次（MAX_ATTEMPTS），耗尽标记 failed - 失败可配置本地重试（TASK_MAX_RETRY） - 输入/输出都用预签名URL，Worker不需要OSS凭证 ## Worker脚本（deploy/gpu_worker/，零项目代码依赖） - `gpu_worker.py`：register→5s轮询→下载视频/音频→POST本地MuseTalk:7861/inference→multipart回传结果 - `requirements.txt`：仅 requests>=2.31 - `xiaoxia-gpu-worker.service`：systemd 开机自启 - `.env.example` + `README.md`：Linux/Windows部署+联调+故障排查 ## 配置 - `GPU_WORKER_TOKEN`：长期API Token（staging/production必填，development空值放行本地调试） - `GPU_TASK_TIMEOUT_SECONDS`：任务超时，默认300s - 已在.env.example、deploy/configs/.env.{staging,production}、scripts/render_env.sh、ci-pipeline.yml 中添加 ## 测试 - 单元测试 10 个全绿：创建/轮询空/原子认领/并发单胜/成功上报/失败重试/耗尽失败/超时回退/注册更新/按lipsync_job查询 - 业务侧集成（在lipsync_service里切到GPU路径）留作下一步，当前先交付API+Worker基础设施，可以先手动调GpuLipsyncService.create_task做端到端联调 ## 待运维配置 1. 生成随机 GPU_WORKER_TOKEN 写入 Gitea Actions Secrets + staging/production .env 2. staging部署后给 RTX2060 电脑部署 Worker，.env 配 staging API + 同一个Token 3. Worker启动后观察日志，手动创建测试任务验证端到端流程
xiaoxia added 1 commit 2026-09-18 19:40:43 +08:00
feat(gpu): #1978 MuseTalk GPU Worker 反向轮询对接 - 后端API+Worker脚本
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
Details
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 3s
Details
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
Details
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
Details
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
Details
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
Details
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
Details
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
Details
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
Details
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 52s
Details
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 55s
Details
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m12s
Details
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m10s
Details
CI/CD Pipeline / Integration Tests (pull_request) Successful in 5m26s
Details
CI/CD Pipeline / Validate - Style (pull_request) Successful in 6m46s
Details
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 7m42s
Details
AI Code Review / AI Code Review (pull_request) Successful in 7m59s
Details
CI/CD Pipeline / Unit Tests (pull_request) Successful in 16m43s
Details
CI/CD Pipeline / Validate - Security (pull_request) Successful in 18m23s
Details
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
Details
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
Details
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
Details
CI/CD Pipeline / CI Gate (pull_request) Successful in 2s
Details
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 11m6s
Details
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 25s
Details
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 1m4s
Details
5ab30f7706 
新增 MuseTalk 本地 GPU Worker 反向轮询能力，解决阿里云CPU服务器无法跑GPU推理的问题。

后端API层（新增）：
- 表 gpu_lipsync_tasks（id/status/video_url/audio_url/result_url/worker_id/
  attempt/error_msg/created_at/started_at/finished_at/last_heartbeat_at）
- 表 gpu_workers（worker_id/hostname/gpu_name/free_vram_mb/capabilities/last_heartbeat_at）
- alembic 迁移 081_add_gpu_lipsync_tasks
- 4个接口（/api/v1/gpu 前缀，长期 GPU_WORKER_TOKEN Bearer 鉴权）：
  * POST /gpu/register — Worker 注册/心跳
  * GET  /gpu/lipsync/poll?worker_id=xxx — 拉取 pending 任务（带预签名下载+PUT上传URL），
    原子 UPDATE WHERE status=pending 防并发；无任务返回204
  * POST /gpu/lipsync/result — multipart 上报结果（支持Worker代传文件到OSS，
    或先自PUT到预签名URL再无文件上报）
  * GET  /gpu/lipsync/status/{task_id} — 任务状态查询
- 超时回退：processing 任务超过 gpu_task_timeout_seconds（默认300s）无心跳
  自动回退 pending 重试，最多 MAX_ATTEMPTS(3) 次
- 配置项：GPU_WORKER_TOKEN / GPU_TASK_TIMEOUT_SECONDS（SharedSettings）
- OSS 存储：SharedStorageService 新增 get_upload_url 预签名 PUT URL
- CI/环境：.env/.env.staging/.env.production + render_env.sh + ci-pipeline.yml
  均注入 GPU_WORKER_TOKEN

Worker脚本（deploy/gpu_worker/，零项目代码依赖，仅依赖requests）：
- gpu_worker.py：启动register→5s轮询→下载视频/音频→POST本地MuseTalk /inference
  →multipart回传结果→失败本地重试+上报failed→单任务串行
- requirements.txt：仅 requests>=2.31
- xiaoxia-gpu-worker.service：systemd 开机自启单元
- .env.example：环境变量样例
- README.md：Linux/Windows部署+联调+故障排查

单元测试：10个用例覆盖创建/轮询/并发认领/成功/失败重试/
最大重试失败/超时回退/注册心跳/按lipsync_job查询，全绿。
gitea-actions bot commented 2026-09-18 19:43:00 +08:00
Copy Link
Copy Source

🚀 预览环境已部署

项目 详情
PR号 #1979
预览链接 https://pr-1979.preview.xiaoxiajianji.com
API环境 staging

💡 预览环境使用 staging API 数据，请勿在预览环境中操作重要数据。

🔄 每次提交新代码后预览环境会自动更新。

🗑️ PR 关闭或合并后，预览环境会自动清理。

🚀 **预览环境已部署** | 项目 | 详情 | |------|------| | PR号 | #1979 | | 预览链接 | [https://pr-1979.preview.xiaoxiajianji.com](https://pr-1979.preview.xiaoxiajianji.com) | | API环境 | staging | > 💡 预览环境使用 staging API 数据，请勿在预览环境中操作重要数据。 > > 🔄 每次提交新代码后预览环境会自动更新。 > > 🗑️ PR 关闭或合并后，预览环境会自动清理。
auto-approve-bot merged commit fbf8844f25 into develop 2026-09-18 19:59:51 +08:00
auto-approve-bot deleted branch feat/gpu-musetalk-worker 2026-09-18 19:59:54 +08:00
gitea-actions bot commented 2026-09-18 20:00:55 +08:00
Copy Link
Copy Source

🗑️ 预览环境已清理

PR #1979 已关闭或合并，对应的预览环境已被清理。

如有需要，可以重新打开 PR 来重新生成预览环境。

🗑️ **预览环境已清理** PR #1979 已关闭或合并，对应的预览环境已被清理。 > 如有需要，可以重新打开 PR 来重新生成预览环境。
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
architecture
架构设计
 backend
后端相关
 blocked
阻塞中
 bug
Bug修复
 ci-bug
CI本身的bug
 ci-optimization
CI优化类任务
 ci-pipeline
CI流水线/workflow相关
 cleanup
代码清理
 devops
运维/部署/CI
 documentation
文档相关
 feature
新功能
 frontend
前端相关
 P0
阻塞性，必须立即处理
 P1
高优先级
 P2
中优先级
 P3
低优先级/后考虑
 question
需要讨论确认
 ready
PR CI通过，可合并
 refactor
代码重构
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
auto-approve-bot frontend-dev xiaoxia
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: xiaoxia/xiaoxia-saas#1979
Delete Branch "feat/gpu-musetalk-worker"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?