17 Commits

Author	SHA1	Message	Date
xiaoxia	1855dc54dc	fix(gpu): 修复MuseTalk蓝色块 — Step5删除多余BGR2RGB转换，直接传BGR给VAE (#2015) ... Co-authored-by: xiaoxia <dev@xiaoxiajianji.com> Co-committed-by: xiaoxia <dev@xiaoxiajianji.com>	2026-09-23 00:08:08 +08:00
saas-backend	130120c8a7	fix(GPU): 修复GFPGAN增强后蓝色遮罩问题 ... 根因：return_rgb=False 返回 BGR 格式，但后续 cvtColor 误用 RGB2BGR 导致 R/B 通道互换，人脸变蓝。 修复： - return_rgb 改为 True，输出为 RGB 格式 - 保持后续 RGB→BGR 转换正确 - 简化逻辑：去掉复杂的色彩校正（mean/std对齐）， 避免引入额外的色偏风险 - 添加清晰的色彩通道注释，防止后续维护出错	2026-09-22 12:53:45 +08:00
xiaoxia	f032152eaa	fix(P0): 修复取消链路断裂，前端取消后 GPU 仍继续推理 (#2009) ... Co-authored-by: xiaoxia <dev@xiaoxiajianji.com> Co-committed-by: xiaoxia <dev@xiaoxiajianji.com>	2026-09-22 01:24:07 +08:00
saas-backend	570a06f8c4	fix: 修复 GPU 视频无声 + GFPGAN 紫色色块 ... Bug1 无声： - Step 8 将 shutil.copy2 替换为 _mux_video_with_audio - 推理输出的无声视频现在会正确封装驱动音频 Bug2 紫色/灰色色块： - GFPGAN 模型加载和推理均改用 FP32，消除 FP16 色偏 - 增强后增加色彩校正（均值/标准差对齐到原始人脸） - weight 从 0.5 降至 0.35，减少增强覆盖比例 - 保留归一化逻辑 [-1, 1] 确保模型输入正确	2026-09-21 22:51:18 +08:00
saas-backend	ffb8ea6790	fix: 移除 MCI 插帧，60fps 输入直接输出 25fps 结果 ... - 移除 Step 8 的 minterpolate MCI 运动补偿插帧（CPU密集，343帧超6分钟导致超时504） - 高帧率视频降帧至 25fps 后直接输出，对口型数字人场景完全够用 - 插帧失败不再导致整个任务 504，直接降级返回 25fps 结果	2026-09-21 19:22:52 +08:00
xiaoxia	cc01a5da69	perf: RTX3060 12G显存适配，batch_size上限提升到8 (#2007) ... Co-authored-by: xiaoxia <dev@xiaoxiajianji.com> Co-committed-by: xiaoxia <dev@xiaoxiajianji.com>	2026-09-21 19:09:01 +08:00
xiaoxia	7babea3d24	fix: musetalk_server 60fps降帧+超时终止+health GFPGAN字段 (#2006) ... Co-authored-by: xiaoxia <dev@xiaoxiajianji.com> Co-committed-by: xiaoxia <dev@xiaoxiajianji.com>	2026-09-21 17:24:13 +08:00
saas-backend-agent	3424e55a32	feat(gpu): #1978 MuseTalk v1.5 + GFPGAN face enhancement + ffmpeg pipe optimization (#2003) ... - MuseTalk v1.5 完整推理pipeline（DWPose+SFD GPU人脸检测） - GFPGANv1.4 人脸超分增强（FP16, 可通过 MUSE_USE_GFPGAN=0 关闭） - ffmpeg rawvideo pipe 编码替代PNG磁盘IO，省~2s - 异步推理架构 + /cancel 端点 + 线程锁并发控制 - 长音频驱动架构，禁止ffmpeg预循环视频（避免16x性能退化） # Conflicts: # deploy/gpu_worker/musetalk_server.py	2026-09-21 00:09:57 +08:00
ying	93cb3e12a0	feat(gpu): add GFPGAN face enhancement + ffmpeg pipe optimization ... - Add GFPGANv1.4 face super-resolution (FP16, +170MB VRAM, ~64ms/frame) - Controllable via MUSE_USE_GFPGAN env var (default enabled) - Enhances generated face region before blending with original frame - Replace PNG disk I/O with ffmpeg stdin pipe encoding (~2s faster) - Frames streamed directly to ffmpeg as raw BGR24, no temp files - Eliminates output_frames_dir/ PNG write/read cycle - Total warm inference: 54s for 5s video (was 56s with PNG, 48s without GFPGAN) - VRAM usage: ~3.3GB with MuseTalk+GFPGAN loaded (RTX 2060 6GB OK)	2026-09-20 22:51:02 +08:00
ying	e7bf85ca86	fix(gpu): #1978 MuseTalk v1.5 推理修复与Worker稳定性补丁 ... musetalk_server.py: - v3 API兼容性：支持新的请求格式（audio_url/video_url字段） - v4 正确pipeline：使用vae.get_latents_for_unet()生成8ch输入 - v4.4 VRAM优化：batch_size=2 + CPU offload + PYTORCH_CUDA_ALLOC_CONF=max_split_size_mb:128 - RTX2060 6GB可稳定运行，22s视频推理约204s（0.11x实时） gpu_worker.py: - 修复健康检查响应格式适配musetalk_server - REQUEST_TIMEOUT提升到900s，与服务端GPU_TASK_TIMEOUT_SECONDS对齐 - TASK_MAX_RETRY降到1，避免与服务端MAX_ATTEMPTS=3相乘放大重试 - 增加TASK_HEARTBEAT_INTERVAL=30s，推理期间独立线程心跳防超时 - 增加MIN_VIDEO_DURATION_SECONDS=3s前置拦截，避免1s视频division by zero	2026-09-20 21:29:31 +08:00
xiaoxia	c9876d70e4	fix(1978): fix xiaoxia-gpu-worker.service bad unit file & log path permissions ... - User=%i -> User=ying (%%i is template-only specifier, not valid here) - After=musetalk.service -> After=musetalk-worker.service (match actual service name) - Log path from /tmp/ to $HOME/ to avoid permission issues for ying user	2026-09-20 13:36:36 +08:00
xiaoxia	db621b4fcb	feat(deploy): #1978 GPU节点自动部署配置文件入库 ... 新增 deploy/gpu_worker/ 下 3 个 systemd 单元 + 3 个运维脚本： - musetalk-worker.service: MuseTalk 推理 API systemd 服务 - gpu-poll.service + gpu-poll.timer: 每30秒轮询Gitea最新commit - scripts/update-gpu-worker.sh: 备份→拉代码→重启→健康检查→失败回滚 - scripts/poll_and_update.sh: SHA比对触发update - scripts/setup-gpu-node.sh: 新节点一键初始化(apt依赖+目录+systemd+sudo免密+首次启动) - README.md 追加第七章「自动部署」说明服务架构/部署步骤/更新机制/日志/注意事项 不改动现有 gpu_worker.py/musetalk_server.py/requirements.txt/xiaoxia-gpu-worker.service/.env.example 不改动CI/Docker/镜像构建。脚本路径写死/home/ying，后续多节点再参数化。	2026-09-20 10:28:24 +08:00
xiaoxia	08de0d9946	perf: async GPU lipsync inference + fix 16x performance regression ... - Move GPU wait_for_result to Celery background task (lipsync_gpu_process_async) POST /lipsync/jobs now returns <1s instead of blocking 200s+ - Rewrite musetalk_server.py: MuseTalk receives full audio directly (v2 architecture) — no pre-looping video before inference Output video length = audio length, mux is fast stream copy - Frontend polls GET /lipsync/jobs/{id} for status updates - refresh_job_status: GPU async path (processing + no mediakit_task_id) skips MediaKit polling; stale jobs (>30min) auto-marked failed - 21 unit tests pass (11 GPU integration + 10 musetalk audio mux) Co-Authored-By: Coze <coze-opensource@bytedance.com>	2026-09-20 09:43:47 +08:00
xiaoxia	b0b81a5d60	fix(1978): MuseTalk 封装强制替换为 TTS 驱动音轨 + 音频长于视频时循环画面 (#1997) ... Co-authored-by: backend-dev <dev@xiaoxiajianji.com> Co-committed-by: backend-dev <dev@xiaoxiajianji.com>	2026-09-20 02:27:44 +08:00
xiaoxia	32473485d7	fix(gpu): #1970 重写 MuseTalk 服务端 + 客户端超时取消，修复 8 项工程 bug ... 服务端新建 deploy/gpu_worker/musetalk_server.py（替代原 ~/projects/MuseTalk/worker.py）： 1. Flask app.run(threaded=True)：推理阻塞时 /health 仍可达 2. _get_video_fps 兜底：ffprobe 返回 0 或失败时 fallback 到 default_fps(25) 3. _run_ffmpeg 统一封装：subprocess.run(check=True) + timeout，失败/超时抛 RuntimeError 4. inference_lock 并发锁：多请求同时到达时第二请求立即 503 5. 推理超时控制：thread.join(timeout=inference_timeout) 默认 600s，超时返回 504 6. finally 块清理临时目录：成功/失败/超时都删除 task_dir 7. 无人脸检测兜底：_run_inference 中帧提取后校验，无帧直接抛错返回 500 8. 上传大小限制：视频 <=100MB / 音频 <=20MB，超限返回 413 9. 新增 POST /cancel 端点：终止当前推理、清理临时文件、释放锁 10. GET /health 返回 GPU 显存信息（nvidia-smi）+ 当前任务状态 客户端 deploy/gpu_worker/gpu_worker.py 配套： - _call_musetalk 超时后 POST /cancel 终止服务端僵尸推理 - _call_musetalk 返回 (ok, duration, err, retryable) 四元组 - _handle_task 仅 retryable=True 时重试，4xx/短视频等确定性失败直接上报 - 新增 _cancel_musetalk_task 辅助方法 测试：新增 15 个单测覆盖服务端全部修复点；全量 15854 passed / 28 skipped 部署提醒：用户需在 RTX2060 上 wget 新 musetalk_server.py 替换旧 worker.py 并重启服务。	2026-09-19 19:35:23 +08:00
xiaoxia	1591259bb8	fix(gpu): #1970 MuseTalk worker 推理期心跳/超时900/重试收敛/短视频前置失败 (#1990) ... Co-authored-by: xiaoxia <dev@xiaoxiajianji.com> Co-committed-by: xiaoxia <dev@xiaoxiajianji.com>	2026-09-19 15:06:51 +08:00
xiaoxia	fbf8844f25	feat(gpu): #1978 MuseTalk GPU Worker 反向轮询对接（后端API + Worker脚本） (#1979) ... Co-authored-by: xiaoxia <dev@xiaoxiajianji.com> Co-committed-by: xiaoxia <dev@xiaoxiajianji.com>	2026-09-18 19:59:49 +08:00