xiaoxia/xiaoxia-saas

fix(1978): MuseTalk 封装强制替换为 TTS 驱动音轨 + 音频长于视频时循环画面 #1997

Merged

auto-approve-bot merged 2 commits from fix/1978-musetalk-audio-mux-loop into develop

2026-09-20 02:27:45 +08:00

Author	SHA1	Message	Date
xiaoxia	bd5b4025a3	Merge remote-tracking branch 'origin/develop' into fix/1978-musetalk-audio-mux-loop CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped Details CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s Details CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped Details CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 1s Details CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped Details CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped Details CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped Details CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 39s Details CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped Details CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped Details CI/CD Pipeline / PR Build API Image (pull_request) Successful in 42s Details CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped Details CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped Details CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped Details CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped Details Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m15s Details CI/CD Pipeline / Integration Tests (pull_request) Successful in 2m39s Details PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m51s Details CI/CD Pipeline / Unit Tests (pull_request) Successful in 3m43s Details CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 4m48s Details CI/CD Pipeline / Validate - Style (pull_request) Successful in 5m16s Details AI Code Review / AI Code Review (pull_request) Successful in 6m28s Details CI/CD Pipeline / Validate - Security (pull_request) Successful in 8m53s Details CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped Details CI/CD Pipeline / CI Gate (pull_request) Successful in 1s Details CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped Details CI/CD Pipeline / Deploy Production (pull_request) Has been skipped Details CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped Details PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 6m46s Details ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 28s Details Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 3m4s Details	2026-09-20 02:15:01 +08:00
xiaoxia	54f9fe0dc1	fix(1978): MuseTalk 封装强制用 TTS 驱动音频替换音轨 + 音频长于视频时循环画面 CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped Details CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 1s Details CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s Details CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped Details CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped Details CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped Details CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped Details CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped Details CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped Details CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped Details CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped Details CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped Details CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 38s Details CI/CD Pipeline / PR Build API Image (pull_request) Successful in 40s Details Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m19s Details PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m54s Details CI/CD Pipeline / Integration Tests (pull_request) Successful in 2m56s Details CI/CD Pipeline / Validate - Style (pull_request) Successful in 3m57s Details AI Code Review / AI Code Review (pull_request) Successful in 6m36s Details CI/CD Pipeline / Validate - Security (pull_request) Successful in 7m53s Details PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 10m43s Details CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 19m33s Details CI/CD Pipeline / Unit Tests (pull_request) Failing after 20m27s Details CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped Details CI/CD Pipeline / CI Gate (pull_request) Failing after 1s Details CI/CD Pipeline / Deploy Production (pull_request) Has been skipped Details CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped Details CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped Details 严重 bug：结果视频保留了源视频音轨（波形比对相关系数 0.9998，与 TTS 驱动音频无关），口型同步配的是原声。根因是最终 ffmpeg 封装没写 -map，默认流选择把第一个输入（源视频/无声画面）的自带音轨带进了结果。修复（deploy/gpu_worker/musetalk_server.py）： - 新增 _mux_video_with_audio() 统一封装：强制 -map 0:v:0 -map 1:a:0，画面只取 MuseTalk 无声产物，音轨只取第二个输入的驱动音频，AAC 128k - 新增 _get_media_duration() 探测音视频时长 - 音频不超过视频（容差 0.25s）：-c:v copy -shortest 无损秒封装 - 音频长于视频（TTS 15s vs 视频 9s 常见）：-stream_loop -1 循环画面， h264_nvenc 硬件重编码（RTX2060），-t 精确卡到音频时长，不截短语音 - _pick_video_encoder()：MUSE_VIDEO_ENCODER=auto 探测 NVENC，不可用或编码失败自动回退 libx264(veryfast)；preset 随编码器选择 - MUSE_ENABLE_VIDEO_LOOP（默认开）/ 请求 form enable_video_loop 单任务覆盖 - 示例推理先产出无声 visual_silent.mp4（-an）再走统一 mux，真实 MuseTalk 接入时直接把无声产物路径传给 _mux_video_with_audio - README：新增 2.2 更新部署（wget 拉新文件+systemctl 重启）、环境变量表、工程记录 #9/#10 测试 test_1978_musetalk_audio_mux.py（10 用例）： - 命令构造：非循环 copy+map、容差边界、循环 stream_loop/NVENC/-t 无 -shortest、关循环、NVENC 失败回退 libx264（p4→veryfast）、copy 失败抛出、编码器探测 - 真实 ffmpeg 端到端（环境无 ffmpeg 自动 skip）：源视频内置 200Hz 音轨+ 驱动音频 800Hz，过零率验证结果音轨主频 800Hz（非源轨），2s 画面+5s 音频循环输出时长≈5s；短音频路径输出≈1s 且音轨 800Hz	2026-09-20 01:47:36 +08:00