fix(ai-avatar): 修复B-roll文案时长全为0.0s的问题
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 1m59s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 1m55s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 2m0s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m7s
CI/CD Pipeline / Validate - Style (pull_request) Failing after 2m43s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 2m24s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 2m25s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 2m42s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 2m52s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Successful in 3m9s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m34s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 5m44s
CI/CD Pipeline / CI Gate (pull_request) Failing after 0s
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 2m18s
AI Code Review / AI Code Review (pull_request) Successful in 6m23s
CI/CD Pipeline / Build Production Worker Image (pull_request) Failing after 98h44m2s
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Failing after 98h49m41s
CI/CD Pipeline / Build Staging Web Image (pull_request) Failing after 98h49m48s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Failing after 98h49m15s
CI/CD Pipeline / ACR Image Cleanup (pull_request) Failing after 98h49m11s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Failing after 98h49m21s
CI/CD Pipeline / Build Staging API Image (pull_request) Failing after 98h49m28s
CI/CD Pipeline / Build Staging Worker Image (pull_request) Failing after 98h49m27s
CI/CD Pipeline / Deploy Production (pull_request) Failing after 98h43m39s
CI/CD Pipeline / Build Production API Image (pull_request) Failing after 98h43m43s
CI/CD Pipeline / Canary Release to Production (pull_request) Failing after 98h43m39s
CI/CD Pipeline / Build Production Web Image (pull_request) Failing after 98h43m42s
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 98h49m11s
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Failing after 98h49m20s
CI/CD Pipeline / Check push changed paths (pull_request) Failing after 98h49m31s
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 99h24m25s
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 1m59s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 1m55s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 2m0s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m7s
CI/CD Pipeline / Validate - Style (pull_request) Failing after 2m43s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 2m24s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 2m25s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 2m42s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 2m52s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Successful in 3m9s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m34s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 5m44s
CI/CD Pipeline / CI Gate (pull_request) Failing after 0s
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 2m18s
AI Code Review / AI Code Review (pull_request) Successful in 6m23s
CI/CD Pipeline / Build Production Worker Image (pull_request) Failing after 98h44m2s
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Failing after 98h49m41s
CI/CD Pipeline / Build Staging Web Image (pull_request) Failing after 98h49m48s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Failing after 98h49m15s
CI/CD Pipeline / ACR Image Cleanup (pull_request) Failing after 98h49m11s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Failing after 98h49m21s
CI/CD Pipeline / Build Staging API Image (pull_request) Failing after 98h49m28s
CI/CD Pipeline / Build Staging Worker Image (pull_request) Failing after 98h49m27s
CI/CD Pipeline / Deploy Production (pull_request) Failing after 98h43m39s
CI/CD Pipeline / Build Production API Image (pull_request) Failing after 98h43m43s
CI/CD Pipeline / Canary Release to Production (pull_request) Failing after 98h43m39s
CI/CD Pipeline / Build Production Web Image (pull_request) Failing after 98h43m42s
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 98h49m11s
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Failing after 98h49m20s
CI/CD Pipeline / Check push changed paths (pull_request) Failing after 98h49m31s
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 99h24m25s
根因: TTS任务步骤2.5重新从OSS下载音频做静音检测,但OSS私有桶URL未签名导致下载失败, 异常被静默吞掉,sentence_timings 永远为 null。前端拿到null后降级估算, 但outputDuration依赖MediaKit返回的视频时长(对口型完成前为0),导致时间全是0。 修复: - 后端: 直接复用步骤2已下载到内存的audio_data,不再重新下载OSS - 后端: 增加详细日志,方便排查静音检测异常 - 后端: 增加空数据防御检查 - 前端: hasBackendTimings判断增加长度匹配检查,防御异常数据
This commit is contained in:
@@ -334,6 +334,7 @@ def tts_synthesize_and_submit(
|
||||
db.commit()
|
||||
|
||||
# 2.5 计算精确句子时间戳(基于 TTS 音频静音检测)
|
||||
# 直接复用步骤 2 已下载到内存的 audio_data,避免重新从 OSS 下载(私有桶未签名会失败)
|
||||
import os as _os
|
||||
|
||||
_st_tmp_path = None
|
||||
@@ -341,49 +342,61 @@ def tts_synthesize_and_submit(
|
||||
import subprocess as _sp
|
||||
import tempfile as _tmpf
|
||||
|
||||
# 下载音频用于探测时长和静音检测
|
||||
if isinstance(job.audio_url, str) and job.audio_url:
|
||||
from packages.shared.url_security import safe_download_bytes as _sdl
|
||||
|
||||
_audio_bytes = _sdl(job.audio_url, purpose="sentence_timings", timeout=30.0)
|
||||
if not audio_data:
|
||||
logger.warning("[lipsync_tts] 无音频数据,跳过句子时间戳计算: job_id=%s", job_id)
|
||||
else:
|
||||
_audio_bytes = audio_data
|
||||
# 写入临时文件供 ffprobe/ffmpeg 使用
|
||||
with _tmpf.NamedTemporaryFile(suffix=".mp3", delete=False) as _atmp:
|
||||
_atmp.write(audio_data)
|
||||
_st_tmp_path = _atmp.name
|
||||
|
||||
# ffprobe 获取音频时长
|
||||
with _tmpf.NamedTemporaryFile(suffix=".mp3", delete=False) as _atmp:
|
||||
_atmp.write(_audio_bytes)
|
||||
_st_tmp_path = _atmp.name
|
||||
# ffprobe 获取音频时长
|
||||
_probe_result = _sp.run(
|
||||
[
|
||||
"ffprobe",
|
||||
"-v",
|
||||
"error",
|
||||
"-show_entries",
|
||||
"format=duration",
|
||||
"-of",
|
||||
"default=noprint_wrappers=1:nokey=1",
|
||||
_st_tmp_path,
|
||||
],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=10,
|
||||
)
|
||||
_audio_duration = float(_probe_result.stdout.strip()) if _probe_result.stdout.strip() else 0.0
|
||||
logger.info(
|
||||
"[lipsync_tts] 音频时长探测: job_id=%s duration=%.2f probe_stdout=%s probe_stderr=%s",
|
||||
job_id,
|
||||
_audio_duration,
|
||||
_probe_result.stdout.strip()[:50],
|
||||
_probe_result.stderr.strip()[:100] if _probe_result.stderr else "",
|
||||
)
|
||||
|
||||
_probe_result = _sp.run(
|
||||
[
|
||||
"ffprobe",
|
||||
"-v",
|
||||
"error",
|
||||
"-show_entries",
|
||||
"format=duration",
|
||||
"-of",
|
||||
"default=noprint_wrappers=1:nokey=1",
|
||||
_st_tmp_path,
|
||||
],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=10,
|
||||
)
|
||||
_audio_duration = float(_probe_result.stdout.strip()) if _probe_result.stdout.strip() else 0.0
|
||||
|
||||
if _audio_duration > 0:
|
||||
_timings = _compute_sentence_timings(_audio_bytes, script_text, _audio_duration)
|
||||
if _timings:
|
||||
job.sentence_timings = _timings
|
||||
logger.info(
|
||||
"[lipsync_tts] 句子时间戳已计算: job_id=%s sentences=%d duration=%.1f",
|
||||
if _audio_duration > 0:
|
||||
_timings = _compute_sentence_timings(audio_data, script_text, _audio_duration)
|
||||
if _timings:
|
||||
job.sentence_timings = _timings
|
||||
logger.info(
|
||||
"[lipsync_tts] 句子时间戳已计算: job_id=%s sentences=%d duration=%.1f",
|
||||
job_id,
|
||||
len(_timings),
|
||||
_audio_duration,
|
||||
)
|
||||
else:
|
||||
logger.warning("[lipsync_tts] 句子时间戳计算返回空结果: job_id=%s", job_id)
|
||||
else:
|
||||
logger.warning(
|
||||
"[lipsync_tts] ffprobe 未获取到有效时长,跳过句子时间戳: job_id=%s stdout=%s stderr=%s",
|
||||
job_id,
|
||||
len(_timings),
|
||||
_audio_duration,
|
||||
_probe_result.stdout.strip()[:100],
|
||||
_probe_result.stderr.strip()[:200] if _probe_result.stderr else "",
|
||||
)
|
||||
db.commit()
|
||||
db.commit()
|
||||
except Exception as _st_err:
|
||||
logger.warning("[lipsync_tts] 句子时间戳计算失败(不影响主流程): job_id=%s err=%s", job_id, _st_err)
|
||||
logger.warning("[lipsync_tts] 句子时间戳计算失败(不影响主流程): job_id=%s err=%s", job_id, _st_err, exc_info=True)
|
||||
finally:
|
||||
if _st_tmp_path:
|
||||
try:
|
||||
|
||||
@@ -42,9 +42,10 @@ export function splitScriptIntoSentences(
|
||||
let accChar = 0
|
||||
const totalChars = rawParts.reduce((sum, p) => sum + p.replace(/\s/g, "").length, 0)
|
||||
// 如果后端返回了完整的时间戳(至少有一个有效结束时间),使用精确时间
|
||||
// 需要满足:数组长度与分句数一致,且至少有一个结束时间 > 0(防御全0的异常数据)
|
||||
const hasBackendTimings =
|
||||
sentenceTimings &&
|
||||
sentenceTimings.length > 0 &&
|
||||
sentenceTimings.length >= rawParts.length &&
|
||||
sentenceTimings.some((t) => (t.end_time ?? 0) > 0)
|
||||
|
||||
// 若有后端时间戳,取音频总时长;否则用外部传入的 outputDuration 做字数比例降级
|
||||
|
||||
Reference in New Issue
Block a user