fix(ai-avatar): B-roll时间戳为0根因——逗号分句+优先后端时间戳
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
CI/CD Pipeline / Frontend Lint (pull_request) Failing after 1m27s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 1m33s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m56s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m56s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 2m57s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Successful in 3m0s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 2m55s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 5m50s
CI/CD Pipeline / Unit Tests (pull_request) Failing after 5m56s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 5m59s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 6m11s
AI Code Review / AI Code Review (pull_request) Successful in 6m44s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 11m28s
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Failing after 1s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 8m23s
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 87h26m27s
CI/CD Pipeline / Deploy Production (pull_request) Failing after 87h15m14s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Failing after 87h26m38s
CI/CD Pipeline / Build Staging API Image (pull_request) Failing after 87h26m44s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Failing after 87h26m10s
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Failing after 87h26m14s
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Failing after 87h26m15s
CI/CD Pipeline / Build Staging Worker Image (pull_request) Failing after 87h26m19s
CI/CD Pipeline / Build Staging Web Image (pull_request) Failing after 87h26m20s
CI/CD Pipeline / Check push changed paths (pull_request) Failing after 87h26m27s
CI/CD Pipeline / ACR Image Cleanup (pull_request) Failing after 87h26m2s
CI/CD Pipeline / Canary Release to Production (pull_request) Failing after 87h14m51s
CI/CD Pipeline / Build Production Worker Image (pull_request) Failing after 87h14m55s
CI/CD Pipeline / Build Production Web Image (pull_request) Failing after 87h14m55s
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 87h26m2s
CI/CD Pipeline / Build Production API Image (pull_request) Failing after 87h50m13s
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
CI/CD Pipeline / Frontend Lint (pull_request) Failing after 1m27s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 1m33s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m56s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m56s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 2m57s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Successful in 3m0s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 2m55s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 5m50s
CI/CD Pipeline / Unit Tests (pull_request) Failing after 5m56s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 5m59s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 6m11s
AI Code Review / AI Code Review (pull_request) Successful in 6m44s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 11m28s
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Failing after 1s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 8m23s
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 87h26m27s
CI/CD Pipeline / Deploy Production (pull_request) Failing after 87h15m14s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Failing after 87h26m38s
CI/CD Pipeline / Build Staging API Image (pull_request) Failing after 87h26m44s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Failing after 87h26m10s
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Failing after 87h26m14s
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Failing after 87h26m15s
CI/CD Pipeline / Build Staging Worker Image (pull_request) Failing after 87h26m19s
CI/CD Pipeline / Build Staging Web Image (pull_request) Failing after 87h26m20s
CI/CD Pipeline / Check push changed paths (pull_request) Failing after 87h26m27s
CI/CD Pipeline / ACR Image Cleanup (pull_request) Failing after 87h26m2s
CI/CD Pipeline / Canary Release to Production (pull_request) Failing after 87h14m51s
CI/CD Pipeline / Build Production Worker Image (pull_request) Failing after 87h14m55s
CI/CD Pipeline / Build Production Web Image (pull_request) Failing after 87h14m55s
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 87h26m2s
CI/CD Pipeline / Build Production API Image (pull_request) Failing after 87h50m13s
根因(用户截图文案:'卖花的叫花无缺,卖姜的叫姜子牙,卖菜的蔡文姬,那我修脚阔头的呢,应该叫什么呢'):
1. 分句正则没包含中文逗号「,」,整段被识别为1句;后端静音检测按TTS音频停顿
切出多句,前端校验 sentenceTimings.length===rawParts.length 条数不等 →
整个后端精确时间戳被丢弃走降级
2. 降级路径 outputDuration=0(对口型预览阶段最终视频未渲染)→ 所有句子时间估算为0
修复:
- 前端 sentences.ts:
a. 分句正则加上中英文逗号「,,」
b. sentenceTimings 校验放宽:只要是有效数组就直接用后端句子列表,
不再强制条数相等(后端按音频停顿的切法才是真实边界)
- 后端 _split_script_into_sentences:正则同步加逗号,前后端一致
- 补单测验证逗号分隔文案分句
This commit is contained in:
@@ -55,13 +55,17 @@ def _sign_media_url(url: str) -> str:
|
||||
|
||||
|
||||
def _split_script_into_sentences(script_text: str) -> list[str]:
|
||||
"""按句号/问号/感叹号/分号/换行分句(与前端 splitScriptIntoSentences 一致)."""
|
||||
"""按句号/问号/感叹号/分号/逗号/换行分句(与前端 SENTENCE_SPLIT_RE 一致).
|
||||
|
||||
中文短视频文案习惯用「,」断小句(如"卖花的叫花无缺,卖姜的叫姜子牙"),
|
||||
必须把逗号也纳入分隔符,否则多句文案会被识别成一整句,导致 B-roll 时间戳错位。
|
||||
"""
|
||||
import re
|
||||
|
||||
text = (script_text or "").strip()
|
||||
if not text:
|
||||
return []
|
||||
parts = re.split(r"[。!?!?;;\n\r]+", text)
|
||||
parts = re.split(r"[。!?!??!;;,,\n\r]+", text)
|
||||
return [p.strip() for p in parts if p.strip()]
|
||||
|
||||
|
||||
|
||||
@@ -1,8 +1,10 @@
|
||||
/**
|
||||
* AI数字人 — 文案分句 & B-roll 时间计算
|
||||
*
|
||||
* 优先使用后端基于 TTS 音频静音检测计算的精确 sentence_timings;
|
||||
* 后端未返回(如对口型还在生成中)时,降级为前端按字数比例估算。
|
||||
* 数据来源优先级:
|
||||
* 1. 后端 sentence_timings(基于 TTS 音频静音检测,精确到句子边界)—— 直接使用,不重新分句
|
||||
* 2. 后端 output_duration(最终渲染视频时长) + 本地分句 —— 按字数比例估算
|
||||
* 3. 两者都没有(对口型还在生成中)—— 返回分句文本但 startTime/endTime 全部 0,等数据到位重算
|
||||
*/
|
||||
|
||||
export interface ScriptSentence {
|
||||
@@ -20,45 +22,46 @@ export interface ScriptSentence {
|
||||
endTime: number
|
||||
}
|
||||
|
||||
/** 句子分隔符:中英文句号/问号/感叹号/分号/逗号/换行(覆盖中文短视频常用断句) */
|
||||
const SENTENCE_SPLIT_RE = /[。!?!??!;;,,\n\r]+/
|
||||
|
||||
/**
|
||||
* 按句号/问号/感叹号/分号/换行分句(兼容中英文标点)。
|
||||
* 空文案返回空数组。时间优先使用后端 sentence_timings;否则按字数线性估算。
|
||||
* 分句并计算每句的起止时间。
|
||||
*
|
||||
* @param sentenceTimings 后端返回的精确句子时间戳(来自 lipsync_job.sentence_timings)。
|
||||
* 非空且有效时优先采用,跳过前端估算。
|
||||
* 非空时直接按后端返回的句子列表渲染,不再本地分句(避免前后端分句不一致导致时间错位)。
|
||||
* @param outputDuration 最终视频时长(秒)。对口型预览阶段可能为 0,此时降级估算只能给 0。
|
||||
*/
|
||||
export function splitScriptIntoSentences(
|
||||
scriptText: string,
|
||||
sentenceTimings?: { index: number; text: string; start_time: number; end_time: number }[] | null,
|
||||
sentenceTimings?: { index?: number; text?: string; start_time: number; end_time: number }[] | null,
|
||||
outputDuration: number = 0,
|
||||
): ScriptSentence[] {
|
||||
const text = (scriptText || "").trim()
|
||||
if (!text) return []
|
||||
|
||||
// 1. 先做基础分句(仅用于降级估算 / 没有 sentenceTimings 时)
|
||||
const rawParts = text
|
||||
.split(/[。!?!?;;\n\r]+/)
|
||||
.map((part) => part.trim())
|
||||
.filter((part) => part.length > 0)
|
||||
|
||||
// 2. 优先使用后端精确时间戳
|
||||
// 校验:必须是数组、条数一致、每条都有 start_time/end_time,否则降级估算
|
||||
if (Array.isArray(sentenceTimings) && sentenceTimings.length === rawParts.length) {
|
||||
// 1. 后端返回了 sentence_timings:校验通过就直接用,跳过本地分句
|
||||
// 校验条件放宽:只要是数组、至少1条、每条 start_time/end_time 是数字即可
|
||||
// (不再强制要求条数相等——后端静音检测可能按停顿切出更多/更少边界,
|
||||
// 比如文案用逗号连写时本地只分1句、后端按停顿切4句,后端的切法才是对的)
|
||||
if (Array.isArray(sentenceTimings) && sentenceTimings.length > 0) {
|
||||
const valid = sentenceTimings.every(
|
||||
(t) =>
|
||||
t &&
|
||||
typeof t.start_time === "number" &&
|
||||
typeof t.end_time === "number" &&
|
||||
isFinite(t.start_time) &&
|
||||
isFinite(t.end_time) &&
|
||||
t.end_time >= t.start_time,
|
||||
)
|
||||
if (valid) {
|
||||
let accChar = 0
|
||||
return sentenceTimings.map((t, i) => {
|
||||
const part = rawParts[i] ?? t.text ?? ""
|
||||
const charCount = part.replace(/\s/g, "").length
|
||||
const sentenceText = (t.text || "").trim() || `句子${i + 1}`
|
||||
const charCount = sentenceText.replace(/\s/g, "").length
|
||||
const sentence: ScriptSentence = {
|
||||
index: t.index ?? i,
|
||||
text: part,
|
||||
index: typeof t.index === "number" ? t.index : i,
|
||||
text: sentenceText,
|
||||
charCount,
|
||||
startChar: accChar,
|
||||
startTime: round1(t.start_time),
|
||||
@@ -70,7 +73,12 @@ export function splitScriptIntoSentences(
|
||||
}
|
||||
}
|
||||
|
||||
// 3. 降级:按字数比例线性估算
|
||||
// 2. 本地分句 + 按字数比例估算(降级路径)
|
||||
const rawParts = text
|
||||
.split(SENTENCE_SPLIT_RE)
|
||||
.map((part) => part.trim())
|
||||
.filter((part) => part.length > 0)
|
||||
|
||||
const totalChars = rawParts.reduce((sum, part) => sum + part.replace(/\s/g, "").length, 0)
|
||||
const duration = outputDuration > 0 ? outputDuration : 0
|
||||
|
||||
|
||||
Reference in New Issue
Block a user