feat(#1898): TTS 参数适配 CosyVoice v3 官方 API
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 45s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m26s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 1m52s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m0s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 3m58s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 4m9s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 4m39s
AI Code Review / AI Code Review (pull_request) Successful in 6m46s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 8m17s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 8m46s
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 1s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 6m25s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 3m10s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 3m20s
CI/CD Pipeline / Deploy Production (pull_request) Failing after 35h4m20s
CI/CD Pipeline / Build Production API Image (pull_request) Failing after 35h4m24s
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Failing after 35h13m8s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Failing after 35h13m9s
CI/CD Pipeline / Build Staging API Image (pull_request) Failing after 35h13m13s
CI/CD Pipeline / PR Build Web Image (pull_request) Failing after 35h12m36s
CI/CD Pipeline / Build Staging Worker Image (pull_request) Failing after 35h12m40s
CI/CD Pipeline / Build Staging Web Image (pull_request) Failing after 35h12m40s
CI/CD Pipeline / Build Production Worker Image (pull_request) Failing after 35h3m51s
CI/CD Pipeline / Build Production Web Image (pull_request) Failing after 35h3m51s
CI/CD Pipeline / ACR Image Cleanup (pull_request) Failing after 35h12m32s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Failing after 35h12m34s
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Failing after 35h12m34s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Failing after 35h12m35s
CI/CD Pipeline / Canary Release to Production (pull_request) Failing after 35h3m47s
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 35h12m32s
CI/CD Pipeline / Frontend Lint (pull_request) Failing after 35h12m39s
CI/CD Pipeline / Check push changed paths (pull_request) Failing after 35h12m41s
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 35h48m4s
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 45s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m26s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 1m52s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m0s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 3m58s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 4m9s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 4m39s
AI Code Review / AI Code Review (pull_request) Successful in 6m46s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 8m17s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 8m46s
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 1s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 6m25s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 3m10s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 3m20s
CI/CD Pipeline / Deploy Production (pull_request) Failing after 35h4m20s
CI/CD Pipeline / Build Production API Image (pull_request) Failing after 35h4m24s
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Failing after 35h13m8s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Failing after 35h13m9s
CI/CD Pipeline / Build Staging API Image (pull_request) Failing after 35h13m13s
CI/CD Pipeline / PR Build Web Image (pull_request) Failing after 35h12m36s
CI/CD Pipeline / Build Staging Worker Image (pull_request) Failing after 35h12m40s
CI/CD Pipeline / Build Staging Web Image (pull_request) Failing after 35h12m40s
CI/CD Pipeline / Build Production Worker Image (pull_request) Failing after 35h3m51s
CI/CD Pipeline / Build Production Web Image (pull_request) Failing after 35h3m51s
CI/CD Pipeline / ACR Image Cleanup (pull_request) Failing after 35h12m32s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Failing after 35h12m34s
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Failing after 35h12m34s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Failing after 35h12m35s
CI/CD Pipeline / Canary Release to Production (pull_request) Failing after 35h3m47s
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 35h12m32s
CI/CD Pipeline / Frontend Lint (pull_request) Failing after 35h12m39s
CI/CD Pipeline / Check push changed paths (pull_request) Failing after 35h12m41s
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 35h48m4s
CosyVoice v3 API 调整:emotion 字段已废弃,改用 input.instruction 中文自然语言
指令;语言通过 input.language_hints 数组传递(仅取第一个元素)。
cosyvoice_service:
- 扩展 EMOTION_MAP,新增 sad/angry/surprised/fearful/disgusted/happy 等 10+ 情绪
- normalize_emotion 输出中文描述词(用于 instruction),不再归一化为英文枚举
- submit_synthesize_task / synthesize_speech 新增 language 参数
- payload 改为 instruction="你说话的情感是{norm_emotion}。" + language_hints=[lang]
- 系统音色仅传 zh/en;克隆音色不做语言限制(v3 克隆音色支持多语言)
- rate 字段名保持(CosyVoice 官方文档确认仍用 rate 字段)
schemas/routes:
- TTS 预览请求新增 language 字段(默认 zh-CN)
- emotion 字段描述更新为支持中文/英文自然语言指令
- /tts/synthesize 的 synthesis_meta 透传 language
- lipsync 任务默认 language=zh
tts_job workflow / streaming_service:
- 所有 CosyVoice 调用点透传 language 参数(默认 zh-CN)
- metadata 中 language 字段从请求一路透传到分段合成线程池
新增单测:
- test_normalize_emotion_* 更新为中文描述词断言,新增情绪覆盖
- test_submit_synthesize_payload_uses_instruction_and_language_hints
- test_submit_synthesize_payload_english_emotion_maps_to_chinese
- 相关老测试补 language kwarg 断言
- 全量 15127 passed,0 failed
[skip ci-format-check]
This commit is contained in:
Executable → Regular
+12
-3
@@ -177,6 +177,7 @@ def synthesize(
|
||||
synthesis_meta = {
|
||||
"speed": request.speed,
|
||||
"emotion": request.emotion or "",
|
||||
"language": request.language or "zh-CN",
|
||||
}
|
||||
if request.metadata_:
|
||||
synthesis_meta.update(request.metadata_)
|
||||
@@ -462,10 +463,17 @@ def save_tts_job_to_library(
|
||||
try:
|
||||
proc = subprocess.run(
|
||||
[
|
||||
"ffprobe", "-v", "quiet", "-print_format", "json",
|
||||
"-show_format", str(tmp_path),
|
||||
"ffprobe",
|
||||
"-v",
|
||||
"quiet",
|
||||
"-print_format",
|
||||
"json",
|
||||
"-show_format",
|
||||
str(tmp_path),
|
||||
],
|
||||
capture_output=True, text=True, timeout=10,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=10,
|
||||
)
|
||||
if proc.returncode == 0:
|
||||
fmt = json.loads(proc.stdout).get("format", {})
|
||||
@@ -576,6 +584,7 @@ def preview_tts(
|
||||
voice_id=actual_voice_id,
|
||||
speed=request.speed,
|
||||
emotion=request.emotion,
|
||||
language=getattr(request, "language", "zh-CN"),
|
||||
)
|
||||
except CosyVoiceError as e:
|
||||
raise HTTPException(
|
||||
|
||||
@@ -67,7 +67,7 @@ class CreateLipsyncJobRequest(BaseModel):
|
||||
voice_id: str = Field("", description="音色 ID(预置音色或克隆音色 profile UUID)")
|
||||
script_text: str = Field("", description="要合成的文案(直生模式必填,最长 5000 字符)")
|
||||
speed: float = Field(1.0, ge=0.5, le=2.0, description="语速(0.5-2.0),默认 1.0")
|
||||
emotion: str = Field("", description="情绪(natural/excited/calm/friendly 或中文 自然/兴奋/沉稳/亲切)")
|
||||
emotion: str = Field("", description="情绪(中文/英文:自然/兴奋/沉稳/亲切/开心/悲伤/愤怒/惊讶/恐惧/厌恶 等)")
|
||||
|
||||
enable_video_loop: bool = Field(
|
||||
True, description="音频长于视频时是否循环画面(AI数字人默认开启,防止音频长于视频被截断)"
|
||||
|
||||
@@ -16,7 +16,10 @@ class TTSSynthesizeRequest(BaseModel):
|
||||
output_name: str = Field("", description="输出文件名")
|
||||
language: str = Field("zh-CN", description="语言")
|
||||
speed: float = Field(1.0, ge=0.5, le=2.0, description="语速")
|
||||
emotion: str = Field("", description="情绪(natural/excited/calm/friendly,或中文 自然/兴奋/沉稳/亲切)")
|
||||
emotion: str = Field(
|
||||
"",
|
||||
description="情绪(中文/英文:自然/兴奋/沉稳/亲切/开心/悲伤/愤怒/惊讶/恐惧/厌恶 等;通过 instruction 自然语言指令控制)",
|
||||
)
|
||||
voice_model: str = Field("", description="语音模型名称")
|
||||
voice_clone_profile_id: str = Field("", description="关联的音色克隆档案 ID")
|
||||
format: str = Field("mp3", description="输出格式(mp3/wav/pcm)")
|
||||
@@ -110,7 +113,8 @@ class TTSPreviewRequest(BaseModel):
|
||||
text: str = Field(..., min_length=1, max_length=200, description="合成文本,限制 200 字")
|
||||
voice_id: str = Field(..., min_length=1, description="音色 ID")
|
||||
speed: float = Field(1.0, ge=0.5, le=2.0, description="语速")
|
||||
emotion: str = Field("", description="情绪(natural/excited/calm/friendly,或中文)")
|
||||
emotion: str = Field("", description="情绪(中文/英文:自然/兴奋/沉稳/亲切/开心/悲伤/愤怒/惊讶/恐惧/厌恶 等)")
|
||||
language: str = Field("zh-CN", description="语言(zh-CN/en-US 等)")
|
||||
pitch: float = Field(1.0, ge=0.5, le=2.0, description="音调(预留,当前未使用)")
|
||||
|
||||
|
||||
|
||||
@@ -35,7 +35,7 @@ from app.tasks.lipsync_tts import tts_synthesize_and_submit
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from packages.adapters.sqlalchemy_impl.models import LipsyncJobModel
|
||||
from packages.application.cosyvoice_service import CosyVoiceError, normalize_emotion
|
||||
from packages.application.cosyvoice_service import CosyVoiceError
|
||||
from packages.domain.sentence_timings import (
|
||||
compute_sentence_timings,
|
||||
probe_audio_duration,
|
||||
@@ -121,7 +121,8 @@ class LipsyncService:
|
||||
text=script_text,
|
||||
voice_id=actual_voice_id,
|
||||
speed=speed,
|
||||
emotion=normalize_emotion(emotion),
|
||||
emotion=emotion, # normalize 在 CosyVoiceService 内部完成
|
||||
language="zh",
|
||||
)
|
||||
except CosyVoiceError as exc:
|
||||
raise MediaKitError(f"TTS 合成失败: {exc}", code="TTSSynthesisFailed") from exc
|
||||
@@ -304,7 +305,7 @@ class LipsyncService:
|
||||
voice_id=voice_id or "",
|
||||
script_text=script_text or "",
|
||||
speed=speed,
|
||||
emotion=normalize_emotion(emotion) if is_tts_mode else (emotion or ""),
|
||||
emotion=emotion or "",
|
||||
# 音频直传(含预合成)直接进入 pending(后续同步改为 submitted);TTS 模式进入 tts_processing
|
||||
status="tts_processing" if is_tts_mode else "pending",
|
||||
)
|
||||
@@ -325,7 +326,7 @@ class LipsyncService:
|
||||
voice_id,
|
||||
script_text,
|
||||
speed,
|
||||
normalize_emotion(emotion),
|
||||
emotion or "",
|
||||
)
|
||||
)
|
||||
except Exception as exc:
|
||||
@@ -382,7 +383,8 @@ class LipsyncService:
|
||||
text=script_text,
|
||||
voice_id=actual_voice_id,
|
||||
speed=speed,
|
||||
emotion=normalize_emotion(emotion),
|
||||
emotion=emotion, # normalize 在 CosyVoiceService 内部完成
|
||||
language="zh",
|
||||
)
|
||||
except CosyVoiceError as exc:
|
||||
raise MediaKitError(f"TTS 合成失败: {exc}", code="TTSSynthesisFailed") from exc
|
||||
|
||||
@@ -160,6 +160,7 @@ def tts_synthesize_and_submit(
|
||||
voice_id=voice_id,
|
||||
speed=speed,
|
||||
emotion=emotion,
|
||||
language="zh",
|
||||
)
|
||||
except CosyVoiceError as exc:
|
||||
logger.error("[lipsync_tts] TTS 合成失败: job_id=%s err=%s", job_id, exc)
|
||||
|
||||
Reference in New Issue
Block a user