feat(tts): 正式合成支持克隆音色、保存到配音库改写 assets 素材体系、克隆接口支持 asset_id
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 0s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 3m0s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 3m14s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 3m20s
CI/CD Pipeline / Validate - Migration (alembic) (pull_request) Successful in 3m35s
CI/CD Pipeline / Validate - Type Check (mypy) (pull_request) Successful in 3m49s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 1m10s
AI Code Review / AI Code Review (pull_request) Failing after 4m17s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 1m12s
CI/CD Pipeline / Validate - Code Quality (pull_request) Successful in 6m2s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 6m4s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 2m5s
CI/CD Pipeline / Unit Tests (pull_request) Failing after 8m35s
CI/CD Pipeline / CI Gate (pull_request) Failing after 23s
CI/CD Pipeline / Production Browser E2E (pull_request) Failing after 407h0m37s
CI/CD Pipeline / Build Production Web Image (pull_request) Failing after 407h0m39s
CI/CD Pipeline / Build Production API Image (pull_request) Failing after 407h0m39s
CI/CD Pipeline / Canary Release to Production (pull_request) Failing after 407h0m37s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Failing after 407h9m15s
CI/CD Pipeline / Deploy Production (pull_request) Failing after 407h0m37s
CI/CD Pipeline / Frontend Lint (pull_request) Failing after 407h9m15s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Failing after 407h12m12s
CI/CD Pipeline / Build Staging Worker Image (pull_request) Failing after 407h12m14s
CI/CD Pipeline / Build Staging API Image (pull_request) Failing after 407h12m15s
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 407h12m9s
CI/CD Pipeline / Check push changed paths (pull_request) Failing after 407h12m17s
CI/CD Pipeline / ACR Image Cleanup (pull_request) Failing after 407h12m9s
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 407h12m10s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Failing after 407h12m11s
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Failing after 407h12m11s
CI/CD Pipeline / Build Production Worker Image (pull_request) Failing after 407h34m47s
CI/CD Pipeline / PR Build Web Image (pull_request) Failing after 407h43m21s
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Failing after 407h46m19s
CI/CD Pipeline / Build Staging Web Image (pull_request) Failing after 407h46m23s

1. TTS synthesize() 复用 preview 的克隆音色解析逻辑:voice_id 传克隆 profile UUID 时
   校验归属(403)/克隆完成状态(400 音色克隆尚未完成),job.voice_id 存解析后的
   CosyVoice voice_id,voice_clone_profile_id 记录 profile id。
2. save_tts_job_to_library 改写为新素材体系:TTS 音频持久化独立副本到
   uploads/voice/tts/,创建 file_type=audio/status=ready 的 asset 挂用户 voice
   素材库(无则自动创建),duration/metadata(source=tts_job) 完整记录;
   返回结构保持 id/name/audio_url/duration 兼容前端;移除旧 voice_library 表写入。
3. voice clone 创建接口支持 asset_id 字段:校验素材归属与音频类型,用 asset
   storage_key 生成 7 天有效签名 URL 供 worker 下载,不依赖前端签名;
   source_audio_url 方式保留兼容。

补集成测试:synthesize 克隆音色 4 用例、save-to-library asset 断言、
clone from asset 5 用例(成功/404/403/非音频400/两参同传400)。
This commit is contained in:
xiaoxia
2026-08-30 16:12:57 +08:00
parent 4f7746926c
commit 539c71f5cb
5 changed files with 581 additions and 76 deletions
+162 -60
View File
@@ -7,13 +7,15 @@ from typing import Optional
from app.auth import AuthenticatedUser, get_current_user
from app.core.celery_app import celery_app
from app.core.storage import get_storage_service
from app.dependencies import (
get_asset_library_repository,
get_asset_repository,
get_audio_url_signer,
get_cosyvoice_service,
get_db_session,
get_user_repository,
get_project_repository,
get_voice_clone_profile_repository,
get_voice_library_repository,
)
from app.schemas.tts import (
ListTTSJobResponse,
@@ -32,7 +34,6 @@ from sqlalchemy.orm import Session
from packages.adapters.sqlalchemy_impl.tts_job_repository import (
SQLAlchemyTTSJobRepository,
)
from packages.adapters.sqlalchemy_impl.voice_library_repository import SQLAlchemyVoiceLibraryRepository
from packages.application.cosyvoice_service import CosyVoiceError, CosyVoiceService
from packages.application.tts_job.streaming_service import TTSStreamingService
from packages.application.tts_job.use_cases import (
@@ -44,13 +45,12 @@ from packages.application.tts_job.use_cases import (
TTSJobNotFoundError,
)
from packages.application.tts_job.workflow import TTSWorkflowService
from packages.application.voice_library.commands import CreateVoiceLibraryCommand
from packages.application.voice_library.use_cases import (
CreateVoiceLibraryUseCase,
QuotaExceededError,
)
from packages.domain import Asset, AssetLibrary, AssetLibraryKind, AssetStatus, ClassificationStatus
from packages.domain.voice_presets import list_voices
from packages.ports.user_repository import UserRepository
from packages.ports.asset_library_repository import AssetLibraryRepository
from packages.ports.asset_repository import AssetRepository
from packages.ports.project_repository import ProjectRepository
from packages.shared.storage import SharedStorageService
logger = logging.getLogger(__name__)
@@ -134,27 +134,47 @@ def synthesize(
"""
user_id = authenticated_user.user.id
# 校验 voice_clone_profile_id 归属(防止越权使用他人克隆音色)
if request.voice_clone_profile_id:
profile = voice_clone_repo.get(request.voice_clone_profile_id)
if profile is None:
# 解析 voice_id:前端可能传克隆音色 profile UUID(而非 CosyVoice voice_id),
# 与 /tts/preview 保持一致:命中 profile → 校验归属 → 取 CosyVoice voice_id
actual_voice_id = request.voice_id
voice_clone_profile_id = request.voice_clone_profile_id
resolved_profile = None
if actual_voice_id:
resolved_profile = voice_clone_repo.get(actual_voice_id)
if resolved_profile is not None:
voice_clone_profile_id = actual_voice_id
# 显式传了 voice_clone_profile_id(且与 voice_id 不同)时再查一次归属
if voice_clone_profile_id and (resolved_profile is None or resolved_profile.id != voice_clone_profile_id):
resolved_profile = voice_clone_repo.get(voice_clone_profile_id)
if resolved_profile is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="Voice clone profile not found",
)
if profile.user_id != user_id:
if resolved_profile is not None:
if resolved_profile.user_id != user_id:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Access denied to voice clone profile",
detail="无权访问该音色",
)
if not resolved_profile.voice_id:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="音色克隆尚未完成,请稍后再试",
)
# 命中克隆音色:job 存解析后的 CosyVoice voice_id
if voice_clone_profile_id == actual_voice_id:
actual_voice_id = resolved_profile.voice_id
use_case = CreateTTSJobUseCase(repository)
job = use_case.execute(
user_id=user_id,
input_text=request.text,
voice_id=request.voice_id,
voice_id=actual_voice_id,
voice_model=request.voice_model,
voice_clone_profile_id=request.voice_clone_profile_id,
voice_clone_profile_id=voice_clone_profile_id,
metadata=request.metadata_,
)
@@ -284,6 +304,41 @@ def delete_tts_job(
return
def _find_or_create_voice_library(
*,
user_id: str,
project_repository: ProjectRepository,
asset_library_repository: AssetLibraryRepository,
) -> AssetLibrary:
"""在用户可访问的项目中找到(或自动创建)voice 素材库。
与前端配音素材页逻辑一致:素材库挂在项目下,配音素材读取
getAssetsByKind("voice") → 用户所有可访问项目中的 voice 库。
优先使用已有 voice 库;没有则在第一个可访问项目中自动创建。
"""
projects = project_repository.find_accessible_projects(user_id)
if not projects:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="没有可用的项目,请先创建项目后再保存配音素材",
)
for project in projects:
for lib in asset_library_repository.find_by_project(project.id):
kind = lib.kind.value if hasattr(lib.kind, "value") else lib.kind
if kind == AssetLibraryKind.VOICE.value:
return lib
# 所有项目都没有 voice 库 → 在第一个可访问项目中自动创建默认配音素材库
project = projects[0]
library = AssetLibrary.create(
project_id=project.id,
name="配音素材库",
kind=AssetLibraryKind.VOICE,
)
return asset_library_repository.create(library)
@router.post(
"/jobs/{job_id}/save-to-library",
response_model=SaveToLibraryResponse,
@@ -294,13 +349,17 @@ def save_tts_job_to_library(
request: SaveToLibraryRequest = SaveToLibraryRequest(),
authenticated_user: AuthenticatedUser = Depends(get_current_user),
tts_repository: SQLAlchemyTTSJobRepository = Depends(_get_repository),
voice_library_repository: SQLAlchemyVoiceLibraryRepository = Depends(get_voice_library_repository),
user_repository: UserRepository = Depends(get_user_repository),
asset_repository: AssetRepository = Depends(get_asset_repository),
asset_library_repository: AssetLibraryRepository = Depends(get_asset_library_repository),
project_repository: ProjectRepository = Depends(get_project_repository),
storage_service: SharedStorageService = Depends(get_storage_service),
sign_url=Depends(get_audio_url_signer),
) -> SaveToLibraryResponse:
"""将已完成的 TTS 合成结果保存到配音库。
"""将已完成的 TTS 合成结果保存到配音素材库(assets 表新素材体系)。
自动携带音色名、时长、语速等元信息。
流程:把 TTS 输出音频转存到用户素材 OSS 路径 → 创建 file_type=audio、
status=ready 的 asset(挂用户 voice 素材库)→ 返回前端可用结构。
配额策略与素材上传一致(上传/ingest 链路无额外配额拦截)。
"""
user_id = authenticated_user.user.id
@@ -318,64 +377,107 @@ def save_tts_job_to_library(
detail="TTS job is not completed yet",
)
# 构建配音素材名称
if not job.output_audio_url and not job.output_audio_key:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="TTS job 缺少输出音频,无法保存",
)
# 素材名称
name = request.name or f"TTS-{job.id[:8]}"
# 构建元信息
metadata_ = {
# 找到(或自动创建)用户 voice 素材库
library = _find_or_create_voice_library(
user_id=user_id,
project_repository=project_repository,
asset_library_repository=asset_library_repository,
)
# 转存音频到素材 OSS 路径(tts-outputs/ 下的产物归 TTS 任务所有,
# 素材独立持有副本,删除 TTS 任务不影响配音库素材)
import tempfile
from pathlib import Path
audio_format = (job.format or "mp3").strip() or "mp3"
content_type_map = {
"mp3": "audio/mpeg",
"wav": "audio/wav",
"pcm": "audio/pcm",
"opus": "audio/opus",
}
content_type = content_type_map.get(audio_format, "audio/mpeg")
storage_key = f"uploads/voice/tts/{job.id}.{audio_format}"
tmp_path: Path | None = None
try:
with tempfile.NamedTemporaryFile(suffix=f".{audio_format}", delete=False) as tmp:
tmp_path = Path(tmp.name)
# 优先用 OSS storage_key(走 oss2 SDK,私有 bucket 也可下载);
# 兜底用 output_audio_url(旧任务可能没有 key)
download_source = job.output_audio_key or job.output_audio_url
downloaded = storage_service.download_asset(download_source, tmp_path)
if not downloaded or not tmp_path.exists() or tmp_path.stat().st_size == 0:
raise HTTPException(
status_code=status.HTTP_502_BAD_GATEWAY,
detail="TTS 音频下载失败,无法保存到配音库",
)
file_size = tmp_path.stat().st_size
storage_service.upload_file(tmp_path, storage_key, content_type=content_type)
except HTTPException:
raise
except Exception as e:
logger.error("TTS 音频转存素材失败: job_id=%s, error=%s", job.id, e, exc_info=True)
raise HTTPException(
status_code=status.HTTP_502_BAD_GATEWAY,
detail="TTS 音频转存失败,无法保存到配音库",
) from e
finally:
if tmp_path and tmp_path.exists():
try:
tmp_path.unlink()
except OSError:
pass
# 构建素材元信息
metadata_: dict[str, object] = {
"source": "tts_job",
"tts_job_id": job.id,
"format": job.format,
"sample_rate": job.sample_rate,
"voice_id": job.voice_id,
"voice_name": job.voice_model or "",
}
if job.metadata:
# 保留原始 job 的有用元信息
for key in ("speed", "language"):
if key in job.metadata:
metadata_[key] = job.metadata[key]
# 获取用户套餐(用于配额检查)
user = user_repository.find_by_id(user_id)
plan_name = getattr(user, "subscription_plan", "free") if user else "free"
# 构建命令并执行
command = CreateVoiceLibraryCommand(
user_id=user_id,
asset = Asset.create(
project_id=library.project_id,
library_id=library.id,
name=name,
text=job.input_text,
voice_provider="cosyvoice",
voice_id=job.voice_id,
voice_name=job.voice_model or "",
audio_url=job.output_audio_url,
duration=job.duration,
file_size=job.file_size,
status="completed",
project_id=job.project_id or "",
tags=[],
metadata_=metadata_,
storage_key=storage_key,
mime_type=content_type,
metadata=metadata_,
file_size=file_size,
duration=job.duration or None,
status=AssetStatus.READY,
classification_status=ClassificationStatus.PENDING, # 音频不参与内容分类,保持 pending 与 ingest 链路一致
uploaded_by_user_id=user_id,
)
use_case = CreateVoiceLibraryUseCase(voice_library_repository)
try:
item = use_case.execute(command, plan_name=plan_name or "free")
except QuotaExceededError as exc:
raise HTTPException(
status_code=status.HTTP_429_TOO_MANY_REQUESTS,
detail=f"配音库配额已满({exc.used}/{exc.limit}),请升级套餐",
) from exc
asset = asset_repository.create(asset)
return SaveToLibraryResponse(
id=item.id,
name=item.name,
audio_url=sign_url(item.audio_url) if item.audio_url else "",
duration=item.duration,
voice_id=item.voice_id,
voice_name=item.voice_name,
status=item.status,
id=asset.id,
name=asset.name,
audio_url=sign_url(storage_key),
duration=asset.duration or 0.0,
voice_id=job.voice_id,
voice_name=job.voice_model or "",
status="completed",
)
@router.post("/preview", response_model=TTSPreviewResponse)
def preview_tts(
request: TTSPreviewRequest,
+59 -4
View File
@@ -7,7 +7,13 @@ from typing import Optional
from app.auth import AuthenticatedUser, get_current_user
from app.core.celery_app import celery_app
from app.dependencies import get_cosyvoice_service, get_voice_clone_profile_repository
from app.core.storage import get_storage_service
from app.dependencies import (
get_asset_repository,
get_cosyvoice_service,
get_project_repository,
get_voice_clone_profile_repository,
)
from app.schemas.voice_clone import (
CreateVoiceCloneRequest,
ListVoiceCloneResponse,
@@ -32,6 +38,9 @@ from packages.application.voice_clone.use_cases import (
from packages.application.voice_clone.workflow import (
VoiceCloneWorkflowService,
)
from packages.ports.asset_repository import AssetRepository
from packages.ports.project_repository import ProjectRepository
from packages.shared.storage import SharedStorageService
logger = logging.getLogger(__name__)
@@ -83,23 +92,69 @@ def create_voice_clone(
request: CreateVoiceCloneRequest,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
workflow: VoiceCloneWorkflowService = Depends(_get_workflow_service),
asset_repository: AssetRepository = Depends(get_asset_repository),
project_repository: ProjectRepository = Depends(get_project_repository),
storage_service: SharedStorageService = Depends(get_storage_service),
) -> VoiceCloneProfileResponse:
"""创建音色克隆任务。
创建 VoiceCloneProfile → 提交 CosyVoice 克隆任务 → 触发 Celery 异步轮询。
如果有 source_audio_url,状态会变为 processing;否则保持 pending。
参考音频两种来源(二选一):
- source_audio_url:前端直传后的音频 URL(兼容旧流程)
- asset_id:配音素材库中的音频素材,服务端用其 OSS storage_key 生成
预签名下载 URL(不依赖前端签名,避免签名过期导致克隆失败)
如果有参考音频,状态会变为 processing;否则保持 pending。
"""
user_id = authenticated_user.user.id
source_audio_url = request.source_audio_url
clone_metadata = dict(request.metadata_ or {})
if request.asset_id:
if source_audio_url:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="asset_id 与 source_audio_url 只能传一个",
)
asset = asset_repository.find_by_id(request.asset_id)
if asset is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="素材不存在",
)
# 归属校验:素材挂在项目素材库下,用户必须能访问该项目
project = project_repository.find_by_id(asset.project_id)
if project is None or not project.can_access(user_id):
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="无权使用该素材",
)
# 类型校验:仅支持音频素材
file_type = asset.file_type if hasattr(asset, "file_type") else ""
if file_type != "audio":
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="仅支持音频素材进行音色克隆",
)
if not asset.storage_key:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="该素材缺少音频文件,无法用于克隆",
)
# 用 OSS storage_key 生成服务端预签名 URL(7 天有效,覆盖克隆重试周期)
source_audio_url = storage_service.get_download_url(asset.storage_key, expires_seconds=7 * 24 * 3600)
clone_metadata["source_asset_id"] = asset.id
profile = workflow.start_clone(
user_id=user_id,
name=request.name,
description=request.description,
source_audio_url=request.source_audio_url,
source_audio_url=source_audio_url,
voice_model=request.voice_model,
language=request.language,
gender=request.gender,
max_retries=request.max_retries,
metadata=request.metadata_,
metadata=clone_metadata,
)
# 如果 profile 处于 processing 且有 task_id,触发 Celery 异步轮询
+2 -1
View File
@@ -13,7 +13,8 @@ class CreateVoiceCloneRequest(BaseModel):
name: str = Field(..., min_length=1, max_length=100, description="音色名称")
description: str = Field("", description="音色描述")
source_audio_url: str = Field("", description="参考音频 URL")
source_audio_url: str = Field("", description="参考音频 URL(与 asset_id 二选一)")
asset_id: str = Field("", description="参考音频素材 ID(配音素材库中的音频 asset,与 source_audio_url 二选一)")
voice_model: str = Field("", description="语音模型名称")
language: str = Field("zh-CN", description="语言")
gender: str = Field("unknown", description="性别")