Compare commits

...

3 Commits

Author SHA1 Message Date
saas-backend-agent 229f9dddeb feat: 正式生成时片段随机重排(降重,默认开启无开关)#1663
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Unit Tests (pull_request) Successful in 1m33s
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m38s
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 10s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 8s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Validate - Style (pull_request) Failing after 2m47s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m50s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m50s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 4m45s
AI Code Review / AI Code Review (pull_request) Successful in 6m36s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 10m33s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 8s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 1m37s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 27m14s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Failing after 3s
在 _distribute_assets 方法中,smart_match 评分排序完成后、
distribute_assets 之前,对 asset_ids 做 random.shuffle。

- smart_match 决定选哪些素材(评分排序保留)
- shuffle 只改变最终分配到 clips 的顺序
- scene_points 缓存不受影响(shuffle 之前已读取)
- asset_ids 先 list() 复制再 shuffle,不修改调用方原列表
- 新增 3 个单元测试验证 shuffle 行为

Closes #1663
2026-09-03 20:01:25 +08:00
xiaoxia 6d2d63da7e feat(voices): TTS时长修复 + 提取视频配音 + AI配音标识 (#1656)
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 2s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 3s
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Check push changed paths (push) Successful in 4s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 3s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / Validate - Style (pull_request) Has been skipped
CI/CD Pipeline / Validate - Security (pull_request) Has been skipped
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (push) Successful in 29s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 12s
CI/CD Pipeline / Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 13s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 16s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 23s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m11s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 20s
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 3s
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 1m48s
AI Code Review / AI Code Review (pull_request) Failing after 2m6s
CI/CD Pipeline / Integration Tests (push) Successful in 2m12s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 1m9s
CI/CD Pipeline / Validate - Style (push) Failing after 3m5s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m5s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m30s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 4m35s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 3m10s
CI/CD Pipeline / Validate - Security (push) Successful in 5m43s
CI/CD Pipeline / Staging E2E Tests (push) Successful in 4m0s
CI/CD Pipeline / Unit Tests (push) Successful in 7m31s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
Co-authored-by: xiaoxia <dev@xiaoxiajianji.com>
Co-committed-by: xiaoxia <dev@xiaoxiajianji.com>
2026-09-03 19:04:35 +08:00
xiaoxia e6e4090f3c feat(voice): 提取视频配音接口 + 素材来源标识 (#1654)
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 8s
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Check push changed paths (push) Successful in 8s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 2m17s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m41s
CI/CD Pipeline / Build Staging Web Image (push) Has been skipped
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m13s
CI/CD Pipeline / Integration Tests (push) Successful in 2m45s
CI/CD Pipeline / Build Staging API Image (push) Successful in 42s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 38s
CI/CD Pipeline / Validate - Style (push) Failing after 3m52s
AI Code Review / AI Code Review (pull_request) Failing after 6m30s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (push) Successful in 27s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 1m13s
CI/CD Pipeline / Validate - Security (push) Failing after 11m41s
CI/CD Pipeline / Frontend Unit Tests (push) Successful in 10m53s
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m22s
CI/CD Pipeline / Unit Tests (push) Successful in 13m10s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Successful in 5m43s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 5m46s
CI/CD Pipeline / Canary Release to Production (push) Has been cancelled
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 1s
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 16s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 16s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 1m40s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m47s
CI/CD Pipeline / Validate - Style (pull_request) Failing after 3m13s
CI/CD Pipeline / Unit Tests (pull_request) Failing after 4m11s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 6m8s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Failing after 3s
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
2026-09-03 18:33:53 +08:00
12 changed files with 845 additions and 6 deletions
+22 -1
View File
@@ -2,7 +2,9 @@
from __future__ import annotations
import json
import logging
import subprocess
import tempfile
from pathlib import Path
from typing import Any, Optional
@@ -430,6 +432,8 @@ def save_tts_job_to_library(
storage_key = f"uploads/voice/tts/{job.id}.{audio_format}"
tmp_path: Path | None = None
audio_duration: float | None = None
file_size = 0
try:
with tempfile.NamedTemporaryFile(suffix=f".{audio_format}", delete=False) as tmp:
tmp_path = Path(tmp.name)
@@ -445,6 +449,23 @@ def save_tts_job_to_library(
)
file_size = tmp_path.stat().st_size
storage_service.upload_file(tmp_path, storage_key, content_type=content_type)
# 从音频文件提取时长(ffprobe),作为 job.duration 的兜底
try:
proc = subprocess.run(
[
"ffprobe", "-v", "quiet", "-print_format", "json",
"-show_format", str(tmp_path),
],
capture_output=True, text=True, timeout=10,
)
if proc.returncode == 0:
fmt = json.loads(proc.stdout).get("format", {})
dur = float(fmt.get("duration", 0))
if dur > 0:
audio_duration = dur
except Exception:
logger.warning("ffprobe 提取时长失败: job_id=%s", job.id, exc_info=True)
except HTTPException:
raise
except Exception as e:
@@ -482,7 +503,7 @@ def save_tts_job_to_library(
mime_type=content_type,
metadata=metadata_,
file_size=file_size,
duration=job.duration or None,
duration=job.duration or audio_duration or None,
status=AssetStatus.READY,
classification_status=ClassificationStatus.PENDING, # 音频不参与内容分类,保持 pending 与 ingest 链路一致
uploaded_by_user_id=user_id,
+260 -2
View File
@@ -6,12 +6,26 @@
from __future__ import annotations
import logging
import shutil
import subprocess
import tempfile
import time
from pathlib import Path
from typing import Literal, Optional
from uuid import uuid4
from app.api.routes._helpers import get_user_plan
from app.auth import AuthenticatedUser, get_current_user
from app.dependencies import get_audio_url_signer, get_cosyvoice_service, get_db_session, get_user_repository
from app.core.storage import get_storage_service
from app.dependencies import (
get_asset_library_repository,
get_asset_repository,
get_audio_url_signer,
get_cosyvoice_service,
get_db_session,
get_project_repository,
get_user_repository,
)
from app.schemas.voice import (
PresetVoiceItemResponse,
PresetVoiceListResponse,
@@ -24,7 +38,7 @@ from app.schemas.voice_library import (
UpdateVoiceLibraryRequest,
VoiceLibraryItemResponse,
)
from fastapi import APIRouter, Depends, HTTPException, Query, Response, status
from fastapi import APIRouter, Depends, File, Form, HTTPException, Query, Response, UploadFile, status
from sqlalchemy.orm import Session
from packages.adapters.sqlalchemy_impl.voice_clone_profile_repository import SQLAlchemyVoiceCloneProfileRepository
@@ -40,8 +54,12 @@ from packages.application.voice_library.use_cases import (
QuotaExceededError,
UpdateVoiceLibraryUseCase,
)
from packages.domain import Asset, AssetStatus
from packages.domain.classification import AssetLibraryKind, ClassificationStatus
from packages.domain.entities import AssetLibrary
from packages.domain.preset_voices import PRESET_VOICES, get_preset_voice_by_id
from packages.ports.user_repository import UserRepository
from packages.shared.storage import SharedStorageService
router = APIRouter()
logger = logging.getLogger(__name__)
@@ -507,3 +525,243 @@ def delete_voice(
if not deleted:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Voice not found")
return
# ── 提取视频配音 ─────────────────────────────────────────────────────
# 支持的视频格式
EXTRACT_VIDEO_MIMES = frozenset({"video/mp4", "video/quicktime", "video/webm", "video/x-msvideo"})
MAX_EXTRACT_SIZE = 500 * 1024 * 1024 # 500MB
@router.post(
"/extract-voice",
status_code=status.HTTP_201_CREATED,
)
def extract_voice_from_video(
file: UploadFile = File(...),
project_id: str = Form(...),
authenticated_user: AuthenticatedUser = Depends(get_current_user),
project_repository=Depends(get_project_repository),
asset_library_repository=Depends(get_asset_library_repository),
asset_repository=Depends(get_asset_repository),
storage_service: SharedStorageService = Depends(get_storage_service),
sign_url=Depends(get_audio_url_signer),
):
"""从上传的视频中提取人声配音。
流程:
1. 接收视频文件(mp4/mov/webm
2. ffmpeg 提取音频 + 降噪 + 编码为 mp3
3. 上传到 OSS,创建 Asset 记录到配音素材库
4. 返回素材信息(时长、文件大小、URL)
"""
user_id = authenticated_user.user.id
# 校验文件类型
content_type = file.content_type or ""
if content_type and content_type not in EXTRACT_VIDEO_MIMES:
# 兜底:按扩展名判断
ext = (file.filename or "").rsplit(".", 1)[-1].lower()
ext_to_mime = {"mp4": "video/mp4", "mov": "video/quicktime", "webm": "video/webm", "avi": "video/x-msvideo"}
if ext not in ext_to_mime:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="仅支持 mp4/mov/webm/avi 格式的视频文件",
)
content_type = ext_to_mime[ext]
# 找到(或自动创建)用户 voice 素材库(复用 TTS 的逻辑)
library = _find_or_create_voice_library_for_extract(
user_id=user_id,
project_repository=project_repository,
asset_library_repository=asset_library_repository,
)
tmp_dir = None
try:
tmp_dir = Path(tempfile.mkdtemp(prefix="voice_extract_"))
video_path = tmp_dir / f"input_{uuid4().hex[:8]}_{file.filename or 'video.mp4'}"
audio_path = tmp_dir / f"output_{uuid4().hex[:8]}.mp3"
# 保存上传的视频到临时文件
with open(video_path, "wb") as f:
total = 0
while chunk := file.file.read(1024 * 1024): # 1MB chunks
total += len(chunk)
if total > MAX_EXTRACT_SIZE:
raise HTTPException(
status_code=status.HTTP_413_REQUEST_ENTITY_TOO_LARGE,
detail="视频文件过大,最大支持 500MB",
)
f.write(chunk)
if video_path.stat().st_size == 0:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="视频文件为空")
# ffmpeg: 提取音频 + 降噪 + 编码 mp3
# 滤镜链:highpass(去低频噪声) → afftdn(FFT降噪) → lowpass(去高频噪声)
ffmpeg_cmd = [
"ffmpeg",
"-y",
"-i",
str(video_path),
"-vn", # 不要视频
"-af",
"highpass=f=80,afftdn=nf=-25:tn=1,lowpass=f=8000",
"-acodec",
"libmp3lame",
"-ab",
"192k",
"-ar",
"44100",
"-ac",
"1", # 单声道(人声足够)
str(audio_path),
]
result = subprocess.run(
ffmpeg_cmd,
capture_output=True,
timeout=300, # 5 分钟超时
)
if result.returncode != 0:
stderr_text = result.stderr.decode("utf-8", errors="replace")[-500:]
logger.error("ffmpeg 提取配音失败: %s", stderr_text)
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
detail="视频音频提取失败,可能该视频没有音轨或格式不支持",
)
if not audio_path.exists() or audio_path.stat().st_size == 0:
raise HTTPException(
status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
detail="音频提取结果为空",
)
# 获取音频时长
duration = _get_audio_duration(audio_path)
file_size = audio_path.stat().st_size
# 上传到 OSS
audio_ext = "mp3"
storage_key = f"uploads/voice/extracted/{uuid4().hex}.{audio_ext}"
storage_service.upload_file(audio_path, storage_key, content_type="audio/mpeg")
# 创建 Asset 记录
original_name = (file.filename or "video").rsplit(".", 1)[0]
asset_name = f"{original_name}-配音"
asset = Asset.create(
project_id=library.project_id,
library_id=library.id,
name=asset_name,
storage_key=storage_key,
mime_type="audio/mpeg",
metadata={
"source": "video_extract",
"original_video": file.filename or "unknown",
},
file_size=file_size,
duration=duration,
status=AssetStatus.READY,
classification_status=ClassificationStatus.PENDING,
uploaded_by_user_id=user_id,
)
asset = asset_repository.create(asset)
return {
"id": asset.id,
"name": asset.name,
"audio_url": sign_url(storage_key),
"duration": duration,
"file_size": file_size,
"status": "completed",
"source": "video_extract",
}
except HTTPException:
raise
except subprocess.TimeoutExpired:
raise HTTPException(
status_code=status.HTTP_504_GATEWAY_TIMEOUT,
detail="视频处理超时,请尝试较短的视频",
)
except Exception as e:
logger.exception("提取视频配音失败: %s", e)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="提取配音失败,请稍后重试",
)
finally:
# 清理临时文件
if tmp_dir and Path(tmp_dir).exists():
shutil.rmtree(tmp_dir, ignore_errors=True)
def _find_or_create_voice_library_for_extract(*, user_id, project_repository, asset_library_repository):
"""为用户找到或创建 voice 素材库(与 TTS 保存逻辑一致)。"""
projects = project_repository.find_accessible_projects(user_id)
if not projects:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="没有可用的项目,请先创建项目",
)
for project in projects:
for lib in asset_library_repository.find_by_project(project.id):
kind = lib.kind.value if hasattr(lib.kind, "value") else lib.kind
if kind == AssetLibraryKind.VOICE.value:
return lib
# 自动创建
from sqlalchemy.exc import IntegrityError
project = projects[0]
library = AssetLibrary.create(
project_id=project.id,
name="配音素材库",
kind=AssetLibraryKind.VOICE,
)
try:
return asset_library_repository.create(library)
except IntegrityError:
session = getattr(asset_library_repository, "session", None)
if session is not None:
try:
session.rollback()
except Exception:
pass
for lib in asset_library_repository.find_by_project(project.id):
kind = lib.kind.value if hasattr(lib.kind, "value") else lib.kind
if kind == AssetLibraryKind.VOICE.value:
return lib
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="配音素材库创建失败",
)
def _get_audio_duration(audio_path: Path) -> float:
"""用 ffprobe 获取音频时长(秒)。"""
try:
result = subprocess.run(
[
"ffprobe",
"-v",
"quiet",
"-show_entries",
"format=duration",
"-of",
"csv=p=0",
str(audio_path),
],
capture_output=True,
timeout=10,
)
if result.returncode == 0 and result.stdout.strip():
return float(result.stdout.strip())
except (ValueError, subprocess.TimeoutExpired):
pass
return 0.0
@@ -234,6 +234,11 @@ class PlanGeneratorService:
# 有缓存的素材片段起点从随机镜头段选取,无缓存走随机起点兜底
asset_scene_points = self._fetch_asset_scene_points(asset_ids)
# 正式生成也随机重排片段顺序(降重,默认开启无开关)
# smart_match 决定选哪些素材,shuffle 只改变分配到 clips 的顺序
asset_ids = list(asset_ids) # 复制避免修改调用方原列表
random.shuffle(asset_ids)
distribute_assets(
clips,
asset_ids,
+1
View File
@@ -28,4 +28,5 @@ export {
deleteTTSJob,
getTtsVoices,
previewTts,
extractVideoVoice,
} from "./jobs"
+53
View File
@@ -70,3 +70,56 @@ export const previewTts = async (data: TTSPreviewRequest): Promise<TTSPreviewRes
const response = await apiClient.post<TTSPreviewResponse>("/tts/preview", data)
return response.data
}
/**
* 从视频中提取配音(上传视频 → 后端提取人声 → 保存到配音素材库)
* 支持 mp4/mov/webm 格式
*/
export const extractVideoVoice = async (
file: File,
onProgress?: (percent: number) => void,
): Promise<{ asset_id: string; duration: number }> => {
const formData = new FormData()
formData.append("file", file)
return new Promise((resolve, reject) => {
const xhr = new XMLHttpRequest()
xhr.open("POST", "/api/v1/tts/extract-video-voice")
// 携带认证 token(从 localStorage 获取,与 apiClient 拦截器一致)
const token = localStorage.getItem("access_token")
if (token) {
xhr.setRequestHeader("Authorization", `Bearer ${token}`)
}
xhr.timeout = 10 * 60 * 1000 // 10 分钟超时
xhr.upload.onprogress = (e) => {
if (e.lengthComputable && onProgress) {
onProgress(Math.round((e.loaded / e.total) * 100))
}
}
xhr.onload = () => {
if (xhr.status >= 200 && xhr.status < 300) {
try {
resolve(JSON.parse(xhr.responseText))
} catch {
reject(new Error("服务器返回数据解析失败"))
}
} else {
try {
const err = JSON.parse(xhr.responseText)
reject(new Error(err.detail || err.message || `提取失败: HTTP ${xhr.status}`))
} catch {
reject(new Error(`提取失败: HTTP ${xhr.status}`))
}
}
}
xhr.onerror = () => reject(new Error("网络错误,请检查网络连接"))
xhr.ontimeout = () => reject(new Error("上传超时(10分钟),请检查网络或尝试更小的文件"))
xhr.send(formData)
})
}
+40 -2
View File
@@ -16,7 +16,12 @@
*/
import React, { useCallback, useEffect, useState } from "react"
import { useSearchParams } from "react-router-dom"
import { UploadOutlined, AudioOutlined, RobotOutlined } from "@ant-design/icons"
import {
UploadOutlined,
AudioOutlined,
RobotOutlined,
VideoCameraOutlined,
} from "@ant-design/icons"
import { Button } from "@/components/ui"
import PageHead from "@/components/layout/PageHead"
import { type AssetItem } from "@/api/assets"
@@ -35,6 +40,8 @@ import { useTtsSynthesize } from "./hooks/useTtsSynthesize"
import { useVoiceUpload } from "./hooks/useVoiceUpload"
import { useMaterialDelete } from "./hooks/useMaterialDelete"
import { useMaterialBatchDelete } from "./hooks/useMaterialBatchDelete"
import { useVideoExtract } from "./hooks/useVideoExtract"
import VideoExtractModal from "./components/VideoExtractModal"
import "./voices.css"
let toastIdSeq = 0
@@ -159,6 +166,18 @@ const VoiceLibrary: React.FC = () => {
handleUploadClose,
} = useVoiceUpload({ showToast })
// ── 提取视频配音 ──────────────────────────────────────
const {
extractOpen,
extractFile,
extractProgress,
isExtracting,
setExtractOpen,
handleFileSelect: handleExtractFileSelect,
handleExtract,
handleExtractClose,
} = useVideoExtract({ showToast })
// ── URL 参数自动打开上传弹窗 ────────────────────────────
const [searchParams, setSearchParams] = useSearchParams()
@@ -200,6 +219,14 @@ const VoiceLibrary: React.FC = () => {
>
</Button>
<Button
buttonType="primary"
buttonSize="sm"
icon={<VideoCameraOutlined />}
onClick={() => setExtractOpen(true)}
>
</Button>
<Button
buttonType="ghost"
buttonSize="sm"
@@ -300,7 +327,18 @@ const VoiceLibrary: React.FC = () => {
/>
)}
{/* ── 弹窗集合 ──────────────────────────────────── */}
{/* ── 视频提取配音弹窗 ─────────────────────────────── */}
<VideoExtractModal
open={extractOpen}
file={extractFile}
progress={extractProgress}
isExtracting={isExtracting}
onClose={handleExtractClose}
onFileSelect={handleExtractFileSelect}
onExtract={handleExtract}
/>
{/* ── 弹窗集合 ─────────────────────────────────── */}
<VoiceModals
cloneModalOpen={cloneModalOpen}
onCloneClose={() => setCloneModalOpen(false)}
@@ -136,6 +136,8 @@ export const MaterialVoiceTab: React.FC<MaterialVoiceTabProps> = ({
const material = mapAssetToMaterial(asset)
// duration 优先取顶层(后端从 metadata 提取),兜底 metadata
const cardDuration = asset.duration || material.duration || 0
// AI 生成素材标识(metadata.source === "tts_job"
const isAiMaterial = (asset.metadata as Record<string, unknown>)?.source === "tts_job"
const isPlaying = playingId === asset.id
const isSelected = selectedIds.has(asset.id)
// 播放中以 audio 真实时长为准,未播放显示卡片时长
@@ -184,6 +186,7 @@ export const MaterialVoiceTab: React.FC<MaterialVoiceTabProps> = ({
<div className="xx-voice-info vmat-info">
<div className="xx-voice-name" title={asset.name}>
{asset.name}
{isAiMaterial && <span className="vmat-ai-badge">AI</span>}
</div>
<div className="xx-voice-subtitle">
{asset.file_size ? `${formatFileSize(asset.file_size)}` : "--"}
@@ -0,0 +1,207 @@
import React, { useRef } from "react"
import { Modal } from "antd"
import { InboxOutlined, CloseOutlined } from "@ant-design/icons"
interface VideoExtractModalProps {
open: boolean
file: File | null
progress: number | null
isExtracting: boolean
onClose: () => void
onFileSelect: (file: File | null) => void
onExtract: () => void
}
const ACCEPT_TYPES = ".mp4,.mov,.webm"
const VideoExtractModal: React.FC<VideoExtractModalProps> = ({
open,
file,
progress,
isExtracting,
onClose,
onFileSelect,
onExtract,
}) => {
const inputRef = useRef<HTMLInputElement>(null)
return (
<Modal
title={<span style={{ fontSize: 16, fontWeight: 600 }}></span>}
open={open}
onCancel={() => {
if (isExtracting) return
onClose()
}}
footer={null}
width={480}
maskClosable={!isExtracting}
>
{!file ? (
<div
className="vmat-upload-dropzone"
onClick={() => inputRef.current?.click()}
style={{
border: "2px dashed #d9d9d9",
borderRadius: 8,
padding: "40px 20px",
textAlign: "center",
cursor: "pointer",
transition: "border-color 0.3s",
}}
onMouseEnter={(e) => (e.currentTarget.style.borderColor = "#7c3aed")}
onMouseLeave={(e) => (e.currentTarget.style.borderColor = "#d9d9d9")}
>
<InboxOutlined style={{ fontSize: 32, color: "#7c3aed", marginBottom: 12 }} />
<p style={{ margin: "0 0 8px", fontSize: 14, color: "#333" }}></p>
<span style={{ fontSize: 12, color: "#999" }}> MP4MOVWebM </span>
<input
ref={inputRef}
type="file"
accept={ACCEPT_TYPES}
style={{ display: "none" }}
onChange={(e) => {
const f = e.target.files?.[0]
if (f) onFileSelect(f)
}}
/>
</div>
) : (
<div>
<div
style={{
display: "flex",
alignItems: "center",
justifyContent: "space-between",
padding: "12px 16px",
background: "#fafafa",
borderRadius: 8,
marginBottom: 16,
}}
>
<span
style={{
flex: 1,
overflow: "hidden",
textOverflow: "ellipsis",
whiteSpace: "nowrap",
fontSize: 14,
fontWeight: 500,
}}
title={file.name}
>
{file.name}
</span>
<span style={{ fontSize: 12, color: "#999", marginLeft: 8, flexShrink: 0 }}>
{(file.size / (1024 * 1024)).toFixed(1)} MB
</span>
{!isExtracting && (
<button
type="button"
onClick={() => {
if (inputRef.current) inputRef.current.value = ""
onFileSelect(null)
}}
style={{
border: "none",
background: "none",
cursor: "pointer",
color: "#999",
marginLeft: 8,
fontSize: 14,
}}
aria-label="移除文件"
>
<CloseOutlined />
</button>
)}
</div>
{progress !== null && (
<div style={{ marginBottom: 12 }}>
<div
style={{
height: 6,
background: "#f0f0f0",
borderRadius: 3,
overflow: "hidden",
}}
>
<div
style={{
height: "100%",
width: `${progress}%`,
background: "linear-gradient(90deg, #7c3aed, #a78bfa)",
borderRadius: 3,
transition: "width 0.3s",
}}
/>
</div>
<div
style={{
textAlign: "right",
fontSize: 12,
color: "#999",
marginTop: 4,
}}
>
{progress}%
</div>
</div>
)}
{isExtracting && (
<p style={{ textAlign: "center", fontSize: 13, color: "#7c3aed", margin: "12px 0 0" }}>
{progress === 100 ? "正在提取人声,请稍候..." : "正在上传视频..."}
</p>
)}
</div>
)}
<div
style={{
display: "flex",
justifyContent: "flex-end",
gap: 8,
marginTop: 24,
}}
>
<button
type="button"
onClick={onClose}
disabled={isExtracting}
style={{
padding: "6px 16px",
borderRadius: 6,
border: "1px solid #d9d9d9",
background: "#fff",
cursor: isExtracting ? "not-allowed" : "pointer",
fontSize: 14,
opacity: isExtracting ? 0.5 : 1,
}}
>
</button>
<button
type="button"
onClick={onExtract}
disabled={!file || isExtracting}
style={{
padding: "6px 16px",
borderRadius: 6,
border: "none",
background: !file || isExtracting ? "#d9d9d9" : "#7c3aed",
color: "#fff",
cursor: !file || isExtracting ? "not-allowed" : "pointer",
fontSize: 14,
fontWeight: 500,
}}
>
{isExtracting ? "提取中..." : "开始提取"}
</button>
</div>
</Modal>
)
}
export default VideoExtractModal
@@ -0,0 +1,74 @@
import { useState, useCallback } from "react"
import { useQueryClient } from "@tanstack/react-query"
import { extractVideoVoice } from "@/api/tts"
/**
* 视频提取配音 Hook
* 封装视频上传弹窗状态、提取进度、提取 mutation 逻辑
*/
interface UseVideoExtractProps {
showToast: (message: string, type: "success" | "error") => void
}
export function useVideoExtract({ showToast }: UseVideoExtractProps) {
const queryClient = useQueryClient()
const [extractOpen, setExtractOpen] = useState(false)
const [extractFile, setExtractFile] = useState<File | null>(null)
const [extractProgress, setExtractProgress] = useState<number | null>(null)
const [isExtracting, setIsExtracting] = useState(false)
const handleExtractClose = useCallback(() => {
setExtractOpen(false)
setExtractFile(null)
setExtractProgress(null)
setIsExtracting(false)
}, [])
const handleExtract = useCallback(async () => {
if (!extractFile) return
setIsExtracting(true)
setExtractProgress(0)
try {
await extractVideoVoice(extractFile, (p) => setExtractProgress(p))
// 刷新素材列表
queryClient.invalidateQueries({ queryKey: ["assets", "voice"] })
queryClient.invalidateQueries({ queryKey: ["voice-materials"] })
showToast("视频配音提取成功", "success")
handleExtractClose()
} catch (err: unknown) {
const msg = err instanceof Error ? err.message : "提取失败,请重试"
showToast(msg, "error")
} finally {
setIsExtracting(false)
setExtractProgress(null)
}
}, [extractFile, queryClient, showToast, handleExtractClose])
const handleFileSelect = useCallback(
(file: File | null) => {
if (!file) {
setExtractFile(null)
return
}
const validTypes = ["video/mp4", "video/quicktime", "video/webm"]
if (!validTypes.includes(file.type)) {
showToast("仅支持 MP4、MOV、WebM 格式的视频文件", "error")
return
}
setExtractFile(file)
},
[showToast],
)
return {
extractOpen,
setExtractOpen,
extractFile,
extractProgress,
isExtracting,
handleFileSelect,
handleExtract,
handleExtractClose,
}
}
+18
View File
@@ -193,6 +193,24 @@
overflow: hidden;
text-overflow: ellipsis;
flex: 1;
display: flex;
align-items: center;
}
/* AI 配音标识 */
.vmat-ai-badge {
display: inline-block;
margin-left: 6px;
padding: 1px 6px;
font-size: 11px;
font-weight: 600;
color: #7c3aed;
background: #f3f0ff;
border: 1px solid #ddd6fe;
border-radius: 4px;
line-height: 16px;
vertical-align: middle;
flex-shrink: 0;
}
.xx-voice-star {
+3 -1
View File
@@ -630,7 +630,7 @@ def ingest_asset(job_id: str) -> dict:
name=filename,
storage_key=job.storage_key,
mime_type=mime_type,
metadata={"ingest_error": error_reason},
metadata={"source": "upload", "ingest_error": error_reason},
file_size=int(metadata.get("size_bytes", 0)),
duration=float(metadata.get("duration", 0)),
width=int(metadata.get("width", 0)),
@@ -666,6 +666,7 @@ def ingest_asset(job_id: str) -> dict:
if existing_asset is None:
# 兜底:如果 API 端没有预先创建 Asset(旧版本兼容),则创建新记录
logger.info("No pre-created asset found for storage_key=%s, creating new", job.storage_key)
metadata["source"] = "upload"
asset = Asset.create(
project_id=job.project_id,
library_id=job.library_id,
@@ -687,6 +688,7 @@ def ingest_asset(job_id: str) -> dict:
# 更新已有的 Asset 记录,补充元数据并将状态改为 READY
asset = existing_asset
asset.mime_type = mime_type
metadata["source"] = "upload"
asset.metadata = metadata
asset.file_size = int(metadata.get("size_bytes", 0))
asset.duration = float(metadata.get("duration", 0))
+159
View File
@@ -1007,3 +1007,162 @@ class TestAssetDurationsAlwaysFetched:
call_kwargs = mock_distribute.call_args
asset_durations = call_kwargs.kwargs.get("asset_durations", call_kwargs[1].get("asset_durations"))
assert asset_durations is None
# ---------------------------------------------------------------------------
# 测试:正式生成片段随机重排(Issue #1663)
# ---------------------------------------------------------------------------
class TestFormalGenerationShuffle:
"""验证正式生成时片段顺序随机化。
Issue #1663: 正式生成时 smart_match 排序后对 asset_ids 做 random.shuffle
使得同一批素材每次生成的视频片段顺序不同,有利于查重降重。
"""
def _make_service_with_asset_repo(self):
"""创建带 mock asset_repo 的 PlanGeneratorService(复用 TestAssetDurationsAlwaysFetched 模式)"""
from apps.api.app.services.plan_generator_service import PlanGeneratorService
plan_repo = StubEditPlanRepository()
clip_repo = StubEditPlanClipRepository()
asset_repo = MagicMock()
def fake_get(asset_id):
mock_asset = MagicMock()
mock_asset.duration = 30.0
mock_asset.quality_score = None
mock_asset.created_at = None
mock_asset.metadata = {}
return mock_asset
asset_repo.get = MagicMock(side_effect=fake_get)
with (
patch(
"apps.api.app.services.plan_generator_service.SQLAlchemyEditPlanRepository",
return_value=plan_repo,
),
patch(
"apps.api.app.services.plan_generator_service.SQLAlchemyEditPlanClipRepository",
return_value=clip_repo,
),
):
db = MagicMock()
svc = PlanGeneratorService(db, asset_repo=asset_repo)
svc._plan_repo = plan_repo
svc._clip_repo = clip_repo
return svc, asset_repo
def test_formal_generation_shuffles_asset_ids(self):
"""正式生成路径下 asset_ids 应被打乱,多次调用顺序应不同"""
svc, _ = self._make_service_with_asset_repo()
template = _make_template("one_take")
# 6 个 clip 容纳 6 个素材
clip_configs = _make_clip_configs(
template_id=template.id,
specs=[
{"clip_type": ClipType.MAIN, "order": i, "min_duration": 3.0, "max_duration": 5.0} for i in range(6)
],
)
asset_ids = ["a1", "a2", "a3", "a4", "a5", "a6"]
# 收集多次调用中 distribute_assets 收到的 asset_ids 顺序
captured_orders = []
with patch(
"apps.api.app.services.plan_generator_service.distribute_assets",
side_effect=lambda clips, asset_ids, *a, **kw: captured_orders.append(list(asset_ids)),
):
# mock _sort_assets_by_smart_score 返回固定顺序,验证 shuffle 会打乱
with patch.object(
svc,
"_sort_assets_by_smart_score",
side_effect=lambda ids: list(ids), # 原样返回
):
with patch.object(
svc,
"_fetch_asset_scene_points",
return_value={},
):
for _ in range(10):
svc.generate_from_template(
template=template,
clip_configs=clip_configs,
asset_ids=list(asset_ids), # 每次传新列表
random_preview=False, # 正式生成
)
assert len(captured_orders) == 10
# 每次 order 应该是 asset_ids 的一个排列
expected_set = set(asset_ids)
for order in captured_orders:
assert set(order) == expected_set
# 10 次调用中应至少出现 2 种不同顺序(概率 > 99.9%)
unique_orders = set(tuple(o) for o in captured_orders)
assert (
len(unique_orders) >= 2
), f"Expected shuffled orders to vary, but got only {len(unique_orders)} unique order(s): {unique_orders}"
def test_formal_generation_does_not_mutate_original_list(self):
"""shuffle 不应修改调用方的原始 asset_ids 列表"""
svc, _ = self._make_service_with_asset_repo()
template = _make_template("one_take")
clip_configs = _make_clip_configs(
template_id=template.id,
specs=[
{"clip_type": ClipType.MAIN, "order": i, "min_duration": 3.0, "max_duration": 5.0} for i in range(4)
],
)
original = ["a1", "a2", "a3", "a4"]
original_copy = list(original)
with patch("apps.api.app.services.plan_generator_service.distribute_assets"):
with patch.object(svc, "_sort_assets_by_smart_score", side_effect=lambda ids: list(ids)):
with patch.object(svc, "_fetch_asset_scene_points", return_value={}):
svc.generate_from_template(
template=template,
clip_configs=clip_configs,
asset_ids=original,
random_preview=False,
)
assert original == original_copy, "Original asset_ids list should not be mutated"
def test_preview_random_mode_unaffected_by_shuffle(self):
"""预览随机模式不走 shuffle 路径,行为不变"""
svc, _ = self._make_service_with_asset_repo()
template = _make_template("one_take")
clip_configs = _make_clip_configs(
template_id=template.id,
specs=[
{"clip_type": ClipType.MAIN, "order": i, "min_duration": 3.0, "max_duration": 5.0} for i in range(4)
],
)
asset_ids = ["a1", "a2", "a3", "a4"]
captured_orders = []
with patch(
"apps.api.app.services.plan_generator_service.distribute_assets",
side_effect=lambda clips, asset_ids, *a, **kw: captured_orders.append(list(asset_ids)),
):
for _ in range(5):
svc.generate_from_template(
template=template,
clip_configs=clip_configs,
asset_ids=list(asset_ids),
random_preview=True, # 预览随机模式
)
assert len(captured_orders) == 5
# 预览模式下 random.shuffle 不应被调用(在 _distribute_assets 的 if not random_selection 块内)
# 所以 asset_ids 应该保持调用方传入的顺序(可能已由上层 shuffle 过)