Compare commits

...

21 Commits

Author SHA1 Message Date
CI Bot 276342520f cleanup: Backend Phase 1 — 5项代码清理任务
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Failing after 8s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 3m28s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
1. 修复密码重置接口路径不一致 Bug (auth.py)
   - forgot_password 硬编码 localhost → 使用 APP_BASE_URL 配置

2. 删除 8 处死代码(未使用 import/变量)
   - 清理多个文件中的未使用导入和变量

3. 删除 8 个空文件/空模块
   - 删除无内容的 __init__.py 文件

4. 合并 3 对 100% 完全重复的函数
   - 提取 check_project_access/get_user_plan/require_project_and_library
   - 新建 apps/api/app/api/routes/_helpers.py 作为共享模块
   - 6 个路由文件改为从 _helpers 导入

5. 对齐 6 个废弃/异常环境变量
   - 修复 DATABASE_POOL_RECYLE 拼写错误 → DATABASE_POOL_RECYCLE
   - 添加 JWT_ALGORITHM/JWT_ACCESS_TOKEN_EXPIRE_MINUTES/JWT_REFRESH_TOKEN_EXPIRE_DAYS 到 Settings
   - 修复 jwt_service.py hasattr 字段名匹配
   - .env.example: CORS_ORIGINS → CORS_ORIGINS_RAW(逗号分隔格式)
   - .env.example: 启用 APP_ENV
   - 修复 OSS_ENDPOINT 默认值拼写错误 (aliiyuncs.com → aliyuncs.com)
   - 添加 COSYVOICE_* 变量来源注释

修改文件: 52 个(新增 1,删除 8,修改 43)
2026-07-13 13:23:30 +08:00
用户CI Test 6feb541127 fix: P0-2 sign_url 返回 HTTPS URL(endpoint 加 https:// 前缀)
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Failing after 10s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 1m20s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
根因:oss2.Bucket 的 endpoint 参数不带 scheme 时,sign_url() 默认生成
HTTP URL(如 http://bucket.oss-cn-hangzhou.aliyuncs.com/...),
前端/浏览器视为不安全请求拒绝加载。

修复:初始化 oss2.Bucket 前检查 endpoint 是否带 http(s):// 前缀,
不带则自动补 https://,确保 sign_url 输出 HTTPS URL。

新增 2 个测试验证 endpoint scheme 处理逻辑。
2026-07-10 17:27:57 +08:00
用户CI Test 6efac8de4b fix: P0-2 OSS 凭证启动验证 + 诊断日志
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Failing after 16s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 1m38s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
- config.py: 非开发环境 OSS_ACCESS_KEY_ID/SECRET 为空时启动失败(fail-fast)
- storage.py: 新增 diagnose() 方法,启动时输出 OSS 配置状态
- 新增 7 个单元测试覆盖凭证验证和诊断逻辑

注意:staging 实际已有 OSS 凭证配置,预签名 URL 可正常生成。
真正问题是 sign_url 返回 HTTP 而非 HTTPS,后续修复。
2026-07-10 17:23:58 +08:00
xiaoxia 8ea8a608f9 Merge pull request 'fix: P0-2 预签名URL诊断日志 + P0-3 xfade exit 234 三重修复' (#207) from fix/p02-p03-deep-fix into develop
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 9s
CI/CD Pipeline / Frontend Lint (push) Successful in 1m50s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
2026-07-10 15:26:21 +08:00
用户CI Test 9fc82df6b9 fix: P1 xfade td 钳制到第二个输入片段时长
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Failing after 10s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 1m32s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
- build_xfade_filter_chain: 加 safe_td = min(safe_td, clip_durations[i])
- 新增 test_middle_clip_shorter_than_td 测试用例 [5.0, 0.3, 5.0]
- 压力测试增加 P1 场景 + td ≤ second_input 断言
- 修复 PR #207 Comment #1282 审计发现
2026-07-10 15:01:59 +08:00
用户CI Test 81984ae938 fix: P0-2 预签名URL诊断日志 + P0-3 xfade exit 234 三重修复
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Failing after 13s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 2m21s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
P0-2 (预签名URL staging未生效):
- storage.py get_download_url() 增加诊断日志(bucket=None / sign_url失败)
- 代码逻辑已验证正确,8个单元测试覆盖全路径
- staging问题最可能是未重新部署或OSS凭证未配置

P0-3 (FFmpeg xfade exit 234):
- Fix 1: _build_single_clip_filter 始终 trim 到 effective_duration
- Fix 2: _build_filter_complex 用 effective_duration 计算 xfade offset
- Fix 3: build_xfade_filter_chain 逐步安全钳制 td/offset
  - 动态 first_input_dur 跟踪
  - per-step td 钳制到 available 和 remaining
  - 保证 offset + td ≤ first_input_duration

新增 20 个单元测试全部通过,全量 1213 测试无回归

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 14:47:09 +08:00
xiaoxia f7bcebecc4 Merge pull request 'fix: generation_tasks results 端点返回 OSS 预签名 URL(24h 有效期)' (#206) from fix/generated-video-url-403 into develop
CI/CD Pipeline / Frontend Lint (push) Successful in 1m1s
CI/CD Pipeline / Validate Code Quality And Tests (push) Successful in 1m22s
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Successful in 3m8s
CI/CD Pipeline / Staging E2E Tests (push) Successful in 1m11s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 3m38s
2026-07-10 12:09:52 +08:00
用户CI Test 8fa287036d fix: generation_tasks results 端点返回 OSS 预签名 URL(24h 有效期)
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 1m18s
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Successful in 1m19s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
P0-2 修复:list_generation_results 端点原先直接返回原始 file_url,
OSS bucket 为 private 导致前端访问视频返回 403。

改动:
- 注入 storage_service 依赖(复用已有 OSSStorageService.get_download_url)
- 对每个视频的 file_url 生成预签名 URL,有效期 86400 秒(24h)
- _to_generated_video_response 支持 download_url 参数
- 与 generated_videos.py 中的模式保持一致

新增测试:
- test_results_endpoint_generates_presigned_urls:验证预签名 URL 生成
- test_results_endpoint_handles_empty_videos:验证空结果正常返回

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 11:25:01 +08:00
xiaoxia 1b5a315bd1 Merge pull request 'fix: 一键生成 P0 修复 + P1 校验' (#203) from fix/oneclick-gen-p0-fixes into develop
CI/CD Pipeline / Frontend Lint (push) Successful in 1m3s
CI/CD Pipeline / Validate Code Quality And Tests (push) Successful in 1m43s
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Successful in 2m14s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 45s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 3m17s
2026-07-10 09:27:02 +08:00
用户CI Test ad5036c93a fix: 删除 _validate_asset_ids_ownership 死代码函数
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 55s
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Successful in 1m23s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
代码审计发现 commit 6d1f4da1 只删了测试用例,函数本体(53行)和注释引用仍残留。

本次修复:
- 删除 _validate_asset_ids_ownership 函数(原第489-538行,53行死代码)
- 更新第391行注释,移除对已删函数的引用
- grep 全仓库确认无其他引用
- 单测 27 passed,无回归

归属校验逻辑已在 P3-2 优化中合并进 _download_library_assets 的同一 DB session。
2026-07-10 09:19:06 +08:00
用户CI Test 6d1f4da112 refactor: 删除 _validate_asset_ids_ownership 死代码函数
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 41s
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Successful in 1m30s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
该函数已在 P3-2 优化中合并到 _download_library_assets,不再被调用。
同时删除 test_oneclick_gen_p0_fixes.py 中对应的 5 个测试用例,
归属校验覆盖已由 test_generation_p3_optimizations.py 提供。
2026-07-10 08:33:18 +08:00
xiaoxia 31c833745f Merge pull request 'fix: expand all UUID fields from varchar(32) to varchar(36) [P0]' (#204) from fix/ingest-job-library-id-length into develop
CI/CD Pipeline / Frontend Lint (push) Successful in 35s
CI/CD Pipeline / Validate Code Quality And Tests (push) Successful in 1m14s
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Successful in 2m20s
CI/CD Pipeline / Staging E2E Tests (push) Successful in 1m11s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 3m55s
2026-07-10 08:02:29 +08:00
用户CI Test ab413413b7 perf(P3): 三项代码审查优化 — strict模式/session合并/HEAD重试
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 1m34s
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Successful in 1m47s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
P3-1: _download_library_assets 增加 strict 参数(默认 true)
  - strict 模式:任意素材下载失败立即抛 RuntimeError
  - non-strict 模式:跳过失败素材返回成功列表,全部失败仍抛异常
  - 归属校验合并到同一 DB session(P3-2),减少连接开销

P3-2: 移除 generate_video 中独立的 _validate_asset_ids_ownership 调用
  - 归属校验已内联到 _download_library_assets 同一 session 中

P3-3: _verify_url_accessible 增加 HEAD 重试(默认 2 次,间隔 1s)
  - 防止 OSS 抖动导致误报不可访问

新增 15 个单元测试覆盖三项优化
更新 1 个旧测试适配新 strict 模式错误信息

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 07:42:39 +08:00
用户CI Test f5f24e828c chore: refresh schema-metadata-snapshot.json for UUID varchar(36) expansion
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 1m1s
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Successful in 1m10s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-10 01:28:26 +08:00
用户CI Test abc54ace4e fix: expand all UUID fields from varchar(32) to varchar(36)
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Failing after 13s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 50s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
P0 hotfix: ingest_jobs.library_id (and all other UUID fields across 10
tables) were varchar(32), but standard UUIDs with hyphens are 36 chars,
causing StringDataRightTruncation on insert — mini-program upload was
completely broken.

Changes:
- models.py: All String(32) UUID fields → String(36) across 10 tables
  (projects, edit_templates, edit_plans, template_clip_configs,
   edit_plan_clips, ingest_jobs, classification_jobs, generation_tasks,
   generated_videos, jobs)
- Alembic migration 036: ALTER COLUMN for all affected fields
- Unit tests: 43 tests verifying String(36) on all UUID columns and
  standard UUID construction with hyphens

Fixes: upload/direct/complete StringDataRightTruncation error
2026-07-10 01:23:24 +08:00
用户CI Test c947abc038 fix: 一键生成 P0 修复 + P1 校验
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Successful in 1m10s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 3m40s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
P0-1: _download_library_assets 支持双模式查询(asset_library_id / project_id)
       下载失败时抛出 RuntimeError,不再静默忽略
P0-2: OSS 上传失败抛异常 + 上传后 HEAD 校验 URL 可访问性
P0-3: run_ffmpeg 失败时完整 stderr 打到日志(排查 exit code 183)
       _execute_ffmpeg 额外记录 filter_complex 便于诊断
P1:   template_id 存在性校验(不存在/已禁用 → ValueError)
       asset_ids 归属校验(不属于指定项目/素材库 → ValueError)

新增 16 个单元测试覆盖全部修复点,全量 1170 测试通过无回归
2026-07-10 00:28:47 +08:00
xiaoxia 4328854f58 feat: 统一渲染引擎 + 打通一键生成全链路
CI/CD Pipeline / Validate Code Quality And Tests (push) Successful in 1m7s
CI/CD Pipeline / Frontend Lint (push) Successful in 2m31s
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Successful in 2m57s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 5m5s
CI/CD Pipeline / Staging E2E Tests (push) Successful in 5m29s
2026-07-09 22:59:19 +08:00
xiaoxia ed972a230c feat: 扩展剪辑模板体系 + 实现剪辑计划生成器
CI/CD Pipeline / Validate Code Quality And Tests (push) Successful in 1m10s
CI/CD Pipeline / Frontend Lint (push) Successful in 2m34s
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Successful in 3m40s
CI/CD Pipeline / Staging E2E Tests (push) Successful in 1m51s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 5m1s
2026-07-09 21:21:23 +08:00
xiaoxia 949a5a3206 fix(deps): 锁定cryptography版本以修复CI环境GEN_EMAIL兼容性错误 (#199)
CI/CD Pipeline / Validate Code Quality And Tests (push) Successful in 1m16s
CI/CD Pipeline / Frontend Lint (push) Successful in 2m57s
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Successful in 7m46s
CI/CD Pipeline / Staging E2E Tests (push) Successful in 1m3s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 3m46s
2026-07-09 16:06:16 +08:00
xiaoxia 5ec71f5e1e fix(generation): 修复生成任务状态机缺失导致状态永不更新的问题 (#198)
CI/CD Pipeline / Validate Code Quality And Tests (push) Successful in 1m22s
CI/CD Pipeline / Frontend Lint (push) Successful in 1m47s
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Successful in 3m56s
CI/CD Pipeline / Staging E2E Tests (push) Successful in 1m25s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 3m47s
2026-07-09 15:42:43 +08:00
xiaoxia f8b004930d feat(test): P0级测试质量体系 - 环境变量清单 + 覆盖率门禁 (#197)
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 19s
CI/CD Pipeline / Frontend Lint (push) Successful in 1m30s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
2026-07-09 15:42:19 +08:00
89 changed files with 6807 additions and 782 deletions
+4 -1
View File
@@ -3,6 +3,7 @@
# ==================== 应用配置 ====================
APP_NAME=小虾 SaaS
APP_BASE_URL=http://localhost:3000
APP_ENV=development
# ==================== 数据库配置 ====================
DATABASE_URL=postgresql://xiaoxia_user:your_password@localhost:5432/xiaoxia_saas
@@ -35,7 +36,8 @@ ENVIRONMENT=development
DEBUG=true
# ==================== CORS 配置 ====================
CORS_ORIGINS=["http://localhost:3000","http://localhost:5173"]
# 逗号分隔的域名列表(Settings 读取 CORS_ORIGINS_RAW
CORS_ORIGINS_RAW=http://localhost:3000,http://localhost:5173
# ==================== 阿里云 OSS 配置 ====================
OSS_ENDPOINT=oss-cn-hangzhou.aliyuncs.com
@@ -44,6 +46,7 @@ OSS_ACCESS_KEY_SECRET=your-access-key-secret
OSS_BUCKET_NAME=xiaoxia-autocut
# ==================== CosyVoice 语音合成配置 ====================
# 注意:COSYVOICE_* 变量由 packages/shared/config.py 的 SharedSettings 读取
COSYVOICE_API_KEY=your-cosyvoice-api-key
COSYVOICE_BASE_URL=https://dashscope.aliyuncs.com/api/v1/services/aigc/text2audio
COSYVOICE_MODEL=cosyvoice-v1
+4 -2
View File
@@ -164,7 +164,8 @@ jobs:
USE_IN_MEMORY_DB: "true"
run: |
set -eu
PYTHONPATH="$PWD/apps/api:$PWD" python3 -m pytest tests/unit -q
PYTHONPATH="$PWD/apps/api:$PWD" python3 -m pytest tests/unit -q \
--cov=apps --cov-report=term --cov-report=xml
- name: Start PostgreSQL for integration tests
shell: sh
@@ -209,7 +210,8 @@ jobs:
run: |
set -eu
pip install -q pytest-rerunfailures
PYTHONPATH="$PWD/apps/api:$PWD" python3 -m pytest tests/integration -q --timeout=60 -x --reruns 2 --reruns-delay 1 -m "not performance"
PYTHONPATH="$PWD/apps/api:$PWD" python3 -m pytest tests/integration -q --timeout=60 -x --reruns 2 --reruns-delay 1 -m "not performance" \
--cov=apps --cov-append --cov-report=term --cov-report=xml --cov-fail-under=50
- name: Run API performance baseline tests
shell: sh
@@ -0,0 +1,26 @@
"""Add editing_mode to edit_templates
Revision ID: 035_editing_mode
Revises: 034_cms_enhance
Create Date: 2026-07-09
"""
import sqlalchemy as sa
from alembic import op
revision = "035_editing_mode"
down_revision = "034_cms_enhance"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"edit_templates",
sa.Column("editing_mode", sa.String(20), nullable=False, server_default="one_take"),
)
def downgrade() -> None:
op.drop_column("edit_templates", "editing_mode")
@@ -0,0 +1,68 @@
"""Expand UUID fields from varchar(32) to varchar(36)
All UUID fields across all tables were varchar(32), but standard UUIDs with
hyphens are 36 characters (e.g. 550e8400-e29b-41d4-a716-446655440000).
This caused StringDataRightTruncation errors on insert.
Revision ID: 036_expand_uuid_36
Revises: 035_editing_mode
Create Date: 2026-07-10
"""
import sqlalchemy as sa
from alembic import op
revision = "036_expand_uuid_36"
down_revision = "035_editing_mode"
branch_labels = None
depends_on = None
# ── 表 → 需要扩容的列 ─────────────────────────────────────────────────────────
_TABLES: dict[str, list[str]] = {
"projects": ["id", "owner_user_id"],
"edit_templates": ["id"],
"edit_plans": ["id", "template_id", "source_edit_plan_id", "project_id", "created_by_user_id"],
"template_clip_configs": ["id", "template_id"],
"edit_plan_clips": ["id", "plan_id", "template_clip_config_id", "asset_id"],
"ingest_jobs": ["id", "project_id", "library_id", "result_asset_id"],
"classification_jobs": ["id", "project_id", "asset_id"],
"generation_tasks": [
"id",
"project_id",
"strategy_id",
"asset_library_id",
"voice_library_id",
"created_by_user_id",
"source_edit_plan_id",
"batch_id",
],
"generated_videos": ["id", "project_id", "generation_task_id", "duplicate_of"],
"jobs": ["id", "project_id", "source_id", "created_by_user_id"],
}
def upgrade() -> None:
for table, columns in _TABLES.items():
for col in columns:
op.alter_column(
table,
col,
existing_type=sa.String(32),
type_=sa.String(36),
existing_nullable=None,
)
def downgrade() -> None:
for table, columns in reversed(list(_TABLES.items())):
for col in columns:
op.alter_column(
table,
col,
existing_type=sa.String(36),
type_=sa.String(32),
existing_nullable=None,
)
-1
View File
@@ -1 +0,0 @@
"""API application package."""
-1
View File
@@ -1 +0,0 @@
"""API package."""
+41
View File
@@ -0,0 +1,41 @@
"""路由层共享辅助函数 — 消除跨文件重复定义。"""
from typing import Any
from fastapi import HTTPException, status
from packages.application import GetProjectUseCase
from packages.ports.user_repository import UserRepository
def check_project_access(project_id: str, user_id: str, project_repository) -> None:
"""检查用户是否有项目访问权限。"""
project = project_repository.find_by_id(project_id)
if project is None:
raise HTTPException(status_code=404, detail=f"Project {project_id} not found")
if not project.can_access(user_id):
raise HTTPException(status_code=403, detail="Access denied to project")
def get_user_plan(user_id: str, user_repository: UserRepository) -> str:
"""获取用户的订阅计划名称。"""
user = user_repository.find_by_id(user_id)
if user is None:
return "free"
return getattr(user, "subscription_plan", "free") or "free"
def require_project_and_library(
project_id: str,
library_id: str,
project_repository: Any,
asset_library_repository: Any,
) -> None:
"""Verify project and asset library exist."""
project = GetProjectUseCase(project_repository).execute(project_id)
if project is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Project not found")
libraries = asset_library_repository.find_by_project(project_id)
if not any(item.id == library_id for item in libraries):
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Asset library not found")
+13 -19
View File
@@ -27,6 +27,8 @@ from packages.application import (
)
from packages.domain import AssetStatus, ClassificationStatus
from app.api.routes._helpers import check_project_access
logger = logging.getLogger(__name__)
router = APIRouter()
@@ -72,14 +74,6 @@ def _to_asset_response(item, storage_service=None) -> AssetResponse:
)
def _check_project_access(project_id: str, user_id: str, project_repository) -> None:
"""检查用户是否有项目访问权限"""
project = project_repository.find_by_id(project_id)
if project is None:
raise HTTPException(status_code=404, detail=f"Project {project_id} not found")
if not project.can_access(user_id):
raise HTTPException(status_code=403, detail="Access denied to project")
@router.get("", response_model=ListAssetsResponse)
def list_assets(
@@ -136,7 +130,7 @@ def list_assets(
library = asset_library_repository.get(library_id)
if library is None:
raise HTTPException(status_code=404, detail=f"AssetLibrary {library_id} not found")
_check_project_access(library.project_id, user_id, project_repository)
check_project_access(library.project_id, user_id, project_repository)
if ft:
items = asset_repository.find_by_library_and_file_type(library_id, ft, skip=skip, limit=limit)
total = asset_repository.count_by_project(library.project_id) if not kind else len(items)
@@ -152,7 +146,7 @@ def list_assets(
# 模式2:指定 project_id
if project_id:
_check_project_access(project_id, user_id, project_repository)
check_project_access(project_id, user_id, project_repository)
if ft:
# 无直接方法,加载后按 file_type 过滤(仍比全量加载好)
all_items = asset_repository.find_by_project(project_id)
@@ -210,13 +204,13 @@ def list_assets(
library = asset_library_repository.get(library_id)
if library is None:
raise HTTPException(status_code=404, detail=f"AssetLibrary {library_id} not found")
_check_project_access(library.project_id, user_id, project_repository)
check_project_access(library.project_id, user_id, project_repository)
if kind:
all_items = asset_repository.find_by_library_and_file_type(library_id, kind_to_file_type[kind])
else:
all_items = asset_repository.find_by_library(library_id)
elif project_id:
_check_project_access(project_id, user_id, project_repository)
check_project_access(project_id, user_id, project_repository)
all_items = asset_repository.find_by_project(project_id)
else:
try:
@@ -262,7 +256,7 @@ def update_asset_review_status(
item = asset_repository.get(asset_id)
if item is None:
raise HTTPException(status_code=404, detail=f"Asset {asset_id} not found")
_check_project_access(item.project_id, authenticated_user.user.id, project_repository)
check_project_access(item.project_id, authenticated_user.user.id, project_repository)
_apply_asset_review_status(item, request.review_status)
updated = asset_repository.update(item)
return _to_asset_response(updated)
@@ -286,7 +280,7 @@ def batch_delete_assets(
failed_ids.append(asset_id)
continue
try:
_check_project_access(item.project_id, user_id, project_repository)
check_project_access(item.project_id, user_id, project_repository)
deleted_ids.append(asset_id)
except HTTPException:
failed_ids.append(asset_id)
@@ -307,7 +301,7 @@ def get_asset(
item = asset_repository.find_by_id(asset_id)
if item is None:
raise HTTPException(status_code=404, detail=f"Asset {asset_id} not found")
_check_project_access(item.project_id, authenticated_user.user.id, project_repository)
check_project_access(item.project_id, authenticated_user.user.id, project_repository)
return _to_asset_response(item)
@@ -322,7 +316,7 @@ def update_asset(
item = asset_repository.find_by_id(asset_id)
if item is None:
raise HTTPException(status_code=404, detail=f"Asset {asset_id} not found")
_check_project_access(item.project_id, authenticated_user.user.id, project_repository)
check_project_access(item.project_id, authenticated_user.user.id, project_repository)
# 合并可修改字段
if request.name is not None:
@@ -346,7 +340,7 @@ def delete_asset(
item = asset_repository.find_by_id(asset_id)
if item is None:
raise HTTPException(status_code=404, detail=f"Asset {asset_id} not found")
_check_project_access(item.project_id, authenticated_user.user.id, project_repository)
check_project_access(item.project_id, authenticated_user.user.id, project_repository)
asset_repository.delete(asset_id)
@@ -363,7 +357,7 @@ def tag_asset(
item = asset_repository.find_by_id(asset_id)
if item is None:
raise HTTPException(status_code=404, detail=f"Asset {asset_id} not found")
_check_project_access(item.project_id, authenticated_user.user.id, project_repository)
check_project_access(item.project_id, authenticated_user.user.id, project_repository)
for tag_id in request.tag_ids:
tag = tag_repository.get(tag_id)
if tag is None:
@@ -387,7 +381,7 @@ def untag_asset(
item = asset_repository.find_by_id(asset_id)
if item is None:
raise HTTPException(status_code=404, detail=f"Asset {asset_id} not found")
_check_project_access(item.project_id, authenticated_user.user.id, project_repository)
check_project_access(item.project_id, authenticated_user.user.id, project_repository)
item.remove_tag(tag_id)
asset_repository.update(item)
-1
View File
@@ -243,7 +243,6 @@ async def logout(
current_user: AuthenticatedUser = Depends(get_current_user),
):
"""登出 - 将当前 token 加入黑名单"""
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
if credentials:
try:
+3 -19
View File
@@ -14,7 +14,6 @@ from typing import Any
from uuid import uuid4
from app.auth import AuthenticatedUser, get_current_user
from app.config import get_settings
from app.core.celery_app import celery_app
from app.core.storage import OSSStorageService, get_storage_service
from app.dependencies import (
@@ -35,6 +34,8 @@ from fastapi.params import File
from packages.application import GetProjectUseCase, SubmitIngestJobCommand, SubmitIngestJobUseCase
from app.api.routes._helpers import require_project_and_library
router = APIRouter()
logger = logging.getLogger(__name__)
@@ -113,22 +114,6 @@ def _atomic_check_and_record(upload_id: str, chunk_index: int) -> bool:
fcntl.flock(f.fileno(), fcntl.LOCK_UN)
def _require_project_and_library(
project_id: str,
library_id: str,
project_repository: Any,
asset_library_repository: Any,
) -> None:
"""Verify project and asset library exist"""
project = GetProjectUseCase(project_repository).execute(project_id)
if project is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Project not found")
libraries = asset_library_repository.find_by_project(project_id)
if not any(item.id == library_id for item in libraries):
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Asset library not found")
def _load_upload_meta(upload_id: str) -> dict[str, Any]:
"""Load upload metadata"""
meta_path = _get_upload_meta_path(upload_id)
@@ -206,7 +191,6 @@ async def init_chunked_upload(
asset_library_repository: Any = Depends(get_asset_library_repository),
) -> ChunkedUploadInitResponse:
"""Initialize chunked upload"""
settings = get_settings()
# Validate file size
if request.file_size > MAX_FILE_SIZE:
@@ -221,7 +205,7 @@ async def init_chunked_upload(
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Project not found")
# Verify asset library
_require_project_and_library(
require_project_and_library(
request.project_id,
request.library_id,
project_repository,
+128 -9
View File
@@ -10,6 +10,8 @@ RESTful CRUD for EditPlan:
- GET /api/v1/edit-plans/{id}/generation-status 查询生成进度(任务 2.05)
- POST /api/v1/edit-plans/{id}/ai-recommend AI 推荐片段方案(任务 3.09)
- POST /api/v1/edit-plans/{id}/generate-cover AI 生成封面(任务 3.09)
- GET /api/v1/edit-plans/{id}/timeline 时间线场景数据
- POST /api/v1/edit-plans/generate-from-template 基于模板+素材自动生成剪辑计划
业务逻辑委托给 EditPlanService 服务层。
"""
@@ -24,17 +26,11 @@ from app.auth import AuthenticatedUser, get_current_user
from app.core.celery_app import celery_app
from app.dependencies import get_asset_library_repository, get_asset_repository, get_db_session, get_project_repository
from app.schemas.generation_task import GenerationTaskResponse
from app.services import EditPlanService
from app.services import EditPlanService, PlanGeneratorService
from fastapi import APIRouter, Depends, HTTPException, Query, status
from pydantic import BaseModel, Field
from sqlalchemy.orm import Session
from packages.adapters.sqlalchemy_impl.asset_library_repository import (
SQLAlchemyAssetLibraryRepository,
)
from packages.adapters.sqlalchemy_impl.asset_repository import (
SQLAlchemyAssetRepository,
)
from packages.adapters.sqlalchemy_impl.generation_task_repository import (
SQLAlchemyGenerationTaskRepository,
)
@@ -203,6 +199,44 @@ class GenerateCoverResponse(BaseModel):
cover: dict[str, Any] = Field(..., description="封面数据(type / image_url / frame_time 等)")
# ── 基于模板生成剪辑计划 Schemas ─────────────────────────────────────────────
class GenerateFromTemplateRequest(BaseModel):
"""基于模板生成剪辑计划请求体"""
template_id: str = Field(..., description="剪辑模板 ID")
asset_ids: List[str] = Field(default_factory=list, description="素材 ID 列表")
project_id: str = Field(default="", description="所属项目 ID")
name: str = Field(default="", description="计划名称(为空则自动取模板名)")
class _PlanClipItem(BaseModel):
"""片段响应体"""
id: str
clip_type: str
order: int
asset_id: str
text_content: str
start_time: float
duration: float
transition_effect: str
status: str
config: Optional[dict[str, Any]] = None
created_at: datetime
updated_at: datetime
model_config = {"from_attributes": True}
class GenerateFromTemplateResponse(BaseModel):
"""基于模板生成剪辑计划响应体"""
plan: EditPlanResponse
clips: List[_PlanClipItem]
# ── Helpers ───────────────────────────────────────────────────────────────────
@@ -644,7 +678,7 @@ def generate_plan(
except HTTPException:
# 已处理的 HTTP 异常直接透传
raise
except Exception as exc:
except Exception:
logger.exception("触发剪辑计划生成失败: plan_id=%s", plan_id)
# 尝试将计划标记为失败(RENDERING → FAILED 是合法的状态流转)
try:
@@ -843,7 +877,7 @@ def ai_recommend_clips(
config=normalized_config,
total_duration=result["total_duration"],
)
except Exception as exc:
except Exception:
logger.exception("AI 推荐写入失败,plan_id=%s 数据可能不一致", plan_id)
# 尝试回滚未提交的变更
try:
@@ -1078,3 +1112,88 @@ def get_plan_timeline(
total_duration=total_duration,
scenes=scenes,
)
# ── 基于模板生成剪辑计划 ─────────────────────────────────────────────────────
@router.post(
"/generate-from-template",
response_model=GenerateFromTemplateResponse,
status_code=status.HTTP_201_CREATED,
)
def generate_from_template(
body: GenerateFromTemplateRequest,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> GenerateFromTemplateResponse:
"""基于模板 + 素材自动生成剪辑计划
流程:
1. 获取模板及其片段配置
2. 调用 PlanGeneratorService 生成 EditPlan + EditPlanClips
3. 返回完整的计划和片段列表
"""
from app.services import EditTemplateService
# 项目鉴权
if body.project_id:
_check_project_access(body.project_id, current_user.user.id, project_repository)
template_svc = EditTemplateService(db)
# 获取模板
try:
template = template_svc.get_template_or_raise(body.template_id)
except ValueError as exc:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=str(exc),
)
# 获取模板片段配置
clip_configs = template_svc.list_clip_configs(body.template_id, skip=0, limit=200)
# 调用 PlanGeneratorService 生成计划
generator = PlanGeneratorService(db)
result = generator.generate_from_template(
template=template,
clip_configs=clip_configs,
asset_ids=body.asset_ids,
project_id=body.project_id,
created_by_user_id=current_user.user.id,
name=body.name,
)
plan = result["plan"]
clips = result["clips"]
logger.info(
"基于模板生成剪辑计划: plan_id=%s template_id=%s clips=%d by user=%s",
plan.id,
body.template_id,
len(clips),
current_user.user.id,
)
return GenerateFromTemplateResponse(
plan=_to_response(plan),
clips=[
_PlanClipItem(
id=c.id,
clip_type=c.clip_type,
order=c.order,
asset_id=c.asset_id,
text_content=c.text_content,
start_time=c.start_time,
duration=c.duration,
transition_effect=c.transition_effect,
status=c.status.value if hasattr(c.status, "value") else c.status,
config=c.config,
created_at=c.created_at,
updated_at=c.updated_at,
)
for c in clips
],
)
+10
View File
@@ -41,6 +41,9 @@ class EditTemplateCreateRequest(BaseModel):
name: str = Field(..., min_length=1, max_length=200, description="模板名称")
description: str = Field(default="", max_length=2000, description="模板描述")
template_type: str = Field(default="default", max_length=50, description="模板类型")
editing_mode: str = Field(
default="one_take", max_length=20, description="剪辑模式: one_take/pip/voice_over/voice_pip"
)
config: dict[str, Any] = Field(default_factory=dict, description="模板配置 (JSON)")
preview_url: str = Field(default="", max_length=500, description="预览地址")
sort_weight: int = Field(default=0, ge=0, le=9999, description="排序权重")
@@ -52,6 +55,9 @@ class EditTemplateUpdateRequest(BaseModel):
name: Optional[str] = Field(default=None, min_length=1, max_length=200, description="模板名称")
description: Optional[str] = Field(default=None, max_length=2000, description="模板描述")
template_type: Optional[str] = Field(default=None, max_length=50, description="模板类型")
editing_mode: Optional[str] = Field(
default=None, max_length=20, description="剪辑模式: one_take/pip/voice_over/voice_pip"
)
config: Optional[dict[str, Any]] = Field(default=None, description="模板配置 (JSON)")
preview_url: Optional[str] = Field(default=None, max_length=500, description="预览地址")
sort_weight: Optional[int] = Field(default=None, ge=0, le=9999, description="排序权重")
@@ -65,6 +71,7 @@ class EditTemplateResponse(BaseModel):
name: str
description: str
template_type: str
editing_mode: str
config: dict[str, Any]
preview_url: str
sort_weight: int
@@ -102,6 +109,7 @@ def _to_response(t: EditTemplate) -> EditTemplateResponse:
name=t.name,
description=t.description,
template_type=t.template_type,
editing_mode=t.editing_mode,
config=t.config,
preview_url=t.preview_url,
sort_weight=t.sort_weight,
@@ -195,6 +203,7 @@ def create_template(
name=body.name,
description=body.description,
template_type=body.template_type,
editing_mode=body.editing_mode,
config=normalized_config,
preview_url=body.preview_url,
sort_weight=body.sort_weight,
@@ -239,6 +248,7 @@ def update_template(
name=body.name,
description=body.description,
template_type=body.template_type,
editing_mode=body.editing_mode,
config=config_to_update,
preview_url=body.preview_url,
sort_weight=body.sort_weight,
+13 -13
View File
@@ -4,6 +4,7 @@ from typing import Any
from app.auth import AuthenticatedUser, get_current_user
from app.core.celery_app import celery_app
from app.core.storage import OSSStorageService, get_storage_service
from app.dependencies import (
get_asset_library_repository,
get_asset_repository,
@@ -23,6 +24,8 @@ from app.schemas.generation_task import (
)
from fastapi import APIRouter, Depends, HTTPException
from app.api.routes._helpers import check_project_access
from packages.application import (
CreateGenerationTaskCommand,
CreateGenerationTaskUseCase,
@@ -33,15 +36,6 @@ from packages.application import (
router = APIRouter()
def _check_project_access(project_id: str, user_id: str, project_repository) -> None:
"""检查用户是否有项目访问权限"""
project = project_repository.find_by_id(project_id)
if project is None:
raise HTTPException(status_code=404, detail=f"Project {project_id} not found")
if not project.can_access(user_id):
raise HTTPException(status_code=403, detail="Access denied to project")
def _to_generation_task_response(task) -> GenerationTaskResponse:
return GenerationTaskResponse(
id=task.id,
@@ -63,7 +57,7 @@ def _to_generation_task_response(task) -> GenerationTaskResponse:
)
def _to_generated_video_response(item) -> GeneratedVideoResponse:
def _to_generated_video_response(item, download_url: str | None = None) -> GeneratedVideoResponse:
return GeneratedVideoResponse(
id=item.id,
project_id=item.project_id,
@@ -76,6 +70,7 @@ def _to_generated_video_response(item) -> GeneratedVideoResponse:
width=item.width,
height=item.height,
fps=item.fps,
download_url=download_url,
)
@@ -254,7 +249,7 @@ def get_generation_task(
if task is None:
raise HTTPException(status_code=404, detail=f"GenerationTask {task_id} not found")
if task.project_id:
_check_project_access(task.project_id, authenticated_user.user.id, project_repository)
check_project_access(task.project_id, authenticated_user.user.id, project_repository)
return _to_generation_task_response(task)
@@ -265,15 +260,20 @@ def list_generation_results(
generation_task_repository: Any = Depends(get_generation_task_repository),
generated_video_repository: Any = Depends(get_generated_video_repository),
project_repository: Any = Depends(get_project_repository),
storage_service: OSSStorageService = Depends(get_storage_service),
) -> ListGeneratedVideosResponse:
task = generation_task_repository.get(task_id)
if task is None:
raise HTTPException(status_code=404, detail=f"GenerationTask {task_id} not found")
if task.project_id:
_check_project_access(task.project_id, authenticated_user.user.id, project_repository)
check_project_access(task.project_id, authenticated_user.user.id, project_repository)
use_case = ListGeneratedVideosByTaskUseCase(generated_video_repository)
items = use_case.execute(task_id)
return ListGeneratedVideosResponse(items=[_to_generated_video_response(item) for item in items])
responses = []
for item in items:
download_url = storage_service.get_download_url(item.file_url, expires_seconds=86400)
responses.append(_to_generated_video_response(item, download_url=download_url))
return ListGeneratedVideosResponse(items=responses)
@router.post("/tasks/{task_id}/retry", response_model=GenerationTaskResponse)
+6 -13
View File
@@ -20,7 +20,7 @@ from typing import Any
from app.auth import AuthenticatedUser, get_current_user
from app.core.celery_app import celery_app
from app.dependencies import get_db_session, get_job_repository, get_project_repository
from app.dependencies import get_job_repository, get_project_repository
from app.schemas.job import (
CompleteJobRequest,
CreateJobRequest,
@@ -51,6 +51,8 @@ from packages.application.jobs import (
)
from packages.domain.job import JobType
from app.api.routes._helpers import check_project_access
logger = logging.getLogger(__name__)
router = APIRouter()
@@ -66,15 +68,6 @@ _JOB_TYPE_TO_CELERY_TASK: dict[str, str] = {
}
def _check_project_access(project_id: str, user_id: str, project_repository) -> None:
"""检查用户是否有项目访问权限。"""
project = project_repository.find_by_id(project_id)
if project is None:
raise HTTPException(status_code=404, detail=f"Project {project_id} not found")
if not project.can_access(user_id):
raise HTTPException(status_code=403, detail="Access denied to project")
# ── 创建任务 ──────────────────────────────────────────────────────────────────
@@ -89,7 +82,7 @@ def create_job(
创建后任务处于 pending 状态,需要调用 /submit 提交执行。
"""
_check_project_access(request.project_id, authenticated_user.user.id, project_repository)
check_project_access(request.project_id, authenticated_user.user.id, project_repository)
# 校验 job_type
try:
@@ -182,7 +175,7 @@ def list_project_jobs(
offset: int = Query(default=0, ge=0),
) -> ListJobsResponse:
"""获取项目下的任务列表。"""
_check_project_access(project_id, authenticated_user.user.id, project_repository)
check_project_access(project_id, authenticated_user.user.id, project_repository)
use_case = ListJobsUseCase(job_repo)
jobs = use_case.execute(
@@ -204,7 +197,7 @@ def get_job_statistics(
project_repository: Any = Depends(get_project_repository),
) -> JobStatisticsResponse:
"""获取项目任务统计摘要。"""
_check_project_access(project_id, authenticated_user.user.id, project_repository)
check_project_access(project_id, authenticated_user.user.id, project_repository)
use_case = GetJobStatisticsUseCase(job_repo)
stats = use_case.execute(project_id)
+3 -8
View File
@@ -33,6 +33,8 @@ from packages.application.recipe.use_cases import (
)
from packages.ports.user_repository import UserRepository
from app.api.routes._helpers import get_user_plan
router = APIRouter()
@@ -40,13 +42,6 @@ def _get_recipe_repository(session: Session = Depends(get_db_session)) -> SQLAlc
return SQLAlchemyRecipeRepository(session)
def _get_user_plan(user_id: str, user_repository: UserRepository) -> str:
user = user_repository.find_by_id(user_id)
if user is None:
return "free"
return getattr(user, "subscription_plan", "free") or "free"
def _item_to_response(item) -> RecipeItemResponse:
return RecipeItemResponse(
id=item.id,
@@ -194,7 +189,7 @@ def use_recipe(
user_repository: UserRepository = Depends(get_user_repository),
) -> UseRecipeResponse:
user_id = authenticated_user.user.id
plan_name = _get_user_plan(user_id, user_repository)
plan_name = get_user_plan(user_id, user_repository)
use_case = UseRecipeUseCase(recipe_repository)
try:
result = use_case.execute(recipe_id, user_id, user_plan=plan_name)
+1 -1
View File
@@ -232,7 +232,7 @@ async def payment_callback(
# 创建账单记录
record_id = uuid.uuid4().hex
record = repo.create(
repo.create(
{
"id": record_id,
"user_id": user_id,
+3 -8
View File
@@ -28,6 +28,8 @@ from packages.application.title_library.use_cases import (
)
from packages.ports.user_repository import UserRepository
from app.api.routes._helpers import get_user_plan
router = APIRouter()
@@ -51,13 +53,6 @@ def _to_response(item) -> TitleLibraryItemResponse:
)
def _get_user_plan(user_id: str, user_repository: UserRepository) -> str:
user = user_repository.find_by_id(user_id)
if user is None:
return "free"
return getattr(user, "subscription_plan", "free") or "free"
@router.get("", response_model=ListTitleLibraryResponse)
def list_titles(
category: Optional[str] = Query(None),
@@ -98,7 +93,7 @@ def create_title(
user_repository: UserRepository = Depends(get_user_repository),
) -> TitleLibraryItemResponse:
user_id = authenticated_user.user.id
plan_name = _get_user_plan(user_id, user_repository)
plan_name = get_user_plan(user_id, user_repository)
command = CreateTitleLibraryCommand(
user_id=user_id,
name=request.name,
+7 -21
View File
@@ -1,5 +1,5 @@
import logging
from typing import Annotated, Any
from typing import Any
from uuid import uuid4
from app.auth import AuthenticatedUser, get_current_user
@@ -17,12 +17,13 @@ from app.schemas.upload import (
DirectUploadCompleteResponse,
DirectUploadPrepareRequest,
DirectUploadPrepareResponse,
UploadAssetRequest,
UploadAssetResponse,
)
from fastapi import APIRouter, Depends, File, Form, HTTPException, UploadFile, status
from packages.application import GetProjectUseCase, SubmitIngestJobCommand, SubmitIngestJobUseCase
from packages.application import SubmitIngestJobCommand, SubmitIngestJobUseCase
from app.api.routes._helpers import require_project_and_library
logger = logging.getLogger(__name__)
@@ -80,21 +81,6 @@ def _validate_mime_type(content_type: str | None) -> str:
return base_type
def _require_project_and_library(
project_id: str,
library_id: str,
project_repository: Any,
asset_library_repository: Any,
) -> None:
project = GetProjectUseCase(project_repository).execute(project_id)
if project is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Project not found")
libraries = asset_library_repository.find_by_project(project_id)
if not any(item.id == library_id for item in libraries):
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Asset library not found")
def _submit_ingest_job(
project_id: str,
library_id: str,
@@ -135,7 +121,7 @@ async def prepare_direct_upload(
# P2-5: 服务端验证 MIME 类型
validated_content_type = _validate_mime_type(request.content_type)
_require_project_and_library(
require_project_and_library(
request.project_id,
request.library_id,
project_repository,
@@ -183,7 +169,7 @@ async def complete_direct_upload(
storage_service: OSSStorageService = Depends(get_storage_service),
) -> DirectUploadCompleteResponse:
"""确认浏览器直传完成并创建导入任务。"""
_require_project_and_library(
require_project_and_library(
request.project_id,
request.library_id,
project_repository,
@@ -252,7 +238,7 @@ async def upload_asset(
storage_service: OSSStorageService = Depends(get_storage_service),
) -> UploadAssetResponse:
"""上传素材文件并触发导入流水线。"""
_require_project_and_library(project_id, library_id, project_repository, asset_library_repository)
require_project_and_library(project_id, library_id, project_repository, asset_library_repository)
# ── 素材去重检测:上传前检查同素材库 + 同 file_hash ──
if file_hash:
-1
View File
@@ -28,7 +28,6 @@ from packages.application.voice_clone.use_cases import (
VoiceCloneNotRetryableError,
)
from packages.application.voice_clone.workflow import (
VoiceCloneWorkflowError,
VoiceCloneWorkflowService,
)
+3 -8
View File
@@ -39,6 +39,8 @@ from packages.application.voice_library.use_cases import (
from packages.domain.preset_voices import PRESET_VOICES
from packages.ports.user_repository import UserRepository
from app.api.routes._helpers import get_user_plan
router = APIRouter()
@@ -118,13 +120,6 @@ def _preset_to_unified_response(preset) -> UnifiedVoiceItemResponse:
)
def _get_user_plan(user_id: str, user_repository: UserRepository) -> str:
user = user_repository.find_by_id(user_id)
if user is None:
return "free"
return getattr(user, "subscription_plan", "free") or "free"
# ==================== 统一配音列表(预置 + 克隆)====================
@@ -260,7 +255,7 @@ def create_voice(
user_repository: UserRepository = Depends(get_user_repository),
) -> VoiceLibraryItemResponse:
user_id = authenticated_user.user.id
plan_name = _get_user_plan(user_id, user_repository)
plan_name = get_user_plan(user_id, user_repository)
command = CreateVoiceLibraryCommand(
user_id=user_id,
name=request.name,
+28 -2
View File
@@ -25,7 +25,7 @@ class Settings(BaseSettings):
DATABASE_POOL_SIZE: int = 20
DATABASE_MAX_OVERFLOW: int = 10 # 调整为合理值:pool_size(20) + max_overflow(10) = 最大30连接
DATABASE_POOL_TIMEOUT: int = 30
DATABASE_POOL_RECYLE: int = 3600
DATABASE_POOL_RECYCLE: int = 3600
USE_IN_MEMORY_DB: bool = False
AUTO_CREATE_SCHEMA: bool = False
@@ -41,6 +41,11 @@ class Settings(BaseSettings):
# 密钥轮换天数(到达此天数后建议更换密钥)
SECRET_ROTATION_DAYS: int = 90
# JWT 算法与过期时间(与 .env.example 对齐)
JWT_ALGORITHM: str = "HS256"
JWT_ACCESS_TOKEN_EXPIRE_MINUTES: int = 30
JWT_REFRESH_TOKEN_EXPIRE_DAYS: int = 30
@field_validator("JWT_SECRET_KEY", mode="before")
@classmethod
def validate_jwt_secret_key(cls, v):
@@ -75,10 +80,31 @@ class Settings(BaseSettings):
CELERY_RESULT_BACKEND: str = "redis://localhost:6379/1"
# OSS 七牛云相关
OSS_ENDPOINT: str = "oss-cn-hangzhou.aliiyuncs.com"
OSS_ENDPOINT: str = "oss-cn-hangzhou.aliyuncs.com"
OSS_ACCESS_KEY_ID: str = ""
OSS_ACCESS_KEY_SECRET: str = ""
OSS_BUCKET_NAME: str = "xiaoxia-autocut"
@field_validator("OSS_ACCESS_KEY_ID", mode="before")
@classmethod
def validate_oss_access_key_id(cls, v):
if (v is None or v == "") and os.getenv("APP_ENV", "development") != "development":
raise ValueError(
"OSS_ACCESS_KEY_ID must be set via environment variable in non-development environments. "
"Check the server .env file (e.g. /var/lib/xiaoxia-saas-staging/.env)."
)
return v or ""
@field_validator("OSS_ACCESS_KEY_SECRET", mode="before")
@classmethod
def validate_oss_access_key_secret(cls, v):
if (v is None or v == "") and os.getenv("APP_ENV", "development") != "development":
raise ValueError(
"OSS_ACCESS_KEY_SECRET must be set via environment variable in non-development environments. "
"Check the server .env file (e.g. /var/lib/xiaoxia-saas-staging/.env)."
)
return v or ""
OSS_DIRECT_UPLOAD_MAX_MB: int = Field(
default=2000,
validation_alias=AliasChoices("OSS_DIRECT_UPLOAD_MAX_MB", "MAX_UPLOAD_SIZE_MB"),
-1
View File
@@ -1 +0,0 @@
"""Core configuration package."""
+42 -2
View File
@@ -34,13 +34,18 @@ class OSSStorageService:
if has_key_id and has_key_secret:
if oss2 is not None:
try:
# P0-2 修复:oss2.Bucket 的 endpoint 必须带 https:// 前缀,
# 否则 sign_url 默认生成 HTTP URL。
bucket_endpoint = settings.OSS_ENDPOINT
if not bucket_endpoint.startswith(("http://", "https://")):
bucket_endpoint = f"https://{bucket_endpoint}"
auth = oss2.Auth(
settings.OSS_ACCESS_KEY_ID,
settings.OSS_ACCESS_KEY_SECRET,
)
self.bucket = oss2.Bucket(
auth,
settings.OSS_ENDPOINT,
bucket_endpoint,
settings.OSS_BUCKET_NAME,
)
logger.info(
@@ -64,6 +69,26 @@ class OSSStorageService:
self.access_key_secret = settings.OSS_ACCESS_KEY_SECRET
self.endpoint = settings.OSS_ENDPOINT
def diagnose(self) -> None:
"""启动诊断:输出 OSS 配置状态,帮助排查预签名 URL 问题。"""
key_id_display = (
f"{self.access_key_id[:4]}...{self.access_key_id[-4:]}" if len(self.access_key_id) > 8 else "(empty)"
)
logger.info(
"[OSS诊断] endpoint=%s bucket_name=%s access_key_id=%s",
self.endpoint,
self.bucket_name,
key_id_display,
)
if self.bucket is None:
logger.error(
"[OSS诊断] ❌ bucket=None — 预签名URL不可用!"
"原因: OSS_ACCESS_KEY_ID/OSS_ACCESS_KEY_SECRET 未配置或 oss2 未安装。"
"请检查服务器 .env 文件(如 /var/lib/xiaoxia-saas-staging/.env"
)
else:
logger.info("[OSS诊断] ✅ bucket 已配置,预签名URL可用")
def _is_local_generated_url(self, storage_key_or_url: str) -> bool:
parsed = urlparse(storage_key_or_url)
path = parsed.path if parsed.scheme else storage_key_or_url
@@ -166,12 +191,26 @@ class OSSStorageService:
if self.bucket is None:
if self._is_local_generated_url(storage_key_or_url):
return storage_key_or_url
logger.warning(
"get_download_url: OSS bucket not configured, returning raw URL. " "storage_key_or_url=%s",
storage_key_or_url[:200],
)
return self.get_url(self._normalize_storage_key(storage_key_or_url))
storage_key = self._normalize_storage_key(storage_key_or_url)
try:
return self.bucket.sign_url("GET", storage_key, expires_seconds)
signed = self.bucket.sign_url("GET", storage_key, expires_seconds)
logger.info(
"get_download_url: signed URL generated. storage_key=%s url_prefix=%s",
storage_key[:80],
signed[:60],
)
return signed
except Exception:
logger.exception(
"get_download_url: sign_url failed, falling back to raw URL. " "storage_key=%s",
storage_key[:200],
)
return self.get_url(storage_key)
def _normalize_storage_key(self, storage_key_or_url: str) -> str:
@@ -240,4 +279,5 @@ def get_storage_service() -> OSSStorageService:
global _storage_service
if _storage_service is None:
_storage_service = OSSStorageService()
_storage_service.diagnose()
return _storage_service
-12
View File
@@ -50,20 +50,8 @@ from packages.adapters.sqlalchemy_impl.voice_clone_profile_repository import (
from packages.adapters.sqlalchemy_impl.voice_library_repository import (
SQLAlchemyVoiceLibraryRepository,
)
from packages.ports.asset_library_repository import AssetLibraryRepository
from packages.ports.asset_repository import AssetRepository
from packages.ports.classification_job_repository import ClassificationJobRepository
from packages.ports.duplication_repository import DuplicationRecordRepository
from packages.ports.generated_video_repository import GeneratedVideoRepository
from packages.ports.generation_task_repository import GenerationTaskRepository
from packages.ports.ingest_job_repository import IngestJobRepository
from packages.ports.job_repository import JobRepository
from packages.ports.project_repository import ProjectRepository
from packages.ports.tag_repository import TagRepository
from packages.ports.title_library_repository import TitleLibraryRepository
from packages.ports.user_repository import UserRepository
from packages.ports.voice_clone_profile_repository import VoiceCloneProfileRepository
from packages.ports.voice_library_repository import VoiceLibraryRepository
_engine, _SessionLocal = build_session_factory(settings.DATABASE_URL)
+1 -1
View File
@@ -10,7 +10,7 @@ from __future__ import annotations
from app.auth import AuthenticatedUser
from app.auth import get_current_user as get_authenticated_user
from app.dependencies import get_user_repository
from fastapi import Depends
from fastapi import Depends, HTTPException
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
from packages.domain.entities import User
+1 -1
View File
@@ -6,7 +6,7 @@ import logging
import time
from typing import Callable
from fastapi import Request, Response
from fastapi import Request
from starlette.middleware.base import BaseHTTPMiddleware
logger = logging.getLogger(__name__)
+1 -1
View File
@@ -2,7 +2,7 @@
from __future__ import annotations
from typing import List, Optional
from typing import Optional
from pydantic import BaseModel, Field
+2
View File
@@ -4,6 +4,7 @@ from .auto_clip_service import AutoClipService
from .edit_plan_service import EditPlanService
from .edit_template_service import EditTemplateService
from .job_service import JobService
from .plan_generator_service import PlanGeneratorService
from .video_compose_service import VideoComposeService
__all__ = [
@@ -11,5 +12,6 @@ __all__ = [
"EditPlanService",
"EditTemplateService",
"JobService",
"PlanGeneratorService",
"VideoComposeService",
]
+1 -1
View File
@@ -24,7 +24,7 @@ from packages.adapters.sqlalchemy_impl import (
)
from packages.domain.asset import AssetType
from packages.domain.classification import AssetClassification
from packages.domain.edit_plan_clip import EditPlanClip, EditPlanClipStatus
from packages.domain.edit_plan_clip import EditPlanClip
logger = logging.getLogger(__name__)
@@ -18,7 +18,6 @@ from packages.adapters.sqlalchemy_impl import (
)
from packages.domain.edit_plan import EditPlan, EditPlanStatus
from packages.domain.edit_plan_clip import EditPlanClip, EditPlanClipStatus
from packages.domain.generation_task import GenerationTaskStatus
logger = logging.getLogger(__name__)
@@ -100,6 +100,7 @@ class EditTemplateService:
*,
description: str = "",
template_type: str = "default",
editing_mode: str = "one_take",
config: Optional[dict[str, Any]] = None,
preview_url: str = "",
sort_weight: int = 0,
@@ -124,6 +125,7 @@ class EditTemplateService:
name=clean_name,
description=description,
template_type=template_type,
editing_mode=editing_mode,
config=config,
preview_url=preview_url,
sort_weight=sort_weight,
@@ -139,6 +141,7 @@ class EditTemplateService:
name: Optional[str] = None,
description: Optional[str] = None,
template_type: Optional[str] = None,
editing_mode: Optional[str] = None,
config: Optional[dict[str, Any]] = None,
preview_url: Optional[str] = None,
sort_weight: Optional[int] = None,
@@ -165,6 +168,7 @@ class EditTemplateService:
name=new_name,
description=description.strip() if description is not None else existing.description,
template_type=template_type.strip() if template_type is not None else existing.template_type,
editing_mode=editing_mode.strip() if editing_mode is not None else existing.editing_mode,
config=config if config is not None else existing.config,
preview_url=preview_url.strip() if preview_url is not None else existing.preview_url,
sort_weight=sort_weight if sort_weight is not None else existing.sort_weight,
-1
View File
@@ -13,7 +13,6 @@ from __future__ import annotations
import logging
from typing import Any
from sqlalchemy.orm import Session
from packages.application.jobs import (
CancelJobUseCase,
@@ -0,0 +1,391 @@
"""PlanGeneratorService — 基于模板+素材自动生成剪辑计划.
核心职责:
- 根据 EditTemplate 的 editing_mode 和 TemplateClipConfig 列表,
自动生成 EditPlan + EditPlanClip 列表
- 四种模式素材分配策略:
- ONE_TAKE: 素材顺序分配给 main 类型 clips
- PIP: 第1个素材→main(全屏背景),其余→overlay clips
- VOICE_OVER: 素材→main clips (B-roll),标记需要配音叠加
- VOICE_PIP: 第1个→background, 第2个→corner_voice, 其余→b_roll
"""
from __future__ import annotations
import logging
from typing import Any, List
from sqlalchemy.orm import Session
from packages.adapters.sqlalchemy_impl import (
SQLAlchemyEditPlanClipRepository,
SQLAlchemyEditPlanRepository,
)
from packages.domain.config_schemas import normalize_plan_config
from packages.domain.edit_plan import EditPlan
from packages.domain.edit_plan_clip import EditPlanClip
from packages.domain.edit_template import EditTemplate
from packages.domain.editing_mode import EditingMode
from packages.domain.template_clip_config import ClipType, TemplateClipConfig
logger = logging.getLogger(__name__)
# ── 默认片段时长(秒) ────────────────────────────────────────────────────────
_DEFAULT_CLIP_DURATION = 5.0
_DEFAULT_INTRO_DURATION = 3.0
_DEFAULT_OUTRO_DURATION = 3.0
class PlanGeneratorService:
"""剪辑计划生成器
基于模板 + 素材,自动生成 EditPlan 及 EditPlanClip 列表。
"""
def __init__(self, db: Session) -> None:
self._plan_repo = SQLAlchemyEditPlanRepository(db)
self._clip_repo = SQLAlchemyEditPlanClipRepository(db)
# ── 公开接口 ─────────────────────────────────────────────────────────────
def generate_from_template(
self,
template: EditTemplate,
clip_configs: List[TemplateClipConfig],
asset_ids: List[str],
*,
project_id: str = "",
created_by_user_id: str = "",
name: str = "",
) -> dict[str, Any]:
"""基于模板+素材生成剪辑计划
Args:
template: 剪辑模板实体
clip_configs: 模板片段配置列表(可为空,自动生成默认结构)
asset_ids: 素材 ID 列表
project_id: 所属项目 ID
created_by_user_id: 创建者用户 ID
name: 计划名称(为空则自动取模板名)
Returns:
dict: {"plan": EditPlan, "clips": List[EditPlanClip]}
"""
editing_mode = template.editing_mode or EditingMode.ONE_TAKE.value
plan_name = name.strip() or f"{template.name} - 剪辑计划"
# 1. 构建 plan config(继承模板的 title/subtitle/bgm,记录 editing_mode
plan_config = self._build_plan_config(template, editing_mode)
# 2. 创建 EditPlan
plan = EditPlan.create(
template_id=template.id,
name=plan_name,
config=plan_config,
total_duration=0.0,
project_id=project_id,
created_by_user_id=created_by_user_id,
)
plan = self._plan_repo.create(plan)
logger.info(
"生成剪辑计划: plan_id=%s template=%s mode=%s assets=%d",
plan.id,
template.id,
editing_mode,
len(asset_ids),
)
# 3. 生成片段列表
if clip_configs:
clips = self._create_clips_from_configs(plan.id, clip_configs)
else:
clips = self._generate_default_clips(plan.id, editing_mode, len(asset_ids))
# 4. 按 editing_mode 分配素材
if asset_ids:
self._distribute_assets(clips, asset_ids, editing_mode)
# 5. 持久化所有 clips 并计算总时长
created_clips: List[EditPlanClip] = []
total_duration = 0.0
for clip in clips:
saved = self._clip_repo.create(clip)
created_clips.append(saved)
total_duration += saved.duration
# 6. 更新 plan 的 total_duration
plan.total_duration = total_duration
plan = self._plan_repo.update(plan)
# 7. 流转到 editing 状态
try:
plan.start_editing()
plan = self._plan_repo.update(plan)
except ValueError as exc:
logger.warning("计划状态流转失败: plan_id=%s error=%s", plan.id, exc)
logger.info(
"剪辑计划生成完成: plan_id=%s clips=%d duration=%.1f",
plan.id,
len(created_clips),
total_duration,
)
return {"plan": plan, "clips": created_clips}
# ── 内部方法 ─────────────────────────────────────────────────────────────
def _build_plan_config(
self,
template: EditTemplate,
editing_mode: str,
) -> dict[str, Any]:
"""从模板配置构建 plan config"""
template_config = template.config or {}
plan_config: dict[str, Any] = {
"editing_mode": editing_mode,
}
# 继承模板的 cover/title/subtitle/bgm 配置
for key in ("cover", "title", "subtitle", "bgm"):
if key in template_config:
plan_config[key] = template_config[key]
return normalize_plan_config(plan_config)
def _create_clips_from_configs(
self,
plan_id: str,
clip_configs: List[TemplateClipConfig],
) -> List[EditPlanClip]:
"""从 TemplateClipConfig 列表创建 EditPlanClip 列表(未持久化)"""
clips: List[EditPlanClip] = []
# 按 order 排序
sorted_configs = sorted(clip_configs, key=lambda c: c.order)
for cfg in sorted_configs:
# 计算时长:取 min_duration 和 max_duration 的中间值
if cfg.min_duration > 0 and cfg.max_duration > 0:
duration = (cfg.min_duration + cfg.max_duration) / 2
elif cfg.min_duration > 0:
duration = cfg.min_duration
elif cfg.max_duration > 0:
duration = cfg.max_duration
else:
duration = _DEFAULT_CLIP_DURATION
# clip_type 可能是枚举或字符串
clip_type = cfg.clip_type.value if hasattr(cfg.clip_type, "value") else cfg.clip_type
# transition_effect 可能是枚举或字符串
transition = (
cfg.transition_effect.value if hasattr(cfg.transition_effect, "value") else cfg.transition_effect
)
clip = EditPlanClip.create(
plan_id=plan_id,
clip_type=clip_type,
order=cfg.order,
template_clip_config_id=cfg.id,
text_content=getattr(cfg, "text_template", "") or "",
duration=duration,
transition_effect=transition or "cut",
)
clips.append(clip)
return clips
def _generate_default_clips(
self,
plan_id: str,
editing_mode: str,
asset_count: int,
) -> List[EditPlanClip]:
"""无 clip_configs 时,根据 editing_mode 生成默认 clip 结构
- ONE_TAKE: N 个 main clipsN = asset_count,至少1个)
- PIP: 1 个 main + (N-1) 个 overlayN = asset_count
- VOICE_OVER: N 个 main clips + 标记需要配音
- VOICE_PIP: 1 个 background + 1 个 corner_voice + (N-2) 个 b_roll
"""
n = max(asset_count, 1)
clips: List[EditPlanClip] = []
order = 0
if editing_mode == EditingMode.PIP.value:
# 1 个 main(全屏背景)
clips.append(
EditPlanClip.create(
plan_id=plan_id,
clip_type=ClipType.MAIN.value,
order=order,
duration=_DEFAULT_CLIP_DURATION,
)
)
order += 1
# 剩余为 overlay
for i in range(1, n):
clips.append(
EditPlanClip.create(
plan_id=plan_id,
clip_type="overlay",
order=order,
duration=_DEFAULT_CLIP_DURATION,
)
)
order += 1
elif editing_mode == EditingMode.VOICE_OVER.value:
# N 个 main clipsB-roll
for i in range(n):
clips.append(
EditPlanClip.create(
plan_id=plan_id,
clip_type=ClipType.MAIN.value,
order=order,
duration=_DEFAULT_CLIP_DURATION,
config={"role": "b_roll"},
)
)
order += 1
elif editing_mode == EditingMode.VOICE_PIP.value:
# 1 个 background
clips.append(
EditPlanClip.create(
plan_id=plan_id,
clip_type="background",
order=order,
duration=_DEFAULT_CLIP_DURATION,
)
)
order += 1
# 1 个 corner_voice
clips.append(
EditPlanClip.create(
plan_id=plan_id,
clip_type="corner_voice",
order=order,
duration=_DEFAULT_CLIP_DURATION,
)
)
order += 1
# 剩余为 b_roll
for i in range(2, n):
clips.append(
EditPlanClip.create(
plan_id=plan_id,
clip_type="b_roll",
order=order,
duration=_DEFAULT_CLIP_DURATION,
)
)
order += 1
else:
# ONE_TAKE: N 个 main clips
for i in range(n):
clips.append(
EditPlanClip.create(
plan_id=plan_id,
clip_type=ClipType.MAIN.value,
order=order,
duration=_DEFAULT_CLIP_DURATION,
)
)
order += 1
return clips
def _distribute_assets(
self,
clips: List[EditPlanClip],
asset_ids: List[str],
editing_mode: str,
) -> None:
"""按 editing_mode 将素材分配到 clips(就地修改,未持久化)
分配策略:
- ONE_TAKE: 素材按顺序依次分配给 main 类型 clips
- PIP: 第1个素材→main(全屏背景),其余→交替分配给 overlay clips
- VOICE_OVER: 素材→main clips (B-roll)
- VOICE_PIP: 第1个→background, 第2个→corner_voice, 其余→b_roll
"""
if not asset_ids or not clips:
return
if editing_mode == EditingMode.ONE_TAKE.value:
self._distribute_one_take(clips, asset_ids)
elif editing_mode == EditingMode.PIP.value:
self._distribute_pip(clips, asset_ids)
elif editing_mode == EditingMode.VOICE_OVER.value:
self._distribute_voice_over(clips, asset_ids)
elif editing_mode == EditingMode.VOICE_PIP.value:
self._distribute_voice_pip(clips, asset_ids)
else:
# 未知模式,退化为 one_take
self._distribute_one_take(clips, asset_ids)
def _distribute_one_take(
self,
clips: List[EditPlanClip],
asset_ids: List[str],
) -> None:
"""ONE_TAKE: 素材按顺序依次分配给 main 类型 clips"""
main_clips = [c for c in clips if c.clip_type == ClipType.MAIN.value]
for i, clip in enumerate(main_clips):
if i < len(asset_ids):
clip.assign_asset(asset_ids[i])
def _distribute_pip(
self,
clips: List[EditPlanClip],
asset_ids: List[str],
) -> None:
"""PIP: 第1个素材→main(全屏背景),其余→overlay clips"""
# 第1个素材 → main clip
main_clips = [c for c in clips if c.clip_type == ClipType.MAIN.value]
if main_clips and asset_ids:
main_clips[0].assign_asset(asset_ids[0])
# 其余素材 → overlay clips
overlay_clips = [c for c in clips if c.clip_type == "overlay"]
remaining = asset_ids[1:]
for i, clip in enumerate(overlay_clips):
if i < len(remaining):
clip.assign_asset(remaining[i])
def _distribute_voice_over(
self,
clips: List[EditPlanClip],
asset_ids: List[str],
) -> None:
"""VOICE_OVER: 素材→main clips (B-roll)"""
main_clips = [c for c in clips if c.clip_type == ClipType.MAIN.value]
for i, clip in enumerate(main_clips):
if i < len(asset_ids):
clip.assign_asset(asset_ids[i])
def _distribute_voice_pip(
self,
clips: List[EditPlanClip],
asset_ids: List[str],
) -> None:
"""VOICE_PIP: 第1个→background, 第2个→corner_voice, 其余→b_roll"""
bg_clips = [c for c in clips if c.clip_type == "background"]
corner_clips = [c for c in clips if c.clip_type == "corner_voice"]
broll_clips = [c for c in clips if c.clip_type == "b_roll"]
# 第1个素材 → background
if bg_clips and len(asset_ids) > 0:
bg_clips[0].assign_asset(asset_ids[0])
# 第2个素材 → corner_voice
if corner_clips and len(asset_ids) > 1:
corner_clips[0].assign_asset(asset_ids[1])
# 其余素材 → b_roll
remaining = asset_ids[2:]
for i, clip in enumerate(broll_clips):
if i < len(remaining):
clip.assign_asset(remaining[i])
@@ -16,7 +16,6 @@ FFmpeg 视频合成编排服务:
from __future__ import annotations
import logging
import shutil
from dataclasses import dataclass, field
from typing import Any
@@ -28,7 +27,7 @@ from packages.adapters.sqlalchemy_impl.edit_plan_clip_repository import (
from packages.adapters.sqlalchemy_impl.edit_plan_repository import (
SQLAlchemyEditPlanRepository,
)
from packages.domain.edit_plan import EditPlan, EditPlanStatus
from packages.domain.edit_plan import EditPlanStatus
from packages.domain.edit_plan_clip import EditPlanClip, EditPlanClipStatus
from packages.domain.template_clip_config import TransitionEffect
+12 -1
View File
@@ -2,6 +2,17 @@
视频处理模块
"""
# 共享工具模块(供 editing_modes / generation / edit_plan_generation 等复用)
from . import dedup_helpers, ffmpeg_utils, oss_helpers
from .processor import VideoProcessor, VideoResult
from .unified_render_service import RenderResult, UnifiedRenderService
__all__ = ["VideoProcessor", "VideoResult"]
__all__ = [
"VideoProcessor",
"VideoResult",
"ffmpeg_utils",
"oss_helpers",
"dedup_helpers",
"UnifiedRenderService",
"RenderResult",
]
@@ -0,0 +1,126 @@
"""查重辅助函数 — 从 generation.py 提取的 GeneratedVideo 记录 + 查重逻辑.
供 render_edit_plan 和 generate_video 共同复用,
创建 GeneratedVideo 记录后计算指纹并执行项目级 + 批次内查重。
"""
from __future__ import annotations
import logging
from uuid import uuid4
from sqlalchemy.orm import Session
logger = logging.getLogger(__name__)
def create_video_record_and_dedup(
*,
generation_task_id: str,
project_id: str,
batch_id: str,
file_url: str,
file_size: int,
duration: float,
video_path: str,
mode: str,
session: Session,
width: int = 1280,
height: int = 720,
fps: float = 25.0,
) -> int:
"""创建 GeneratedVideo 记录,计算指纹并执行查重(历史 + 批次)。
Args:
generation_task_id: 生成任务 ID
project_id: 项目 ID
batch_id: 批次 ID(可为空字符串)
file_url: 视频文件 URL
file_size: 文件大小(字节)
duration: 视频时长(秒)
video_path: 视频本地路径(用于计算指纹)
mode: 剪辑模式名称
session: 数据库会话
width: 视频宽度
height: 视频高度
fps: 视频帧率
Returns:
创建的视频记录数量(1 表示成功,0 表示失败)
"""
from video_processing.dedup import VideoDeduplicator
from packages.adapters.sqlalchemy_impl.generated_video_repository import (
SQLAlchemyGeneratedVideoRepository,
)
from packages.domain import GeneratedVideo
try:
video_id = uuid4().hex
generated_video = GeneratedVideo(
id=video_id,
project_id=project_id,
generation_task_id=generation_task_id,
name=f"generated-{generation_task_id[:8]}.mp4",
file_url=file_url,
file_size=file_size,
duration=duration,
width=width,
height=height,
fps=fps,
status="completed",
generation_params={"mode": mode},
)
video_repo = SQLAlchemyGeneratedVideoRepository(session)
video_repo.create(generated_video)
# 计算视频指纹
deduplicator = VideoDeduplicator()
try:
fingerprint = deduplicator.compute_fingerprint(video_path)
except Exception as fp_err:
logger.warning("Fingerprint computation failed for %s: %s", video_id, fp_err)
session.commit()
return 1
generated_video.video_fingerprint = fingerprint.to_dict()
# (a) 历史成片查重
duplicate_result = deduplicator.check_duplicate(fingerprint, project_id, session)
# (b) 批次内查重(仅当有 batch_id 时)
if not duplicate_result and batch_id:
duplicate_result = deduplicator.check_batch_duplicate(fingerprint, batch_id, video_id, session)
if duplicate_result:
generated_video.is_duplicate = True
generated_video.duplicate_of = duplicate_result["duplicate_of"]
logger.info(
"Duplicate detected: %s -> %s (reason=%s, similarity=%.3f)",
video_id,
duplicate_result["duplicate_of"],
duplicate_result["reason"],
duplicate_result["similarity"],
)
else:
generated_video.is_duplicate = False
generated_video.duplicate_of = None
video_repo.update(generated_video)
session.commit()
logger.info(
"GeneratedVideo record created: %s (task=%s, dup=%s)",
video_id,
generation_task_id,
generated_video.is_duplicate,
)
return 1
except Exception as e:
logger.error(
"Failed to create video record / dedup for task %s: %s",
generation_task_id,
e,
)
session.rollback()
return 0
+41 -84
View File
@@ -5,7 +5,6 @@
import logging
import os
import subprocess
import sys
import tempfile
from dataclasses import dataclass
@@ -22,6 +21,8 @@ else:
from pathlib import Path
from typing import Optional
from video_processing.ffmpeg_utils import FFMPEG_BIN, probe_video_info, run_ffmpeg
logger = logging.getLogger(__name__)
@@ -67,8 +68,6 @@ class EditingModeProcessor:
"""
self.config = config
self.work_dir = work_dir or tempfile.gettempdir()
self._ffmpeg_bin = "ffmpeg"
self._ffprobe_bin = "ffprobe"
def process(
self,
@@ -129,62 +128,20 @@ class EditingModeProcessor:
return os.path.join(self.work_dir, f"output_{self.config.mode}_{os.getpid()}.mp4")
def _run_ffmpeg(self, command: list[str], capture_output: bool = True) -> tuple:
"""执行 FFmpeg 命令"""
logger.debug(f"Running FFmpeg: {' '.join(command)}")
"""执行 FFmpeg 命令 — 委托给共享 ffmpeg_utils.run_ffmpeg"""
try:
result = subprocess.run(
command,
check=True,
stdout=subprocess.PIPE if capture_output else None,
stderr=subprocess.PIPE if capture_output else None,
text=capture_output,
)
return result.stdout or "", result.stderr or ""
except subprocess.CalledProcessError as e:
stderr = e.stderr.decode() if e.stderr else str(e)
logger.error(f"FFmpeg error: {stderr}")
raise RuntimeError(f"FFmpeg execution failed: {stderr}") from e
return run_ffmpeg(command, capture_output=capture_output)
except RuntimeError as e:
logger.error(f"FFmpeg error: {e}")
raise
def _get_video_info(self, video_path: str) -> dict:
"""获取视频信息"""
"""获取视频信息 — 委托给共享 ffmpeg_utils.probe_video_info,补充 codec/size 字段"""
try:
result = subprocess.run(
[
self._ffprobe_bin,
"-v",
"error",
"-show_entries",
"stream=width,height,r_frame_rate,duration,codec_name",
"-show_entries",
"format=duration,size",
"-of",
"json",
video_path,
],
check=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
import json
data = json.loads(result.stdout)
streams = data.get("streams", [{}])
video_stream = next((s for s in streams if s.get("codec_type") == "video"), streams[0] if streams else {})
fmt = data.get("format", {})
fps_str = video_stream.get("r_frame_rate", "25/1")
fps_parts = fps_str.split("/")
fps = float(fps_parts[0]) / float(fps_parts[1]) if len(fps_parts) == 2 else float(fps_parts[0])
return {
"width": int(video_stream.get("width", 0)),
"height": int(video_stream.get("height", 0)),
"fps": fps,
"duration": float(fmt.get("duration", 0)),
"codec": video_stream.get("codec_name", "unknown"),
"size": int(fmt.get("size", 0)),
}
info = probe_video_info(video_path)
info["codec"] = "unknown"
info["size"] = os.path.getsize(video_path) if os.path.exists(video_path) else 0
return info
except Exception as e:
logger.warning(f"Failed to get video info for {video_path}: {e}")
return {"width": 0, "height": 0, "fps": 25, "duration": 0, "codec": "unknown", "size": 0}
@@ -205,7 +162,7 @@ class EditingModeProcessor:
def _normalize_video(self, input_path: str, output_path: str) -> dict:
"""标准化视频格式:先统一帧率,再缩放/填充"""
command = [
self._ffmpeg_bin,
FFMPEG_BIN,
"-y",
"-i",
input_path,
@@ -228,7 +185,7 @@ class EditingModeProcessor:
"-an",
output_path,
]
self._run_ffmpeg(command)
run_ffmpeg(command)
return self._get_video_info(output_path)
def _one_take(self, video_paths: list[str], output_path: str) -> str:
@@ -265,7 +222,7 @@ class EditingModeProcessor:
offset1 = durations[0] - transition / 2
command = [
self._ffmpeg_bin,
FFMPEG_BIN,
"-y",
"-i",
normalized_paths[0],
@@ -285,7 +242,7 @@ class EditingModeProcessor:
"yuv420p",
output_path,
]
self._run_ffmpeg(command)
run_ffmpeg(command)
return output_path
else:
return self._one_take_simple_concat(normalized_paths, output_path)
@@ -298,7 +255,7 @@ class EditingModeProcessor:
f.write(f"file '{os.path.abspath(path)}'\n")
command = [
self._ffmpeg_bin,
FFMPEG_BIN,
"-y",
"-f",
"concat",
@@ -310,7 +267,7 @@ class EditingModeProcessor:
"copy",
output_path,
]
self._run_ffmpeg(command)
run_ffmpeg(command)
try:
os.remove(concat_file)
@@ -344,7 +301,7 @@ class EditingModeProcessor:
if pip_info["duration"] > main_info["duration"]:
temp_pip = os.path.join(self.work_dir, f"pip_temp_{os.getpid()}.mp4")
command = [
self._ffmpeg_bin,
FFMPEG_BIN,
"-y",
"-i",
video_paths[1],
@@ -362,11 +319,11 @@ class EditingModeProcessor:
"yuv420p",
temp_pip,
]
self._run_ffmpeg(command)
run_ffmpeg(command)
pip_normalized_input = temp_pip
else:
command = [
self._ffmpeg_bin,
FFMPEG_BIN,
"-y",
"-i",
video_paths[1],
@@ -382,13 +339,13 @@ class EditingModeProcessor:
"yuv420p",
pip_normalized,
]
self._run_ffmpeg(command)
run_ffmpeg(command)
pip_normalized_input = pip_normalized
if main_info["duration"] > pip_info["duration"]:
looped_pip = os.path.join(self.work_dir, f"pip_looped_{os.getpid()}.mp4")
command = [
self._ffmpeg_bin,
FFMPEG_BIN,
"-y",
"-stream_loop",
"-1",
@@ -408,11 +365,11 @@ class EditingModeProcessor:
"yuv420p",
looped_pip,
]
self._run_ffmpeg(command)
run_ffmpeg(command)
pip_normalized_input = looped_pip
command = [
self._ffmpeg_bin,
FFMPEG_BIN,
"-y",
"-i",
main_normalized,
@@ -432,7 +389,7 @@ class EditingModeProcessor:
"yuv420p",
output_path,
]
self._run_ffmpeg(command)
run_ffmpeg(command)
for temp_file in [main_normalized, pip_normalized]:
if temp_file and temp_file != output_path:
@@ -462,7 +419,7 @@ class EditingModeProcessor:
if bg_info["duration"] < audio_duration:
looped_bg = os.path.join(self.work_dir, f"bg_looped_{os.getpid()}.mp4")
command = [
self._ffmpeg_bin,
FFMPEG_BIN,
"-y",
"-stream_loop",
"-1",
@@ -482,12 +439,12 @@ class EditingModeProcessor:
"yuv420p",
looped_bg,
]
self._run_ffmpeg(command)
run_ffmpeg(command)
bg_normalized = looped_bg
elif bg_info["duration"] > audio_duration:
temp_bg = os.path.join(self.work_dir, f"bg_trimmed_{os.getpid()}.mp4")
command = [
self._ffmpeg_bin,
FFMPEG_BIN,
"-y",
"-i",
bg_normalized,
@@ -497,12 +454,12 @@ class EditingModeProcessor:
"copy",
temp_bg,
]
self._run_ffmpeg(command)
run_ffmpeg(command)
bg_normalized = temp_bg
blurred_bg = os.path.join(self.work_dir, f"bg_blurred_{os.getpid()}.mp4")
command = [
self._ffmpeg_bin,
FFMPEG_BIN,
"-y",
"-i",
bg_normalized,
@@ -518,10 +475,10 @@ class EditingModeProcessor:
"yuv420p",
blurred_bg,
]
self._run_ffmpeg(command)
run_ffmpeg(command)
command = [
self._ffmpeg_bin,
FFMPEG_BIN,
"-y",
"-i",
blurred_bg,
@@ -544,7 +501,7 @@ class EditingModeProcessor:
"-shortest",
output_path,
]
self._run_ffmpeg(command)
run_ffmpeg(command)
for temp_file in [bg_normalized, blurred_bg]:
try:
@@ -582,7 +539,7 @@ class EditingModeProcessor:
voice_adjusted = os.path.join(self.work_dir, f"voice_adj_{os.getpid()}.mp4")
command = [
self._ffmpeg_bin,
FFMPEG_BIN,
"-y",
"-i",
voice_normalized,
@@ -600,11 +557,11 @@ class EditingModeProcessor:
"yuv420p",
voice_adjusted,
]
self._run_ffmpeg(command)
run_ffmpeg(command)
bg_adjusted = os.path.join(self.work_dir, f"bg_adj_{os.getpid()}.mp4")
command = [
self._ffmpeg_bin,
FFMPEG_BIN,
"-y",
"-i",
bg_normalized,
@@ -614,11 +571,11 @@ class EditingModeProcessor:
"copy",
bg_adjusted,
]
self._run_ffmpeg(command)
run_ffmpeg(command)
if audio_path:
command = [
self._ffmpeg_bin,
FFMPEG_BIN,
"-y",
"-i",
bg_adjusted,
@@ -645,7 +602,7 @@ class EditingModeProcessor:
]
else:
command = [
self._ffmpeg_bin,
FFMPEG_BIN,
"-y",
"-i",
bg_adjusted,
@@ -668,7 +625,7 @@ class EditingModeProcessor:
"yuv420p",
output_path,
]
self._run_ffmpeg(command)
run_ffmpeg(command)
for temp_file in [voice_normalized, voice_adjusted, bg_normalized, bg_adjusted]:
try:
@@ -0,0 +1,324 @@
"""FFmpeg 工具函数 — 从 editing_modes.py / video_compose_service.py 提取的共享原语.
提供 FFmpeg / FFprobe 调用、视频信息探测、视频标准化、xfade 转场滤镜构建
等底层能力,供 EditingModeProcessor、VideoComposeService、UnifiedRenderService
共同复用。
"""
from __future__ import annotations
import logging
import shutil
import subprocess # nosec B404
from pathlib import Path
from typing import Any
logger = logging.getLogger(__name__)
# ── 常量 ──────────────────────────────────────────────────────────────────────
FFMPEG_BIN: str = shutil.which("ffmpeg") or "ffmpeg"
FFPROBE_BIN: str = shutil.which("ffprobe") or "ffprobe"
DEFAULT_OUTPUT_WIDTH = 1280
DEFAULT_OUTPUT_HEIGHT = 720
DEFAULT_FPS = 25
# xfade 转场映射:transition_effect 名称 → FFmpeg xfade transition 名称
# 键同时支持 TransitionEffect 枚举值和字符串名称(向后兼容)
XFADE_TRANSITION_MAP: dict[str, str] = {
"fade": "fade",
"slideleft": "slideleft",
"slide_left": "slideleft",
"slideright": "slideright",
"slide_right": "slideright",
"dissolve": "dissolve",
"wipe": "wipeleft",
"wipeleft": "wipeleft",
}
DEFAULT_TRANSITION_DURATION = 0.5
# ── FFmpeg 执行 ───────────────────────────────────────────────────────────────
def run_ffmpeg(
command: list[str],
*,
capture_output: bool = True,
) -> tuple[str, str]:
"""执行 FFmpeg 命令。
Args:
command: 完整的 ffmpeg 命令列表(含 "ffmpeg" 本身)
capture_output: 是否捕获 stdout/stderr
Returns:
(stdout, stderr) 元组
Raises:
subprocess.CalledProcessError: 命令执行失败时抛出,
异常信息包含完整 stderr 以便排查。
"""
try:
result = subprocess.run( # nosec B603
command,
check=True,
stdout=subprocess.PIPE if capture_output else None,
stderr=subprocess.PIPE if capture_output else None,
text=True,
)
return (result.stdout or "", result.stderr or "")
except subprocess.CalledProcessError as e:
# 把完整 stderr 打到日志,方便排查 exit code 183 等问题
stderr_text = (e.stderr or "").strip()
logger.error(
"FFmpeg 命令失败: exit_code=%d command=%s\nstderr:\n%s",
e.returncode,
" ".join(str(c) for c in command[:20]), # 截断过长的命令
stderr_text[:5000], # 截断过长的 stderr
)
raise
def probe_duration(local_path: str | Path) -> float:
"""用 ffprobe 获取视频时长(秒)。
失败时返回默认值 5.0 秒。
"""
try:
result = subprocess.run( # nosec B603
[
FFPROBE_BIN,
"-v",
"error",
"-show_entries",
"format=duration",
"-of",
"default=noprint_wrappers=1:nokey=1",
str(local_path),
],
check=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
return round(float(result.stdout.strip()), 3)
except Exception:
return 5.0
def probe_video_info(video_path: str) -> dict[str, Any]:
"""获取视频信息(宽、高、时长、fps)。
Returns:
{"width": int, "height": int, "duration": float, "fps": float}
失败时返回默认值。
"""
try:
result = subprocess.run( # nosec B603
[
FFPROBE_BIN,
"-v",
"error",
"-select_streams",
"v:0",
"-show_entries",
"stream=width,height,r_frame_rate,duration",
"-show_entries",
"format=duration",
"-of",
"json",
video_path,
],
check=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
)
import json
info = json.loads(result.stdout)
stream = info.get("streams", [{}])[0]
fmt = info.get("format", {})
width = int(stream.get("width", DEFAULT_OUTPUT_WIDTH))
height = int(stream.get("height", DEFAULT_OUTPUT_HEIGHT))
# 解析帧率
fps_str = stream.get("r_frame_rate", "25/1")
if "/" in fps_str:
num, den = fps_str.split("/")
fps = float(num) / float(den) if float(den) > 0 else DEFAULT_FPS
else:
fps = float(fps_str) if fps_str else DEFAULT_FPS
# 时长
duration = float(fmt.get("duration", 0)) or float(stream.get("duration", 0))
return {
"width": width,
"height": height,
"duration": duration,
"fps": round(fps, 2),
}
except Exception as e:
logger.warning("获取视频信息失败: %s, error: %s", video_path, e)
return {
"width": DEFAULT_OUTPUT_WIDTH,
"height": DEFAULT_OUTPUT_HEIGHT,
"duration": 0.0,
"fps": DEFAULT_FPS,
}
def normalize_video(
input_path: str,
output_path: str,
*,
width: int = DEFAULT_OUTPUT_WIDTH,
height: int = DEFAULT_OUTPUT_HEIGHT,
fps: int = DEFAULT_FPS,
) -> dict[str, Any]:
"""标准化视频(缩放 + 恒定帧率)。
使用 scale + pad 保持宽高比,黑边填充到目标分辨率。
Returns:
{"width": int, "height": int, "path": str}
"""
command = [
FFMPEG_BIN,
"-y",
"-i",
input_path,
"-vf",
f"scale={width}:{height}:force_original_aspect_ratio=decrease,"
f"pad={width}:{height}:(ow-iw)/2:(oh-ih)/2:black,"
f"fps={fps}",
"-c:v",
"libx264",
"-crf",
"23",
"-preset",
"medium",
"-c:a",
"aac",
"-b:a",
"128k",
"-movflags",
"+faststart",
output_path,
]
run_ffmpeg(command)
return {"width": width, "height": height, "path": output_path}
# ── xfade / concat 滤镜构建 ──────────────────────────────────────────────────
def chain_filters(filters: list[str], output_label: str, *, input_label: str = "0:v") -> str:
"""将滤镜列表串联为 FFmpeg 滤镜字符串。
例:chain_filters(["scale=1280:720", "fps=25"], "v0")
"[0:v]scale=1280:720,fps=25[v0]"
"""
filter_body = ",".join(filters)
return f"[{input_label}]{filter_body}[{output_label}]"
def resolve_xfade_transition(transition_name: str) -> str:
"""将转场效果名称映射为 FFmpeg xfade transition 名称。
支持 TransitionEffect 枚举值和字符串名称,未知值回退到 "fade"
"""
# 兼容 TransitionEffect 枚举(有 .value 属性)
if hasattr(transition_name, "value"):
transition_name = transition_name.value
return XFADE_TRANSITION_MAP.get(transition_name, "fade")
def build_xfade_filter_chain(
clip_durations: list[float],
clip_video_labels: list[str],
transitions: list[str],
*,
transition_duration: float = DEFAULT_TRANSITION_DURATION,
output_label: str = "outv",
) -> tuple[str, float]:
"""构建 xfade 转场滤镜链。
对每步 xfade 自动钳制 transition duration,确保
``offset + td ≤ first_input_duration``,避免 FFmpeg exit 234。
Args:
clip_durations: 每个片段的时长(必须与 trim 后的实际时长一致)
clip_video_labels: 每个片段的视频流标签(如 "v0", "v1"
transitions: 每个片段对应的转场效果(第一个片段的转场被忽略)
transition_duration: 转场时长(秒)
output_label: 最终输出标签
Returns:
(filter_string, estimated_total_duration)
"""
n = len(clip_durations)
parts: list[str] = []
if n == 0:
return "", 0.0
if n == 1:
parts.append(f"[{clip_video_labels[0]}]copy[{output_label}]")
return ";".join(parts), clip_durations[0]
# xfade 链 — 每步动态钳制 td,防止 offset + td > first_input_duration
cumulative = 0.0
prev_label = clip_video_labels[0]
total_transition = 0.0 # 累计已使用的转场时长
for i in range(1, n):
cumulative += clip_durations[i - 1]
# 当前 xfade 的第一个输入时长
if i == 1:
first_input_dur = clip_durations[0]
else:
first_input_dur = cumulative - total_transition
# 原始 offset 计算
offset = max(0.0, cumulative - transition_duration * i)
# 安全钳制:offset + td 不能超过第一个输入的时长
available = max(0.0, first_input_dur - offset)
safe_td = min(transition_duration, available)
# 同时不能超过剩余总时长
remaining = max(0.0, sum(clip_durations) - cumulative)
safe_td = min(safe_td, remaining)
# 同时不能超过当前第二个输入(单个片段)的时长
safe_td = min(safe_td, clip_durations[i])
safe_td = max(0.001, safe_td) # 至少 1ms,避免 td=0
transition = transitions[i] if i < len(transitions) else "cut"
xfade_transition = resolve_xfade_transition(transition)
if i == n - 1:
out_label = output_label
else:
out_label = f"xf{i}"
parts.append(
f"[{prev_label}][{clip_video_labels[i]}]"
f"xfade=transition={xfade_transition}"
f":duration={safe_td:.3f}"
f":offset={offset:.3f}"
f"[{out_label}]"
)
prev_label = out_label
total_transition += safe_td
# 总时长减去转场重叠部分
total_duration = sum(clip_durations) - total_transition
return ";".join(parts), max(0.0, total_duration)
+164
View File
@@ -0,0 +1,164 @@
"""OSS 工具函数 — 从 generation.py / edit_plan_generation.py 提取的共享 OSS 操作.
提供 OSS 配置读取、Bucket 创建、素材上传/下载、asset_id → 本地路径解析
等能力,供 render_edit_plan 和 generate_video 共同复用。
"""
from __future__ import annotations
import hashlib
import logging
import os
from pathlib import Path
from typing import Optional
from urllib.parse import urlparse
import oss2
logger = logging.getLogger(__name__)
# ── OSS 配置 ──────────────────────────────────────────────────────────────────
def oss_settings() -> tuple[str, str, str, str] | None:
"""获取 OSS 配置。
Returns:
(access_key_id, access_key_secret, endpoint, bucket_name) 元组,
配置缺失时返回 None。
"""
access_key_id = os.getenv("OSS_ACCESS_KEY_ID")
access_key_secret = os.getenv("OSS_ACCESS_KEY_SECRET")
endpoint = os.getenv("OSS_ENDPOINT")
bucket_name = os.getenv("OSS_BUCKET_NAME")
if not all([access_key_id, access_key_secret, endpoint, bucket_name]):
return None
return access_key_id, access_key_secret, endpoint, bucket_name
def oss_bucket() -> oss2.Bucket | None:
"""获取 OSS Bucket 实例。
Returns:
oss2.Bucket 实例,配置缺失时返回 None。
"""
settings = oss_settings()
if settings is None:
return None
access_key_id, access_key_secret, endpoint, bucket_name = settings
return oss2.Bucket(oss2.Auth(access_key_id, access_key_secret), endpoint, bucket_name)
def normalize_storage_key(storage_key_or_url: str) -> str:
"""标准化存储键 — 如果是完整 URL 则提取 path 部分。
Examples:
"https://bucket.oss-cn-hangzhou.aliyuncs.com/path/to/file.mp4"
"path/to/file.mp4"
"path/to/file.mp4""path/to/file.mp4"
"""
if storage_key_or_url.startswith(("http://", "https://")):
return urlparse(storage_key_or_url).path.lstrip("/")
return storage_key_or_url.lstrip("/")
# ── 上传 / 下载 ───────────────────────────────────────────────────────────────
def download_asset(asset_storage_key: str, local_path: Path) -> bool:
"""从 OSS 下载素材文件到本地路径。
Args:
asset_storage_key: 素材的存储键(或完整 URL)
local_path: 本地保存路径
Returns:
True 表示下载成功,False 表示失败。
"""
bucket = oss_bucket()
if bucket is None:
return False
try:
bucket.get_object_to_file(normalize_storage_key(asset_storage_key), str(local_path))
return local_path.exists() and local_path.stat().st_size > 0
except Exception:
logger.exception("下载素材失败: %s", asset_storage_key)
return False
def upload_to_oss(local_path: Path, storage_key: str) -> str | None:
"""上传文件到 OSS,返回公开 URL。
Args:
local_path: 本地文件路径
storage_key: 目标存储键
Returns:
公开访问 URL,上传失败或 OSS 未配置时返回 None。
"""
bucket = oss_bucket()
if bucket is None:
return None
try:
bucket.put_object_from_file(storage_key, str(local_path))
settings = oss_settings()
if settings:
_, _, endpoint, bucket_name = settings
return f"https://{bucket_name}.{endpoint.replace('https://', '').replace('http://', '')}/{storage_key}"
return None
except Exception:
logger.exception("上传 OSS 失败: %s", storage_key)
return None
# ── Asset 解析 ────────────────────────────────────────────────────────────────
def resolve_asset_path(asset_id: str, work_dir: Path) -> Path | None:
"""从 asset_id 解析到本地文件路径。
策略(按优先级):
1. 如果 asset_id 是本地绝对路径(/var/storage/...)→ 直接返回
2. 如果 work_dir 下已有缓存文件 → 返回缓存路径
3. 从 OSS 下载到 work_dir/{hash}.mp4 → 返回下载路径
4. 下载失败 → 返回 None
缓存策略:以 asset_id 的 SHA256 前 16 位为文件名,避免重复下载。
"""
# 1. 本地绝对路径
if asset_id.startswith("/") and os.path.exists(asset_id):
return Path(asset_id)
# 2. 缓存命中
cache_hash = hashlib.sha256(asset_id.encode()).hexdigest()[:16]
cached_path = work_dir / f"{cache_hash}.mp4"
if cached_path.exists() and cached_path.stat().st_size > 0:
return cached_path
# 3. 从 OSS 下载
if download_asset(asset_id, cached_path):
return cached_path
return None
def resolve_asset_ids_to_paths(
asset_ids: list[str],
work_dir: Path,
) -> dict[str, Path]:
"""批量解析 asset_id → 本地路径。
Args:
asset_ids: 素材 ID 列表
work_dir: 工作目录
Returns:
{asset_id: local_path} 映射,仅包含成功解析的条目。
"""
result: dict[str, Path] = {}
for aid in asset_ids:
local_path = resolve_asset_path(aid, work_dir)
if local_path:
result[aid] = local_path
return result
@@ -0,0 +1,489 @@
"""统一渲染引擎 — 输入 EditPlan + EditPlanClips,按时间线+图层渲染视频.
核心原则(灵应):渲染引擎是统一的,不判断模式,只按 clip_type/config.role
分组为图层再合成。
图层分组:
main (无 config.role) → main (z=0)
main + config.role=b_roll → broll (z=0,与 main 同层替换)
overlay → overlay (z=1,画中画叠加)
background → background (z=0,全屏底图)
corner_voice → corner_voice (z=1,右上角小窗)
b_roll → broll (z=0)
intro / outro → main (z=0,按 order 排在首/尾)
合成流程:
1. 每个 clip 先 trim + scale + setpts 预处理
2. 同层 clips 按 order 用 xfade 串联
3. overlay/corner_voice 层 overlay 到主层
4. 如有独立音频轨,amix 混入
"""
from __future__ import annotations
import logging
import os
import subprocess
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
from video_processing.ffmpeg_utils import (
DEFAULT_FPS,
DEFAULT_OUTPUT_HEIGHT,
DEFAULT_OUTPUT_WIDTH,
DEFAULT_TRANSITION_DURATION,
FFMPEG_BIN,
build_xfade_filter_chain,
probe_duration,
probe_video_info,
run_ffmpeg,
)
logger = logging.getLogger(__name__)
# ── 数据结构 ──────────────────────────────────────────────────────────────────
@dataclass
class ResolvedClip:
"""已解析到本地路径的片段。"""
clip_id: str
asset_id: str
local_path: Path
clip_type: str
order: int
start_time: float = 0.0
duration: float = 0.0 # 0 表示使用素材完整时长
transition_effect: str = "cut"
config: dict[str, Any] = field(default_factory=dict)
# 运行时填充
actual_duration: float = 0.0 # 素材实际时长(probe 后填充)
@dataclass
class RenderLayer:
"""渲染图层。"""
role: str # "main" | "overlay" | "pip" | "background" | "corner_voice" | "broll" | "audio"
clips: list[ResolvedClip] = field(default_factory=list)
z_index: int = 0
opacity: float = 1.0
position: tuple[int, int] | None = None # (x, y) 偏移,None 表示全屏
@dataclass
class RenderResult:
"""渲染结果。"""
output_path: Path
duration: float
file_size: int
width: int
height: int
# ── clip_type → layer role 映射 ──────────────────────────────────────────────
def _resolve_layer_role(clip_type: str, config: dict[str, Any]) -> str:
"""根据 clip_type 和 config.role 确定图层角色。
映射规则:
intro / outro → "main"(按 order 排在首/尾)
overlay → "overlay"(画中画叠加,z=1
corner_voice → "corner_voice"(右上角小窗,z=1
background → "background"(全屏底图,z=0
b_roll → "broll"z=0
main + config.role=b_roll → "broll"
main (default) → "main"
"""
role = config.get("role", "")
if clip_type in ("intro", "outro"):
return "main"
if clip_type == "overlay":
return "overlay"
if clip_type == "corner_voice":
return "corner_voice"
if clip_type == "background":
return "background"
if clip_type == "b_roll":
return "broll"
# main type
if role == "b_roll":
return "broll"
return "main"
# ── 图层默认 z_index ─────────────────────────────────────────────────────────
_LAYER_Z_INDEX: dict[str, int] = {
"background": -1,
"broll": 0,
"main": 0,
"overlay": 1,
"corner_voice": 1,
"audio": 2,
}
# 图层默认 PiP 位置(相对输出画布的偏移)
_PIP_SCALE = 0.25 # PiP 占主画面的比例
# ── 统一渲染引擎 ─────────────────────────────────────────────────────────────
class UnifiedRenderService:
"""统一渲染引擎。
输入 EditPlan + EditPlanClips + 素材路径映射,按时间线+图层执行渲染。
"""
def __init__(
self,
plan: Any, # EditPlan
clips: list[Any], # list[EditPlanClip]
asset_path_map: dict[str, Path], # asset_id → local_path
work_dir: Path,
*,
output_width: int = DEFAULT_OUTPUT_WIDTH,
output_height: int = DEFAULT_OUTPUT_HEIGHT,
output_fps: int = DEFAULT_FPS,
transition_duration: float = DEFAULT_TRANSITION_DURATION,
):
self.plan = plan
self.clips = clips
self.asset_path_map = asset_path_map
self.work_dir = work_dir
self.output_width = output_width
self.output_height = output_height
self.output_fps = output_fps
self.transition_duration = transition_duration
def render(self) -> RenderResult:
"""执行渲染,返回 RenderResult。
Raises:
ValueError: 没有可渲染的片段时抛出
"""
# 1. 解析 clips → ResolvedClips(跳过无素材的 clip
resolved = self._resolve_clips()
if not resolved:
raise ValueError("没有可渲染的片段(所有片段素材缺失或下载失败)")
# 2. 分组为 RenderLayers
layers = self._group_clips_into_layers(resolved)
# 3. 构建 filter_complex
output_path = self.work_dir / f"rendered_{self.plan.id}.mp4"
filter_complex, input_args = self._build_filter_complex(layers)
# 4. 执行 FFmpeg
self._execute_ffmpeg(filter_complex, input_args, output_path)
# 5. 探测输出
duration, file_size, width, height = self._probe_output(output_path)
return RenderResult(
output_path=output_path,
duration=duration,
file_size=file_size,
width=width,
height=height,
)
# ── 内部方法 ──────────────────────────────────────────────────────────────
def _resolve_clips(self) -> list[ResolvedClip]:
"""将 EditPlanClip 列表解析为 ResolvedClip 列表。
跳过 asset_id 为空或在 asset_path_map 中找不到的片段。
"""
resolved: list[ResolvedClip] = []
for clip in self.clips:
asset_id = clip.asset_id
if not asset_id:
logger.warning("片段无素材: clip_id=%s", clip.id)
continue
local_path = self.asset_path_map.get(asset_id)
if local_path is None or not local_path.exists():
logger.warning("素材不存在: clip_id=%s asset_id=%s", clip.id, asset_id)
continue
# 探测实际时长
try:
actual_duration = probe_duration(local_path)
except Exception:
actual_duration = clip.duration or 5.0
rc = ResolvedClip(
clip_id=clip.id,
asset_id=asset_id,
local_path=local_path,
clip_type=clip.clip_type,
order=clip.order,
start_time=clip.start_time,
duration=clip.duration,
transition_effect=clip.transition_effect or "cut",
config=clip.config or {},
actual_duration=actual_duration,
)
resolved.append(rc)
# 按 order 排序
resolved.sort(key=lambda c: c.order)
return resolved
def _group_clips_into_layers(self, resolved_clips: list[ResolvedClip]) -> list[RenderLayer]:
"""将 ResolvedClips 分组为 RenderLayers。
分组规则见 _resolve_layer_role 函数文档。
"""
layer_map: dict[str, RenderLayer] = {}
for clip in resolved_clips:
role = _resolve_layer_role(clip.clip_type, clip.config)
if role not in layer_map:
z = _LAYER_Z_INDEX.get(role, 0)
layer_map[role] = RenderLayer(role=role, z_index=z)
layer_map[role].clips.append(clip)
# 每个 layer 内的 clips 按 order 排序
for layer in layer_map.values():
layer.clips.sort(key=lambda c: c.order)
# 计算 PiP 位置
pip_width = int(self.output_width * _PIP_SCALE)
pip_height = int(self.output_height * _PIP_SCALE)
margin = 20 # 边距
if "overlay" in layer_map:
layer_map["overlay"].position = (
self.output_width - pip_width - margin,
margin,
)
if "corner_voice" in layer_map:
layer_map["corner_voice"].position = (
self.output_width - pip_width - margin,
margin,
)
# 按 z_index 排序返回
layers = sorted(layer_map.values(), key=lambda lyr: lyr.z_index)
return layers
def _build_filter_complex(self, layers: list[RenderLayer]) -> tuple[str, list[str]]:
"""构建 FFmpeg filter_complex 字符串和输入参数列表。
Returns:
(filter_complex_str, input_args_list)
input_args_list 是 ["-i", path1, "-i", path2, ...] 格式
"""
if not layers:
raise ValueError("没有可渲染的图层")
# 收集所有 clips(按图层顺序,同层按 order)
all_clips: list[ResolvedClip] = []
for layer in layers:
all_clips.extend(layer.clips)
# 构建输入参数
input_args: list[str] = []
clip_to_input_idx: dict[str, int] = {}
for i, clip in enumerate(all_clips):
input_args.extend(["-i", str(clip.local_path)])
clip_to_input_idx[clip.clip_id] = i
filter_parts: list[str] = []
# Step 1: 预处理每个 clip — scale + setpts
# 为每个 clip 生成预处理后的标签 [v0], [v1], ...
preprocessed_labels: list[str] = []
for i, clip in enumerate(all_clips):
label = f"v{i}"
role = _resolve_layer_role(clip.clip_type, clip.config)
filters: list[str] = []
# trim — 始终将输出截断到有效时长,防止 xfade offset 与实际时长不匹配
# 有效时长 = min(指定时长, 实际时长);若均未设置则跳过
effective_duration = 0.0
if clip.duration > 0:
effective_duration = min(clip.duration, clip.actual_duration) if clip.actual_duration > 0 else clip.duration
elif clip.actual_duration > 0:
effective_duration = clip.actual_duration
if effective_duration > 0:
filters.append(f"trim=duration={effective_duration}")
filters.append("setpts=PTS-STARTPTS")
# scale
if role in ("overlay", "corner_voice"):
pip_w = int(self.output_width * _PIP_SCALE)
pip_h = int(self.output_height * _PIP_SCALE)
filters.append(f"scale={pip_w}:{pip_h}")
elif role == "background":
filters.append(
f"scale={self.output_width}:{self.output_height}" ":force_original_aspect_ratio=increase"
)
filters.append(f"crop={self.output_width}:{self.output_height}")
else:
# main / broll: scale + pad 保持宽高比
filters.append(
f"scale={self.output_width}:{self.output_height}" ":force_original_aspect_ratio=decrease"
)
filters.append(f"pad={self.output_width}:{self.output_height}" ":(ow-iw)/2:(oh-ih)/2:black")
filters.append(f"fps={self.output_fps}")
filters.append("setpts=PTS-STARTPTS")
filter_str = f"[{i}:v]{','.join(filters)}[{label}]"
filter_parts.append(filter_str)
preprocessed_labels.append(label)
# Step 2: 同层 clips 用 xfade 串联
layer_output_labels: dict[str, str] = {}
for layer in layers:
layer_clip_indices = [all_clips.index(c) for c in layer.clips]
layer_labels = [preprocessed_labels[i] for i in layer_clip_indices]
# 使用 trim 后的有效时长,与 Step 1 的 trim=duration 保持一致
layer_durations = []
for i in layer_clip_indices:
c = all_clips[i]
if c.duration > 0:
eff = min(c.duration, c.actual_duration) if c.actual_duration > 0 else c.duration
else:
eff = c.actual_duration if c.actual_duration > 0 else 0.0
layer_durations.append(eff)
layer_transitions = [all_clips[i].transition_effect for i in layer_clip_indices]
if len(layer_labels) == 1:
# 单 clip 层,直接使用预处理标签
layer_output_labels[layer.role] = layer_labels[0]
else:
# 多 clip 层,用 xfade 串联
out_label = f"{layer.role}_merged"
xfade_filter, _ = build_xfade_filter_chain(
clip_durations=layer_durations,
clip_video_labels=layer_labels,
transitions=layer_transitions,
transition_duration=self.transition_duration,
output_label=out_label,
)
if xfade_filter:
filter_parts.append(xfade_filter)
layer_output_labels[layer.role] = out_label
# Step 3: 合成各层
# 找到主层 — background 优先作为底图,其次 broll / main
final_video_label = None
if "background" in layer_output_labels:
final_video_label = layer_output_labels["background"]
# b_roll / main 叠加到 background 上
for role in ("broll", "main"):
if role in layer_output_labels:
base_label = layer_output_labels[role]
combined_label = f"combined_{role}"
filter_parts.append(
f"[{final_video_label}][{base_label}]" f"overlay=(W-w)/2:(H-h)/2[{combined_label}]"
)
final_video_label = combined_label
else:
# 无 background 时,取 broll 或 main 作为基础
for role in ("broll", "main"):
if role in layer_output_labels:
final_video_label = layer_output_labels[role]
break
if final_video_label is None:
# 没有任何主层,使用第一个层
final_video_label = layer_output_labels[layers[0].role]
# 叠加 overlay 层
for layer in layers:
if layer.role in ("overlay", "corner_voice"):
if layer.role not in layer_output_labels:
continue
overlay_label = layer_output_labels[layer.role]
x, y = layer.position or (
self.output_width - int(self.output_width * _PIP_SCALE) - 20,
20,
)
combined_label = f"combined_{layer.role}"
filter_parts.append(f"[{final_video_label}][{overlay_label}]" f"overlay={x}:{y}[{combined_label}]")
final_video_label = combined_label
filter_parts.append(f"[{final_video_label}]format=yuv420p[final_video]")
filter_complex = ";".join(filter_parts)
return filter_complex, input_args
def _execute_ffmpeg(
self,
filter_complex: str,
input_args: list[str],
output_path: Path,
) -> None:
"""执行 FFmpeg 渲染命令。
失败时记录完整 filter_complex 以便排查(如 exit code 183)。
"""
command = [
FFMPEG_BIN,
"-y",
*input_args,
"-filter_complex",
filter_complex,
"-map",
"[final_video]",
"-c:v",
"libx264",
"-crf",
"23",
"-preset",
"medium",
"-pix_fmt",
"yuv420p",
"-movflags",
"+faststart",
str(output_path),
]
logger.info(
"执行渲染: plan_id=%s inputs=%d output=%s",
self.plan.id,
input_args.count("-i"),
output_path,
)
try:
run_ffmpeg(command)
except subprocess.CalledProcessError as e:
# 额外记录 filter_complex,方便排查滤镜链构建问题
logger.error(
"渲染失败: plan_id=%s exit_code=%d\nfilter_complex:\n%s",
self.plan.id,
e.returncode,
filter_complex[:5000],
)
raise
def _probe_output(self, output_path: Path) -> tuple[float, int, int, int]:
"""探测输出文件的时长、大小、宽高。
Returns:
(duration, file_size, width, height)
"""
info = probe_video_info(str(output_path))
file_size = output_path.stat().st_size if output_path.exists() else 0
return (
info["duration"],
file_size,
info["width"],
info["height"],
)
@@ -3,177 +3,39 @@
Celery 任务 worker.render_edit_plan:
1. 加载 EditPlan + EditPlanClips
2. 下载各片段素材
3. 按 order 顺序拼接片段
3. 使用 UnifiedRenderService 按时间线+图层渲染
4. 上传渲染结果到 OSS
5. 更新 EditPlan / EditPlanClip 状态
6. 更新 GenerationTask 进度
5. 创建 GeneratedVideo 记录 + 查重
6. 更新 EditPlan / EditPlanClip 状态
7. 更新 GenerationTask 进度
"""
from __future__ import annotations
import logging
import os
import shutil
import subprocess # nosec B404
import tempfile
from datetime import datetime, timezone
from pathlib import Path
from typing import Optional
from urllib.parse import urlparse
import oss2
from worker_app.celery_app import celery_app
from worker_app.db import SessionLocal
logger = logging.getLogger(__name__)
FFMPEG_BIN = shutil.which("ffmpeg") or "ffmpeg"
FFPROBE_BIN = shutil.which("ffprobe") or "ffprobe"
OUTPUT_WIDTH = 1280
OUTPUT_HEIGHT = 720
OUTPUT_FPS = 25.0
# ── OSS helpers ───────────────────────────────────────────────────────────────
def _oss_settings() -> tuple[str, str, str, str] | None:
"""获取 OSS 配置"""
access_key_id = os.getenv("OSS_ACCESS_KEY_ID")
access_key_secret = os.getenv("OSS_ACCESS_KEY_SECRET")
endpoint = os.getenv("OSS_ENDPOINT")
bucket_name = os.getenv("OSS_BUCKET_NAME")
if not all([access_key_id, access_key_secret, endpoint, bucket_name]):
return None
return access_key_id, access_key_secret, endpoint, bucket_name
def _oss_bucket() -> oss2.Bucket | None:
"""获取 OSS Bucket"""
settings = _oss_settings()
if settings is None:
return None
access_key_id, access_key_secret, endpoint, bucket_name = settings
return oss2.Bucket(oss2.Auth(access_key_id, access_key_secret), endpoint, bucket_name)
def _normalize_storage_key(storage_key_or_url: str) -> str:
"""标准化存储键"""
if storage_key_or_url.startswith(("http://", "https://")):
return urlparse(storage_key_or_url).path.lstrip("/")
return storage_key_or_url.lstrip("/")
def _download_asset(asset_storage_key: str, local_path: Path) -> bool:
"""下载素材文件到本地"""
bucket = _oss_bucket()
if bucket is None:
return False
try:
bucket.get_object_to_file(_normalize_storage_key(asset_storage_key), str(local_path))
return local_path.exists() and local_path.stat().st_size > 0
except Exception:
logger.exception("下载素材失败: %s", asset_storage_key)
return False
def _upload_to_oss(local_path: Path, storage_key: str) -> str | None:
"""上传文件到 OSS,返回公开 URL"""
bucket = _oss_bucket()
if bucket is None:
return None
try:
bucket.put_object_from_file(storage_key, str(local_path))
settings = _oss_settings()
if settings:
_, _, endpoint, bucket_name = settings
return f"https://{bucket_name}.{endpoint.replace('https://', '').replace('http://', '')}/{storage_key}"
return None
except Exception:
logger.exception("上传 OSS 失败: %s", storage_key)
return None
# ── FFmpeg helpers ────────────────────────────────────────────────────────────
def _run_ffmpeg(command: list[str]) -> None:
"""执行 FFmpeg 命令"""
subprocess.run(command, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True) # nosec B603
def _probe_duration(local_path: Path) -> float:
"""获取视频/音频时长"""
try:
result = subprocess.run( # nosec B603
[
FFPROBE_BIN,
"-v",
"error",
"-show_entries",
"format=duration",
"-of",
"default=noprint_wrappers=1:nokey=1",
str(local_path),
],
check=True,
capture_output=True,
text=True,
)
return float(result.stdout.strip())
except Exception:
return 0.0
def _concatenate_clips(
clip_paths: list[Path],
output_path: Path,
transition_effects: list[str] | None = None,
) -> bool:
"""将多个片段拼接为最终视频
使用 FFmpeg concat demuxer 实现。
"""
if not clip_paths:
return False
if len(clip_paths) == 1:
# 单片段直接复制
try:
shutil.copy2(str(clip_paths[0]), str(output_path))
return True
except Exception:
return False
# 多片段:使用 concat demuxer
concat_file = output_path.parent / "concat_list.txt"
try:
with open(concat_file, "w") as f:
for p in clip_paths:
f.write(f"file '{p}'\n")
command = [
FFMPEG_BIN,
"-y",
"-f",
"concat",
"-safe",
"0",
"-i",
str(concat_file),
"-c",
"copy",
str(output_path),
]
_run_ffmpeg(command)
return output_path.exists() and output_path.stat().st_size > 0
except Exception:
logger.exception("拼接片段失败")
return False
finally:
if concat_file.exists():
concat_file.unlink()
# ── 共享工具模块导入 ──────────────────────────────────────────────────────────
from video_processing.dedup_helpers import create_video_record_and_dedup
from video_processing.oss_helpers import (
download_asset,
upload_to_oss,
)
from video_processing.unified_render_service import UnifiedRenderService
# ── Repository imports (延迟导入避免循环依赖) ─────────────────────────────────
@@ -207,11 +69,12 @@ def render_edit_plan(self, plan_id: str) -> dict:
流程:
1. 加载 EditPlan + EditPlanClips
2. 下载各片段素材到临时目录
3. 按 order 顺序拼接片段
2. 下载各片段素材到临时目录,构建 asset_path_map
3. 使用 UnifiedRenderService 按时间线+图层渲染
4. 上传渲染结果到 OSS
5. 更新 EditPlan → completed, EditPlanClips → rendered
6. 更新 GenerationTask 进度
5. 创建 GeneratedVideo 记录 + 查重
6. 更新 EditPlan → completed, EditPlanClips → rendered
7. 更新 GenerationTask 进度
"""
logger.info("开始渲染剪辑计划: plan_id=%s", plan_id)
@@ -246,10 +109,10 @@ def render_edit_plan(self, plan_id: str) -> dict:
gen_task.started_at = datetime.now(timezone.utc)
gen_task_repo.update(gen_task)
# 3. 下载素材并拼接
# 3. 下载素材并构建 asset_path_map
with tempfile.TemporaryDirectory(prefix="edit_plan_") as tmpdir:
tmpdir_path = Path(tmpdir)
clip_paths: list[Path] = []
asset_path_map: dict[str, Path] = {}
rendered_clip_ids: list[str] = []
failed_clip_ids: list[str] = []
@@ -261,18 +124,23 @@ def render_edit_plan(self, plan_id: str) -> dict:
failed_clip_ids.append(clip.id)
continue
if clip.asset_id in asset_path_map:
# 同一素材已下载(多个 clip 共享同一素材)
rendered_clip_ids.append(clip.id)
continue
# 下载素材
ext = Path(clip.asset_id).suffix or ".mp4"
local_path = tmpdir_path / f"clip_{clip.order:04d}{ext}"
if _download_asset(clip.asset_id, local_path):
clip_paths.append(local_path)
if download_asset(clip.asset_id, local_path):
asset_path_map[clip.asset_id] = local_path
rendered_clip_ids.append(clip.id)
else:
clip.mark_failed()
clip_repo.update(clip)
failed_clip_ids.append(clip.id)
if not clip_paths:
if not asset_path_map:
logger.error("所有片段素材下载失败: %s", plan_id)
plan.mark_failed()
plan_repo.update(plan)
@@ -285,42 +153,75 @@ def render_edit_plan(self, plan_id: str) -> dict:
gen_task_repo.update(gen_task)
return {"status": "error", "message": "所有片段素材下载失败"}
# 4. 拼接片段
output_path = tmpdir_path / f"rendered_{plan_id}.mp4"
transition_effects = [c.transition_effect for c in clips if c.asset_id]
success = _concatenate_clips(clip_paths, output_path, transition_effects)
# 4. 使用 UnifiedRenderService 渲染
render_service = UnifiedRenderService(
plan=plan,
clips=clips,
asset_path_map=asset_path_map,
work_dir=tmpdir_path,
output_width=OUTPUT_WIDTH,
output_height=OUTPUT_HEIGHT,
output_fps=int(OUTPUT_FPS),
)
if not success:
logger.error("片段拼接失败: %s", plan_id)
try:
render_result = render_service.render()
except Exception as render_err:
logger.error("渲染失败: %s%s", plan_id, render_err)
plan.mark_failed()
plan_repo.update(plan)
if generation_task_id:
gen_task = gen_task_repo.get(generation_task_id)
if gen_task:
gen_task.status = "failed"
gen_task.error_message = "片段拼接失败"
gen_task.error_message = f"渲染失败: {render_err}"
gen_task.completed_at = datetime.now(timezone.utc)
gen_task_repo.update(gen_task)
return {"status": "error", "message": "片段拼接失败"}
return {"status": "error", "message": f"渲染失败: {render_err}"}
output_path = render_result.output_path
# 5. 上传到 OSS
storage_key = f"rendered/{plan_id}/output.mp4"
output_url = _upload_to_oss(output_path, storage_key)
output_url = upload_to_oss(output_path, storage_key)
# 6. 更新片段状态为 rendered
# 6. 创建 GeneratedVideo 记录 + 查重
project_id = plan.project_id or ""
batch_id = plan.config.get("batch_id", "")
mode = plan.config.get("mode", "edit_plan")
if generation_task_id and project_id:
try:
create_video_record_and_dedup(
generation_task_id=generation_task_id,
project_id=project_id,
batch_id=batch_id,
file_url=output_url or "",
file_size=render_result.file_size,
duration=render_result.duration,
video_path=str(output_path),
mode=mode,
session=db,
width=render_result.width,
height=render_result.height,
fps=OUTPUT_FPS,
)
except Exception as dedup_err:
logger.warning("查重失败(不影响渲染结果): %s", dedup_err)
# 7. 更新片段状态为 rendered
for clip_id in rendered_clip_ids:
clip = clip_repo.get(clip_id)
if clip and clip.status.value == "ready":
clip.mark_rendered()
clip_repo.update(clip)
# 7. 更新 EditPlan 状态为 completed
# 8. 更新 EditPlan 状态为 completed
plan.config["rendered_url"] = output_url or ""
plan.config["rendered_storage_key"] = storage_key
plan.mark_completed()
plan_repo.update(plan)
# 8. 更新 GenerationTask 状态为 completed
# 9. 更新 GenerationTask 状态为 completed
if generation_task_id:
gen_task = gen_task_repo.get(generation_task_id)
if gen_task:
@@ -331,10 +232,11 @@ def render_edit_plan(self, plan_id: str) -> dict:
gen_task_repo.update(gen_task)
logger.info(
"剪辑计划渲染完成: plan_id=%s rendered=%d failed=%d",
"剪辑计划渲染完成: plan_id=%s rendered=%d failed=%d duration=%.1fs",
plan_id,
len(rendered_clip_ids),
len(failed_clip_ids),
render_result.duration,
)
return {
@@ -343,6 +245,7 @@ def render_edit_plan(self, plan_id: str) -> dict:
"rendered_count": len(rendered_clip_ids),
"failed_count": len(failed_clip_ids),
"output_url": output_url,
"duration": render_result.duration,
}
except Exception as exc:
+504 -260
View File
@@ -1,105 +1,217 @@
"""
视频生成任务
支持四种剪辑模式:一镜到底、画中画、口播、口播+画中画
视频生成任务 — 使用 UnifiedRenderService 统一渲染引擎.
支持四种剪辑模式:一镜到底、画中画、口播、口播+画中画。
模式差异体现在虚拟剪辑计划的 clip_type 分布上,渲染引擎不判断模式。
模式 → clip_type 映射:
ONE_TAKE: N 个 main clips
PIP: 1 main + N-1 overlay
VOICE_OVER: N 个 main(config.role=b_roll)
VOICE_PIP: 1 background + 1 corner_voice + N-2 b_roll
"""
from __future__ import annotations
import logging
import os
import shutil
import subprocess # nosec B404
import tempfile
from dataclasses import dataclass, field
from pathlib import Path
from typing import Optional
from urllib.parse import urlparse
from typing import Any, Optional
import oss2
from worker_app.celery_app import celery_app
from worker_app.db import SessionLocal
OUTPUT_WIDTH = 1280
OUTPUT_HEIGHT = 720
OUTPUT_FPS = 25.0
OUTPUT_DURATION_SECONDS = 5.0
FFMPEG_BIN = shutil.which("ffmpeg") or "ffmpeg"
FFPROBE_BIN = shutil.which("ffprobe") or "ffprobe"
GENERATED_FILES_DIR = Path(os.getenv("GENERATED_FILES_DIR", "/app/generated"))
GENERATED_FILES_URL_PREFIX = os.getenv("GENERATED_FILES_URL_PREFIX", "/generated-files")
PUBLIC_API_BASE_URL = os.getenv("PUBLIC_API_BASE_URL", "https://api.xiaoxiajianji.com").rstrip("/")
logger = logging.getLogger(__name__)
def _run_ffmpeg(command: list[str]) -> None:
"""执行 FFmpeg 命令"""
subprocess.run(command, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True) # nosec B603
# ── 状态更新辅助函数 ──────────────────────────────────────────────────────────
def _oss_settings() -> tuple[str, str, str, str] | None:
"""获取 OSS 配置"""
access_key_id = os.getenv("OSS_ACCESS_KEY_ID")
access_key_secret = os.getenv("OSS_ACCESS_KEY_SECRET")
endpoint = os.getenv("OSS_ENDPOINT")
bucket_name = os.getenv("OSS_BUCKET_NAME")
if not all([access_key_id, access_key_secret, endpoint, bucket_name]):
return None
return access_key_id, access_key_secret, endpoint, bucket_name
def _update_task_status(task_id: str, status_action: str, **kwargs) -> bool:
"""更新 GenerationTask 状态(独立 session,异常不向外抛出)。
Args:
task_id: 任务 ID
status_action: 状态动作名,如 "mark_processing" / "mark_completed" / "mark_failed"
**kwargs: 传递给对应方法的参数
def _oss_bucket() -> oss2.Bucket | None:
"""获取 OSS Bucket"""
settings = _oss_settings()
if settings is None:
return None
access_key_id, access_key_secret, endpoint, bucket_name = settings
return oss2.Bucket(oss2.Auth(access_key_id, access_key_secret), endpoint, bucket_name)
def _normalize_storage_key(storage_key_or_url: str) -> str:
"""标准化存储键"""
if storage_key_or_url.startswith(("http://", "https://")):
return urlparse(storage_key_or_url).path.lstrip("/")
return storage_key_or_url.lstrip("/")
def _download_asset(asset_storage_key: str, local_path: Path) -> bool:
"""下载素材文件"""
bucket = _oss_bucket()
if bucket is None:
return False
Returns:
True 表示更新成功,False 表示更新失败
"""
try:
bucket.get_object_to_file(_normalize_storage_key(asset_storage_key), str(local_path))
return local_path.exists() and local_path.stat().st_size > 0
except Exception:
from packages.adapters.sqlalchemy_impl.generation_task_repository import (
SQLAlchemyGenerationTaskRepository,
)
session = SessionLocal()
try:
repo = SQLAlchemyGenerationTaskRepository(session)
task = repo.get(task_id)
if task is None:
logger.warning("更新任务状态失败:任务不存在 task_id=%s", task_id)
return False
action = getattr(task, status_action, None)
if action is None:
logger.warning("未知的状态动作: %s", status_action)
return False
action(**kwargs)
repo.update(task)
logger.info("GenerationTask 状态更新成功: task_id=%s action=%s", task_id, status_action)
return True
finally:
session.close()
except Exception as e:
logger.error(
"更新 GenerationTask 状态异常: task_id=%s action=%s error=%s",
task_id,
status_action,
e,
exc_info=True,
)
return False
def _probe_duration(local_path: Path) -> float:
"""获取视频时长"""
try:
result = subprocess.run(
[
FFPROBE_BIN,
"-v",
"error",
"-show_entries",
"format=duration",
"-of",
"default=noprint_wrappers=1:nokey=1",
str(local_path),
],
check=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
) # nosec B603
return round(float(result.stdout.strip()), 3)
except Exception:
return OUTPUT_DURATION_SECONDS
# ── 共享工具模块导入 ──────────────────────────────────────────────────────────
from video_processing.dedup_helpers import create_video_record_and_dedup
from video_processing.ffmpeg_utils import FFMPEG_BIN, probe_duration, run_ffmpeg
from video_processing.oss_helpers import (
download_asset,
upload_to_oss,
)
from video_processing.unified_render_service import UnifiedRenderService
# ── 虚拟 Plan / Clip(内存中构建,不写数据库) ────────────────────────────────
@dataclass
class _VirtualPlan:
"""内存中的虚拟剪辑计划,供 UnifiedRenderService 使用。"""
id: str
name: str = ""
@dataclass
class _VirtualClip:
"""内存中的虚拟剪辑片段,供 UnifiedRenderService 使用。"""
id: str
plan_id: str = ""
clip_type: str = "main"
order: int = 0
asset_id: str = ""
text_content: str = ""
start_time: float = 0.0
duration: float = 0.0
transition_effect: str = "cut"
status: str = "ready"
config: dict[str, Any] = field(default_factory=dict)
def _build_plan_and_clips_from_task(
task_id: str,
downloaded_paths: list[Path],
mode: str,
) -> tuple[_VirtualPlan, list[_VirtualClip], dict[str, Path]]:
"""根据模式和下载的素材路径,构建虚拟 plan + clips + asset_path_map。
模式 → clip_type 映射:
ONE_TAKE: N 个 main clips
PIP: 1 main + N-1 overlay
VOICE_OVER: N 个 main(config.role=b_roll)
VOICE_PIP: 1 background + 1 corner_voice + N-2 b_roll
Returns:
(virtual_plan, virtual_clips, asset_path_map)
"""
plan = _VirtualPlan(id=task_id, name=f"Generated-{task_id[:8]}")
# 为每个下载路径生成合成 asset_id
asset_path_map: dict[str, Path] = {}
path_to_asset_id: dict[Path, str] = {}
for i, p in enumerate(downloaded_paths):
asset_id = f"gen_{task_id[:8]}_{i:03d}{p.suffix or '.mp4'}"
asset_path_map[asset_id] = p
path_to_asset_id[p] = asset_id
clips: list[_VirtualClip] = []
n = len(downloaded_paths)
if mode == "pip":
# 1 main + N-1 overlay
for i, p in enumerate(downloaded_paths):
clip_type = "main" if i == 0 else "overlay"
clips.append(
_VirtualClip(
id=f"vc_{i:03d}",
plan_id=task_id,
clip_type=clip_type,
order=i,
asset_id=path_to_asset_id[p],
)
)
elif mode == "voice_over":
# N 个 main(config.role=b_roll)
for i, p in enumerate(downloaded_paths):
clips.append(
_VirtualClip(
id=f"vc_{i:03d}",
plan_id=task_id,
clip_type="main",
order=i,
asset_id=path_to_asset_id[p],
config={"role": "b_roll"},
)
)
elif mode == "voice_pip":
# 1 background + 1 corner_voice + N-2 b_roll
for i, p in enumerate(downloaded_paths):
if i == 0:
clip_type = "background"
elif i == 1:
clip_type = "corner_voice"
else:
clip_type = "b_roll"
clips.append(
_VirtualClip(
id=f"vc_{i:03d}",
plan_id=task_id,
clip_type=clip_type,
order=i,
asset_id=path_to_asset_id[p],
)
)
else:
# ONE_TAKE (default): N 个 main clips
for i, p in enumerate(downloaded_paths):
clips.append(
_VirtualClip(
id=f"vc_{i:03d}",
plan_id=task_id,
clip_type="main",
order=i,
asset_id=path_to_asset_id[p],
)
)
return plan, clips, asset_path_map
def _create_fallback_clip(output_path: Path, title: str) -> None:
"""创建 fallback 视频(无素材时)"""
safe_title = title.replace(":", "\\:").replace("'", "\\'")[:80]
_run_ffmpeg(
run_ffmpeg(
[
FFMPEG_BIN,
"-y",
@@ -120,121 +232,275 @@ def _create_fallback_clip(output_path: Path, title: str) -> None:
)
def _mux_audio_track(video_path: Path, audio_path: str, output_path: Path) -> None:
"""将音频轨混入已渲染的视频(后处理步骤)。
使用 FFmpeg 将视频和音频合并,视频时长为准,音频不足则循环,
音频过长则截断。
"""
command = [
FFMPEG_BIN,
"-y",
"-i",
str(video_path),
"-i",
audio_path,
"-c:v",
"copy",
"-c:a",
"aac",
"-b:a",
"192k",
"-shortest",
"-map",
"0:v:0",
"-map",
"1:a:0",
"-movflags",
"+faststart",
str(output_path),
]
run_ffmpeg(command)
def _download_voice_asset(voice_library_id: str, local_path: Path) -> bool:
"""下载配音文件"""
if not voice_library_id:
return False
bucket = _oss_bucket()
if bucket is None:
return False
storage_key = f"voice/{voice_library_id}.mp3"
try:
bucket.get_object_to_file(_normalize_storage_key(storage_key), str(local_path))
return local_path.exists() and local_path.stat().st_size > 0
except Exception:
return False
return download_asset(storage_key, local_path)
def _verify_url_accessible(url: str, timeout: float = 10.0, retries: int = 2) -> bool:
"""HEAD 请求校验 URL 可访问(含重试,防止 OSS 抖动误报)。
Args:
url: 待校验的 URL
timeout: 单次请求超时时间(秒)
retries: 最大重试次数(默认 2 次,首次失败后间隔 1s 重试)
Returns:
True 表示 URL 可访问(HTTP 2xx/3xx),False 表示所有尝试均失败。
"""
import time
import urllib.request
last_error: Exception | None = None
for attempt in range(1 + retries):
try:
req = urllib.request.Request(url, method="HEAD")
req.add_header("User-Agent", "xiaoxia-saas-worker/1.0")
with urllib.request.urlopen(req, timeout=timeout) as resp: # nosec B310
if resp.status < 400:
return True
last_error = Exception(f"HTTP {resp.status}")
except Exception as e:
last_error = e
if attempt < retries:
logger.warning(
"URL 校验失败,1s 后重试: url=%s attempt=%d/%d error=%s",
url,
attempt + 1,
retries,
last_error,
)
time.sleep(1)
logger.warning("URL 可访问性校验最终失败: url=%s error=%s", url, last_error)
return False
def _download_library_assets(
asset_library_id: str,
temp_path: Path,
video_extensions: tuple = (".mp4", ".mov", ".avi", ".mkv", ".webm"),
*,
asset_library_id: str = "",
project_id: str = "",
asset_ids: list[str] | None = None,
) -> list[str]:
"""
从素材库下载视频素材
video_extensions: tuple = (".mp4", ".mov", ".avi", ".mkv", ".webm"),
strict: bool = True,
) -> list[Path]:
"""下载视频素材 — 同时支持素材库模式和项目级模式。
两种查询路径:
- 素材库模式:asset_library_id 非空时,按 asset_library_id + asset_ids 查
- 项目级模式:project_id 非空时,按 project_id + asset_ids 查
- 两者都提供时优先素材库模式;两者都为空时抛异常
归属校验与下载在同一 DB session 中完成,避免多次连接开销(P3-2)。
Args:
asset_library_id: 素材库 ID
temp_path: 临时目录路径
video_extensions: 支持的视频扩展名
asset_library_id: 素材库 ID(可选,与 project_id 二选一)
project_id: 项目 ID(可选,与 asset_library_id 二选一)
asset_ids: 指定素材 ID 列表,为空则下载全部 ready 视频素材
video_extensions: 支持的视频扩展名(保留兼容,当前按 file_type 过滤)
strict: 严格模式(默认 True)。
True — 任何素材下载失败立即抛 RuntimeError
False — 跳过失败素材,返回成功列表(调用方可通过日志感知失败)。
Returns:
下载成功的视频文件路径列表
"""
# 导入模型和会话
try:
from worker_app.db import SessionLocal
下载成功的视频文件 Path 列表
Raises:
ValueError: 当 asset_library_id 和 project_id 都为空时
RuntimeError: strict=True 时任何下载失败;或指定了 asset_ids 但全部下载失败
"""
if not asset_library_id and not project_id:
raise ValueError("asset_library_id 和 project_id 至少需要提供一个")
try:
from packages.adapters.sqlalchemy_impl.models import AssetModel
session = SessionLocal()
try:
# 查询素材库中的视频素材
# 构建查询:根据模式选择不同的过滤条件
query = session.query(AssetModel).filter(
AssetModel.asset_library_id == asset_library_id,
AssetModel.status == "ready",
AssetModel.file_type.in_(["video", "video/mp4", "video/quicktime"]),
)
# 如果指定了 asset_ids,则只下载这些素材
if asset_library_id:
# 素材库模式
query = query.filter(AssetModel.asset_library_id == asset_library_id)
logger.info(
"下载素材库视频: asset_library_id=%s asset_ids=%s",
asset_library_id,
asset_ids or "all",
)
else:
# 项目级模式
query = query.filter(AssetModel.project_id == project_id)
logger.info(
"下载项目级视频: project_id=%s asset_ids=%s",
project_id,
asset_ids or "all",
)
if asset_ids:
query = query.filter(AssetModel.id.in_(asset_ids))
assets = query.order_by(AssetModel.created_at).all()
if not assets:
logger.info(f"No video assets found in library {asset_library_id}")
return []
mode_desc = f"素材库 {asset_library_id}" if asset_library_id else f"项目 {project_id}"
msg = f"未找到视频素材: {mode_desc}, asset_ids={asset_ids or 'all'}"
logger.error(msg)
raise RuntimeError(msg)
downloaded_videos = []
# P3-2: 归属校验合并到同一 session
if asset_ids:
found_ids = {a.id for a in assets}
missing_ids = set(asset_ids) - found_ids
if missing_ids:
raise ValueError(f"素材不存在: asset_ids={sorted(missing_ids)}")
for asset in assets:
if asset_library_id and asset.asset_library_id != asset_library_id:
raise ValueError(
f"素材不属于指定素材库: asset_id={asset.id}, "
f"expected_asset_library_id={asset_library_id}, "
f"actual_asset_library_id={asset.asset_library_id}"
)
if not asset_library_id and project_id and asset.project_id != project_id:
raise ValueError(
f"素材不属于指定项目: asset_id={asset.id}, "
f"expected_project_id={project_id}, "
f"actual_project_id={asset.project_id}"
)
logger.info(
"素材归属校验通过 (同 session): %d 个 asset_ids",
len(asset_ids),
)
downloaded: list[Path] = []
failed_assets: list[str] = []
for i, asset in enumerate(assets):
# 获取文件 URL 或 storage_key
storage_key = asset.file_url if asset.file_url else None
if not storage_key:
failed_assets.append(f"{asset.name}({asset.id})")
logger.warning("素材缺少 file_url, 跳过: asset_id=%s name=%s", asset.id, asset.name)
if strict:
raise RuntimeError(f"素材缺少 file_url: asset_id={asset.id}, name={asset.name}")
continue
local_file = temp_path / f"asset_{i}_{asset.id}.mp4"
if _download_asset(storage_key, local_file):
downloaded_videos.append(str(local_file))
logger.info(f"Downloaded asset: {asset.name} -> {local_file}")
ext = Path(storage_key).suffix or ".mp4"
local_file = temp_path / f"asset_{i:03d}_{asset.id}{ext}"
if download_asset(storage_key, local_file):
downloaded.append(local_file)
logger.info("Downloaded asset: %s -> %s", asset.name, local_file)
else:
logger.warning(f"Failed to download asset: {asset.name}")
failed_assets.append(f"{asset.name}({asset.id})")
logger.warning("Failed to download asset: %s (id=%s)", asset.name, asset.id)
if strict:
raise RuntimeError(f"素材下载失败: asset_id={asset.id}, name={asset.name}")
return downloaded_videos
# 指定了 asset_ids 但全部下载失败 → 无论 strict 与否都报错
if asset_ids and not downloaded:
msg = f"指定的 {len(asset_ids)} 个素材全部下载失败, failed={failed_assets}"
logger.error(msg)
raise RuntimeError(msg)
# 非严格模式有部分失败,记录警告
if failed_assets and not strict:
logger.warning(
"素材下载部分失败 (非严格模式): failed=%s, succeeded=%d",
failed_assets,
len(downloaded),
)
return downloaded
finally:
session.close()
except (ValueError, RuntimeError):
raise
except Exception as e:
logger.error(f"Error downloading library assets: {e}")
return []
logger.error("Error downloading library assets: %s", e, exc_info=True)
raise RuntimeError(f"素材下载异常: {e}") from e
def _process_with_editing_mode(
video_paths: list[str],
audio_path: Optional[str],
mode: str,
output_path: Path,
) -> None:
"""根据剪辑模式处理视频"""
from video_processing.editing_modes import (
EditingMode,
EditingModeConfig,
EditingModeProcessor,
PIPPosition,
)
# ── P1 校验函数 ──────────────────────────────────────────────────────────────
config = EditingModeConfig(
mode=EditingMode(mode),
output_width=OUTPUT_WIDTH,
output_height=OUTPUT_HEIGHT,
output_fps=int(OUTPUT_FPS),
pip_position=PIPPosition.TOP_RIGHT,
pip_scale=0.25,
transition_duration=0.5,
)
processor = EditingModeProcessor(config=config)
processor.process(
video_paths=video_paths,
audio_path=audio_path,
output_path=str(output_path),
)
def _validate_template_exists(template_id: str) -> None:
"""校验 template_id 是否存在且可用。
Raises:
ValueError: template_id 不存在或已禁用时抛出
"""
from packages.adapters.sqlalchemy_impl.models import TemplateModel
session = SessionLocal()
try:
template = (
session.query(TemplateModel)
.filter(
TemplateModel.id == template_id,
TemplateModel.is_active.is_(True),
)
.first()
)
if template is None:
raise ValueError(f"模板不存在或已禁用: template_id={template_id}")
logger.info("模板校验通过: template_id=%s name=%s", template_id, template.name)
finally:
session.close()
# ── Celery Task ──────────────────────────────────────────────────────────────
@celery_app.task(bind=True, name="worker.generate_video", max_retries=2)
def generate_video(self, task_id: str) -> dict:
"""
生成视频任务
"""生成视频任务 — 使用 UnifiedRenderService 统一渲染。
流程:
1. 加载 GenerationTask 信息
2. 从素材库下载视频素材
3. 根据模式构建虚拟 plan + clips
4. 使用 UnifiedRenderService 渲染
5. 如有配音,后处理混音
6. 上传 OSS + 查重
7. 更新 GenerationTask 状态
Args:
task_id: 任务 ID(从数据库加载完整任务信息)
@@ -242,29 +508,35 @@ def generate_video(self, task_id: str) -> dict:
Returns:
生成结果字典
"""
from worker_app.db import SessionLocal
from packages.domain import EditingMode
from packages.adapters.sqlalchemy_impl.generation_task_repository import (
SQLAlchemyGenerationTaskRepository,
)
from packages.domain import EditingMode, GeneratedVideo, GenerationTaskStatus
logger.info("开始生成视频任务: task_id=%s", task_id)
# 从数据库加载任务信息
session = SessionLocal()
try:
from packages.adapters.sqlalchemy_impl.generation_task_repository import (
SQLAlchemyGenerationTaskRepository,
)
task_repo = SQLAlchemyGenerationTaskRepository(session)
gen_task = task_repo.get(task_id)
if gen_task is None:
logger.error("生成任务不存在: task_id=%s", task_id)
return {"status": "failed", "error": f"generation task {task_id} not found"}
project_id = gen_task.project_id
asset_library_id = gen_task.asset_library_id
voice_library_id = gen_task.voice_library_id or ""
template_id = getattr(gen_task, "template_id", "") or ""
mode = gen_task.strategy_id or "one_take"
task_asset_ids = list(gen_task.asset_ids or [])
batch_id = getattr(gen_task, "batch_id", "") or ""
finally:
session.close()
# 标记任务为 running
_update_task_status(task_id, "mark_processing")
try:
editing_mode = EditingMode(mode)
except ValueError:
@@ -274,58 +546,111 @@ def generate_video(self, task_id: str) -> dict:
storage_key = f"generated/projects/{project_id}/tasks/{task_id}/{output_name}"
try:
# P1: template_id 存在性校验
if template_id:
_validate_template_exists(template_id)
# P1: asset_ids 归属校验 — 已合并到 _download_library_assets 同一 sessionP3-2
with tempfile.TemporaryDirectory(prefix="xiaoxia-generation-") as temp_dir:
temp_path = Path(temp_dir)
output_path = temp_path / output_name
# 从素材库下载视频素材(如果任务指定了 asset_ids 则只下载这些)
downloaded_videos = _download_library_assets(asset_library_id, temp_path, asset_ids=task_asset_ids or None)
# 1. 从素材库/项目下载视频素材
downloaded_videos = _download_library_assets(
temp_path,
asset_library_id=asset_library_id,
project_id=project_id,
asset_ids=task_asset_ids or None,
)
audio_path = None
# 2. 下载配音(如有)
audio_path: str | None = None
if voice_library_id:
local_audio = temp_path / "voice.mp3"
if _download_voice_asset(voice_library_id, local_audio):
audio_path = str(local_audio)
if downloaded_videos:
_process_with_editing_mode(
video_paths=downloaded_videos,
audio_path=audio_path,
mode=editing_mode.value,
output_path=output_path,
# 3. 渲染
if not downloaded_videos:
# 素材下载为空(不应到达此处,_download_library_assets 已做校验)
raise RuntimeError(
f"素材下载结果为空: task_id={task_id}, "
f"asset_library_id={asset_library_id}, project_id={project_id}, "
f"asset_ids={task_asset_ids}"
)
else:
_create_fallback_clip(output_path, f"Generated Video {task_id[:8]}")
file_size = output_path.stat().st_size
duration = _probe_duration(output_path)
# 上传到 OSS
bucket = _oss_bucket()
if bucket:
try:
bucket.put_object_from_file(storage_key, str(output_path))
except Exception as oss_err:
logger.warning(f"OSS upload failed: {oss_err}")
# 构建视频 URL
if bucket:
file_url = f"{PUBLIC_API_BASE_URL}/{storage_key}"
else:
file_url = f"{GENERATED_FILES_URL_PREFIX}/{task_id}/{output_name}"
# 创建 GeneratedVideo 记录 + 查重
_create_video_record_and_dedup(
# 构建虚拟 plan + clips + asset_path_map
virtual_plan, virtual_clips, asset_path_map = _build_plan_and_clips_from_task(
task_id=task_id,
project_id=project_id,
batch_id=batch_id,
file_url=file_url,
file_size=file_size,
duration=duration,
video_path=str(output_path),
downloaded_paths=downloaded_videos,
mode=editing_mode.value,
)
# 使用 UnifiedRenderService 渲染
render_service = UnifiedRenderService(
plan=virtual_plan,
clips=virtual_clips,
asset_path_map=asset_path_map,
work_dir=temp_path,
output_width=OUTPUT_WIDTH,
output_height=OUTPUT_HEIGHT,
output_fps=int(OUTPUT_FPS),
)
render_result = render_service.render()
# 4. 如有配音,后处理混音
if audio_path:
final_path = temp_path / f"final-{task_id}.mp4"
try:
_mux_audio_track(render_result.output_path, audio_path, final_path)
# 混音成功,使用混音后的文件
output_path = final_path
except Exception as mux_err:
logger.warning("音频混合失败,使用无音频版本: %s", mux_err)
output_path = render_result.output_path
else:
output_path = render_result.output_path
file_size = output_path.stat().st_size
duration = probe_duration(output_path)
# 5. 上传到 OSS — 失败必须抛异常,不能静默忽略
file_url = upload_to_oss(output_path, storage_key)
if not file_url:
# OSS 未配置或上传失败
raise RuntimeError(
f"OSS 上传失败: task_id={task_id}, storage_key={storage_key}, " f"output_path={output_path}"
)
# HEAD 校验 URL 可访问
if not _verify_url_accessible(file_url):
raise RuntimeError(f"OSS 上传后 URL 不可访问: file_url={file_url}, " f"storage_key={storage_key}")
logger.info("OSS 上传成功: file_url=%s", file_url)
# 6. 创建 GeneratedVideo 记录 + 查重
dedup_session = SessionLocal()
try:
video_count = create_video_record_and_dedup(
generation_task_id=task_id,
project_id=project_id,
batch_id=batch_id,
file_url=file_url,
file_size=file_size,
duration=duration,
video_path=str(output_path),
mode=editing_mode.value,
session=dedup_session,
)
finally:
dedup_session.close()
# 7. 标记任务为 completed
_update_task_status(task_id, "mark_completed", result_count=video_count or 1)
logger.info("视频生成完成: task_id=%s duration=%.2fs file_size=%d", task_id, duration, file_size)
return {
"status": "completed",
"task_id": task_id,
@@ -337,91 +662,10 @@ def generate_video(self, task_id: str) -> dict:
"mode": editing_mode.value,
}
except Exception as error:
logger.error(f"Video generation failed: {error}")
logger.error("Video generation failed: %s", error, exc_info=True)
_update_task_status(task_id, "mark_failed", error_message=str(error))
return {
"status": "failed",
"task_id": task_id,
"error": str(error),
}
def _create_video_record_and_dedup(
*,
task_id: str,
project_id: str,
batch_id: str,
file_url: str,
file_size: int,
duration: float,
video_path: str,
mode: str,
) -> None:
"""创建 GeneratedVideo 记录,计算指纹并执行查重(历史 + 批次)。"""
from uuid import uuid4
from video_processing.dedup import VideoDeduplicator
from worker_app.db import SessionLocal
from packages.adapters.sqlalchemy_impl.generated_video_repository import (
SQLAlchemyGeneratedVideoRepository,
)
from packages.domain import GeneratedVideo
session = SessionLocal()
try:
video_id = uuid4().hex
generated_video = GeneratedVideo(
id=video_id,
project_id=project_id,
generation_task_id=task_id,
name=f"generated-{task_id[:8]}.mp4",
file_url=file_url,
file_size=file_size,
duration=duration,
width=OUTPUT_WIDTH,
height=OUTPUT_HEIGHT,
fps=OUTPUT_FPS,
status="completed",
generation_params={"mode": mode},
)
video_repo = SQLAlchemyGeneratedVideoRepository(session)
video_repo.create(generated_video)
# 计算视频指纹
deduplicator = VideoDeduplicator()
try:
fingerprint = deduplicator.compute_fingerprint(video_path)
except Exception as fp_err:
logger.warning(f"Fingerprint computation failed for {video_id}: {fp_err}")
session.commit()
return
generated_video.video_fingerprint = fingerprint.to_dict()
# (a) 历史成片查重
duplicate_result = deduplicator.check_duplicate(fingerprint, project_id, session)
# (b) 批次内查重(仅当有 batch_id 时)
if not duplicate_result and batch_id:
duplicate_result = deduplicator.check_batch_duplicate(fingerprint, batch_id, video_id, session)
if duplicate_result:
generated_video.is_duplicate = True
generated_video.duplicate_of = duplicate_result["duplicate_of"]
logger.info(
f"Duplicate detected: {video_id} -> {duplicate_result['duplicate_of']} "
f"(reason={duplicate_result['reason']}, similarity={duplicate_result['similarity']:.3f})"
)
else:
generated_video.is_duplicate = False
generated_video.duplicate_of = None
video_repo.update(generated_video)
session.commit()
logger.info(f"GeneratedVideo record created: {video_id} (task={task_id}, dup={generated_video.is_duplicate})")
except Exception as e:
logger.error(f"Failed to create video record / dedup for task {task_id}: {e}")
session.rollback()
finally:
session.close()
+235
View File
@@ -0,0 +1,235 @@
# CI 必需环境变量清单
> 本文档整理小虾 SaaS 项目中所有从环境变量读取的配置项,明确哪些是 CI 测试必须的、哪些是可选的。
> 最后更新:2026-07-09
## 目录
- [一、配置来源说明](#一配置来源说明)
- [二、CI 必需环境变量(P0](#二ci-必需环境变量p0)
- [三、可选环境变量(有默认值)](#三可选环境变量有默认值)
- [四、测试专用环境变量](#四测试专用环境变量)
- [五、Worker 服务环境变量](#五worker-服务环境变量)
- [六、当前 CI 配置对照](#六当前-ci-配置对照)
---
## 一、配置来源说明
项目的环境变量配置主要来自以下几处:
| 来源 | 文件路径 | 说明 |
|------|---------|------|
| API 主配置 | `apps/api/app/config.py` | pydantic `Settings` 类,API 服务核心配置 |
| Worker 配置 | `apps/worker/worker_app/core/config.py` | pydantic `WorkerSettings` 类,Worker 服务配置 |
| 共享配置 | `packages/shared/config.py` | pydantic `SharedSettings` 类,API + Worker 共享配置 |
| 直接读取 | 各模块中 `os.environ` / `os.getenv` | 散落在各业务模块中的直接读取 |
> **注意**pydantic-settings 配置默认 `case_sensitive=False`,即环境变量名不区分大小写,但习惯上使用大写。
---
## 二、CI 必需环境变量(P0
以下变量是 CI 运行测试**必须配置**的,缺失会导致测试启动失败或核心功能异常。
| 变量名 | 用途说明 | 默认值 | 影响范围 |
|--------|---------|--------|---------|
| `DATABASE_URL` | 数据库连接字符串 | `postgresql+psycopg://postgres:postgres@localhost:5432/xiaoxia_saas` | 集成测试、 Alembic 迁移验证 |
| `USE_IN_MEMORY_DB` | 是否使用内存数据库(SQLite) | `false` | 单元测试(设为 `true` 可跳过 PostgreSQL 依赖) |
| `JWT_SECRET_KEY` | JWT 签名密钥,**无安全默认值**,必须显式设置 | `None`(启动校验失败) | 所有涉及认证的 API 测试 |
> **说明**
> - 单元测试通过 `USE_IN_MEMORY_DB=true` 使用 SQLite 内存数据库,无需 PostgreSQL
> - 集成测试需要真实 PostgreSQL,需设置 `DATABASE_URL`
> - `JWT_SECRET_KEY` 在测试文件中通过 `os.environ.setdefault()` 设置了测试用默认值,CI 中可不额外配置,但生产环境必须配置
---
## 三、可选环境变量(有默认值)
以下变量都有合理的默认值,CI 中可以不配置,使用默认值即可。
### 3.1 应用基础配置
| 变量名 | 用途说明 | 默认值 |
|--------|---------|--------|
| `APP_NAME` | 应用名称 | `xiaoxia-saas` |
| `APP_VERSION` | 应用版本号 | `0.1.61` / `unknown` |
| `ENVIRONMENT` | 运行环境标识 | `development` |
| `DEBUG` | 是否开启调试模式 | `true` |
| `APP_BASE_URL` | 应用基础 URL(用于生成邮件链接等) | `http://localhost:3000` |
| `API_HOST` | API 服务绑定地址 | `0.0.0.0` |
| `API_PORT` | API 服务端口 | `8000` |
| `API_PREFIX` | API 路由前缀 | `/api/v1` |
| `APP_ENV` | 环境标识(用于加载 .env.{env} 文件) | `development` |
| `LOG_LEVEL` | 日志级别 | `INFO` |
### 3.2 数据库连接池配置
| 变量名 | 用途说明 | 默认值 |
|--------|---------|--------|
| `DATABASE_POOL_SIZE` | 连接池大小 | `20` |
| `DATABASE_MAX_OVERFLOW` | 最大溢出连接数 | `10` (API) / `40` (Worker) |
| `DATABASE_POOL_TIMEOUT` | 获取连接超时时间(秒) | `30` |
| `DATABASE_POOL_RECYLE` / `DATABASE_POOL_RECYCLE` | 连接回收时间(秒) | `3600` |
| `AUTO_CREATE_SCHEMA` | 是否自动创建表结构 | `false` |
### 3.3 Redis / Celery 配置
| 变量名 | 用途说明 | 默认值 |
|--------|---------|--------|
| `REDIS_URL` | Redis 连接地址 | `redis://localhost:6379/0` |
| `REDIS_MAX_CONNECTION` | Redis 最大连接数 | `50` |
| `ENABLE_REDIS_SESSIONS` | 是否启用 Redis 会话存储 | `false` |
| `CELERY_BROKER_URL` / `BROKER_URL` | Celery Broker 地址 | `redis://localhost:6379/0` |
| `CELERY_RESULT_BACKEND` / `RESULT_BACKEND` | Celery 结果后端 | `redis://localhost:6379/1` |
### 3.4 JWT 配置
| 变量名 | 用途说明 | 默认值 |
|--------|---------|--------|
| `JWT_ALGORITHM` | JWT 签名算法 | `HS256`(隐式默认) |
| `JWT_ACCESS_TOKEN_EXPIRE_MINUTES` | Access Token 过期时间(分钟) | `30`(隐式默认) |
| `JWT_REFRESH_TOKEN_EXPIRE_DAYS` | Refresh Token 过期时间(天) | `30`(隐式默认) |
| `JWT_SECRET_KEY_OLD` | 旧 JWT 密钥(用于密钥轮换) | `None` |
| `SECRET_ROTATION_DAYS` | 密钥轮换建议天数 | `90` |
### 3.5 邮件配置
| 变量名 | 用途说明 | 默认值 |
|--------|---------|--------|
| `ENABLE_EMAIL_DELIVERY` | 是否启用邮件发送 | `false` |
| `SMTP_HOST` | SMTP 服务器地址 | `smtp.gmail.com` |
| `SMTP_PORT` | SMTP 端口 | `587` |
| `SMTP_USER` | SMTP 用户名 | `""`(空) |
| `SMTP_PASSWORD` | SMTP 密码 | `""`(空) |
| `SMTP_FROM_EMAIL` | 发件人邮箱 | `""`(空) |
| `SMTP_FROM_NAME` | 发件人名称 | `小虾 SaaS` |
| `SMTP_USE_TLS` | 是否使用 TLS | `true` |
### 3.6 OSS 阿里云存储配置
| 变量名 | 用途说明 | 默认值 |
|--------|---------|--------|
| `OSS_ENDPOINT` | OSS Endpoint | `oss-cn-hangzhou.aliiyuncs.com` |
| `OSS_ACCESS_KEY_ID` | OSS Access Key ID | `""`(空) |
| `OSS_ACCESS_KEY_SECRET` | OSS Access Key Secret | `""`(空) |
| `OSS_BUCKET_NAME` | OSS Bucket 名称 | `xiaoxia-autocut` |
| `OSS_DIRECT_UPLOAD_MAX_MB` / `MAX_UPLOAD_SIZE_MB` | 直传最大文件大小(MB | `2000` |
| `OSS_DIRECT_UPLOAD_EXPIRE_SECONDS` | 直传签名过期时间(秒) | `900` |
### 3.7 CosyVoice 语音合成配置
| 变量名 | 用途说明 | 默认值 |
|--------|---------|--------|
| `COSYVOICE_API_KEY` | CosyVoice API Key | `""`(空) |
| `COSYVOICE_BASE_URL` | CosyVoice API 地址 | `https://dashscope.aliyuncs.com/api/v1/services/aigc/text2audio` |
| `COSYVOICE_MODEL` | CosyVoice 模型 | `cosyvoice-v1` |
| `COSYVOICE_VOICE` | 默认音色 | `longxiaochun` |
| `COSYVOICE_SAMPLE_RATE` | 采样率 | `22050` |
| `COSYVOICE_FORMAT` | 输出格式 | `mp3` |
### 3.8 CORS 配置
| 变量名 | 用途说明 | 默认值 |
|--------|---------|--------|
| `CORS_ORIGINS_RAW` | CORS 允许的源(逗号分隔) | `http://localhost:3000,http://localhost:5173,http://localhost:8000` |
### 3.9 文件存储 / 生成文件配置
| 变量名 | 用途说明 | 默认值 |
|--------|---------|--------|
| `GENERATED_FILES_DIR` | 生成文件本地存储目录 | `/app/generated` |
| `GENERATED_FILES_URL_PREFIX` | 生成文件访问 URL 前缀 | `/generated-files` |
| `VIDEO_OUTPUT_DIR` | 视频输出目录 | `{tempdir}/video_output` |
| `PUBLIC_API_BASE_URL` | 公开 API 基础 URL | `https://api.xiaoxiajianji.com` |
### 3.10 监控 / 指标配置
| 变量名 | 用途说明 | 默认值 |
|--------|---------|--------|
| `METRICS_AUTH_TOKEN` | Prometheus 指标接口认证 Token | `""`(空,不启用认证) |
### 3.11 内部 API 配置
| 变量名 | 用途说明 | 默认值 |
|--------|---------|--------|
| `INTERNAL_API_KEYS` | 内部 API 调用密钥列表(逗号分隔) | `""`(空) |
---
## 四、测试专用环境变量
以下变量仅在测试或冒烟测试脚本中使用。
| 变量名 | 用途说明 | 默认值 | 使用位置 |
|--------|---------|--------|---------|
| `SMOKE_TEST_PASSWORD` | 冒烟测试用的测试账号密码 | `changeme` | `scripts/smoke_*.py` |
| `MIGRATION_SINCE_REVISION` | 迁移安全检查的起始版本 | `None` | `scripts/check_migration_safety.py` |
| `MIGRATION_DIFF_AGAINST` | 迁移 diff 对比的目标分支/版本 | `None` | `scripts/check_migration_safety.py` |
---
## 五、Worker 服务环境变量
以下变量主要用于 Worker(Celery)服务,CI 的单元/集成测试通常不涉及。
| 变量名 | 用途说明 | 默认值 |
|--------|---------|--------|
| `WORKER_NAME` | Worker 名称 | `xiaoxia-saas-worker` |
| `WORKER_CONCURRENCY` | Worker 并发数 | `4` |
| `WORKER_MAX_TASKS_PER_CHILD` | 每个子进程最大任务数 | `1000` |
---
## 六、当前 CI 配置对照
当前 `.gitea/workflows/ci-cd.yml``validate` job 配置的环境变量:
| 变量名 | CI 配置值 | 是否必需 | 备注 |
|--------|----------|---------|------|
| `DATABASE_URL` | `postgresql+psycopg://postgres:postgres@127.0.0.1:5432/xiaoxia_saas` | ✅ 是 | Job 级别配置 |
| `USE_IN_MEMORY_DB` | `"false"`Job 级) / `"true"`(单元测试 step 级) | ✅ 是 | 单元测试 step 覆盖为 `true` |
| `JWT_SECRET_KEY` | (未配置) | ⚠️ 测试内置 | 测试文件中通过 `setdefault` 设置了测试密钥 |
### 6.1 CI 环境变量现状评估
-**数据库配置完备**DATABASE_URL + USE_IN_MEMORY_DB 已正确配置
-**JWT 密钥**:测试代码内置默认值,CI 可正常运行
- ⚠️ **缺少 Redis 配置**:但当前测试不依赖 Redis,使用默认值即可
- ⚠️ **缺少邮件/OSS/语音配置**:均为可选,CI 中使用空默认值不影响核心测试
### 6.2 建议后续补充
如果未来测试覆盖到以下功能,需要在 CI 中补充对应配置:
1. **Redis 相关测试** → 配置 `REDIS_URL`
2. **邮件发送测试** → 配置 `ENABLE_EMAIL_DELIVERY` 及 SMTP 相关变量
3. **OSS 上传测试** → 配置 OSS 相关变量(或使用 mock)
4. **语音合成测试** → 配置 CosyVoice 相关变量(或使用 mock
---
## 附录:环境变量读取位置索引
### pydantic Settings 类
- `apps/api/app/config.py``Settings` 类(API 主配置)
- `apps/worker/worker_app/core/config.py``WorkerSettings` 类(Worker 配置)
- `packages/shared/config.py``SharedSettings` 类(共享配置)
### 直接 os.environ / os.getenv 读取
| 变量名 | 文件位置 |
|--------|---------|
| `VIDEO_OUTPUT_DIR` | `apps/worker/video_processing/video_compose_service.py``apps/worker/worker_app/tasks/compose_video.py` |
| `INTERNAL_API_KEYS` | `apps/api/app/api/routes/auth.py` |
| `APP_ENV` / `ENV` | `apps/api/app/api/routes/auth.py`、各 config.py 的 `get_settings()` |
| `GENERATED_FILES_DIR` | `apps/worker/worker_app/tasks/generation.py``apps/api/main.py``scripts/cleanup_generated_files.py` |
| `GENERATED_FILES_URL_PREFIX` | `apps/worker/worker_app/tasks/generation.py``apps/api/main.py``apps/api/app/core/storage.py` |
| `PUBLIC_API_BASE_URL` | `apps/worker/worker_app/tasks/generation.py` |
| `METRICS_AUTH_TOKEN` | `apps/api/app/middleware/prometheus_metrics.py` |
| `APP_VERSION` | `apps/api/app/middleware/prometheus_metrics.py` |
| `SMOKE_TEST_PASSWORD` | `scripts/smoke_*.py` |
| `MIGRATION_SINCE_REVISION` | `scripts/check_migration_safety.py` |
| `MIGRATION_DIFF_AGAINST` | `scripts/check_migration_safety.py` |
| `DATABASE_URL` | `alembic/env.py` |
+45 -37
View File
@@ -473,7 +473,7 @@
"name": "id",
"nullable": false,
"primary_key": true,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -481,7 +481,7 @@
"name": "project_id",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -489,7 +489,7 @@
"name": "asset_id",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -783,7 +783,7 @@
"name": "id",
"nullable": false,
"primary_key": true,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -791,7 +791,7 @@
"name": "plan_id",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -815,7 +815,7 @@
"name": "template_clip_config_id",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -823,7 +823,7 @@
"name": "asset_id",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -939,7 +939,7 @@
"name": "id",
"nullable": false,
"primary_key": true,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -947,7 +947,7 @@
"name": "template_id",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -987,7 +987,7 @@
"name": "source_edit_plan_id",
"nullable": true,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -995,7 +995,7 @@
"name": "project_id",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -1003,7 +1003,7 @@
"name": "created_by_user_id",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -1071,7 +1071,7 @@
"name": "id",
"nullable": false,
"primary_key": true,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -1098,6 +1098,14 @@
"type": "VARCHAR(50)",
"unique": false
},
{
"index": false,
"name": "editing_mode",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(20)",
"unique": false
},
{
"index": false,
"name": "config",
@@ -1181,7 +1189,7 @@
"name": "id",
"nullable": false,
"primary_key": true,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -1189,7 +1197,7 @@
"name": "project_id",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -1197,7 +1205,7 @@
"name": "generation_task_id",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -1333,7 +1341,7 @@
"name": "duplicate_of",
"nullable": true,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
}
],
@@ -1378,7 +1386,7 @@
"name": "id",
"nullable": false,
"primary_key": true,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -1386,7 +1394,7 @@
"name": "project_id",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -1394,7 +1402,7 @@
"name": "strategy_id",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -1402,7 +1410,7 @@
"name": "asset_library_id",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -1410,7 +1418,7 @@
"name": "voice_library_id",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -1506,7 +1514,7 @@
"name": "created_by_user_id",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -1514,7 +1522,7 @@
"name": "source_edit_plan_id",
"nullable": true,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -1530,7 +1538,7 @@
"name": "batch_id",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -1619,7 +1627,7 @@
"name": "id",
"nullable": false,
"primary_key": true,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -1627,7 +1635,7 @@
"name": "project_id",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -1635,7 +1643,7 @@
"name": "library_id",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -1667,7 +1675,7 @@
"name": "result_asset_id",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -1729,7 +1737,7 @@
"name": "id",
"nullable": false,
"primary_key": true,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -1737,7 +1745,7 @@
"name": "project_id",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -1825,7 +1833,7 @@
"name": "source_id",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -1833,7 +1841,7 @@
"name": "created_by_user_id",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -1917,7 +1925,7 @@
"name": "id",
"nullable": false,
"primary_key": true,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -1925,7 +1933,7 @@
"name": "owner_user_id",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -2245,7 +2253,7 @@
"name": "id",
"nullable": false,
"primary_key": true,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
@@ -2253,7 +2261,7 @@
"name": "template_id",
"nullable": false,
"primary_key": false,
"type": "VARCHAR(32)",
"type": "VARCHAR(36)",
"unique": false
},
{
-1
View File
@@ -1 +0,0 @@
"""Packages root."""
-1
View File
@@ -1 +0,0 @@
"""Adapters package for external implementations."""
@@ -71,6 +71,7 @@ class SQLAlchemyEditTemplateRepository:
name=template.name,
description=template.description,
template_type=template.template_type,
editing_mode=template.editing_mode,
config=template.config,
preview_url=template.preview_url,
sort_weight=template.sort_weight,
@@ -89,6 +90,7 @@ class SQLAlchemyEditTemplateRepository:
model.name = template.name
model.description = template.description
model.template_type = template.template_type
model.editing_mode = template.editing_mode
model.config = template.config
model.preview_url = template.preview_url
model.sort_weight = template.sort_weight
@@ -128,6 +130,7 @@ class SQLAlchemyEditTemplateRepository:
name=model.name,
description=model.description or "",
template_type=model.template_type or "default",
editing_mode=model.editing_mode or "one_take",
config=model.config or {},
preview_url=model.preview_url or "",
sort_weight=model.sort_weight or 0,
+39 -38
View File
@@ -1,6 +1,6 @@
from datetime import datetime, timezone
from sqlalchemy import JSON, Boolean, Column, DateTime, Float, Integer, String, Text, UniqueConstraint, create_engine
from sqlalchemy import JSON, Boolean, Column, DateTime, Float, Integer, String, Text, UniqueConstraint
from sqlalchemy.orm import declarative_base
Base = declarative_base()
@@ -39,8 +39,8 @@ class UserModel(Base):
class ProjectModel(Base):
__tablename__ = "projects"
id = Column(String(32), primary_key=True)
owner_user_id = Column(String(32), nullable=False, index=True)
id = Column(String(36), primary_key=True)
owner_user_id = Column(String(36), nullable=False, index=True)
name = Column(String(100), nullable=False)
description = Column(Text, nullable=False, default="")
shared_users = Column(JSON, nullable=False, default=list) # 被共享的用户 ID 列表
@@ -122,10 +122,11 @@ class EditTemplateModel(Base):
__tablename__ = "edit_templates"
id = Column(String(32), primary_key=True)
id = Column(String(36), primary_key=True)
name = Column(String(120), nullable=False)
description = Column(Text, nullable=False, default="")
template_type = Column(String(50), nullable=False, default="default", index=True)
editing_mode = Column(String(20), nullable=False, default="one_take")
config = Column(JSON, nullable=False, default=dict)
preview_url = Column(String(1000), nullable=False, default="")
sort_weight = Column(Integer, nullable=False, default=0, index=True)
@@ -142,15 +143,15 @@ class EditPlanModel(Base):
__tablename__ = "edit_plans"
id = Column(String(32), primary_key=True)
template_id = Column(String(32), nullable=False, index=True)
id = Column(String(36), primary_key=True)
template_id = Column(String(36), nullable=False, index=True)
name = Column(String(200), nullable=False)
status = Column(String(20), nullable=False, default="draft", index=True)
total_duration = Column(Float, nullable=False, default=0.0)
config = Column(JSON, nullable=False, default=dict)
source_edit_plan_id = Column(String(32), nullable=True, index=True)
project_id = Column(String(32), nullable=False, default="", index=True)
created_by_user_id = Column(String(32), nullable=False, default="", index=True)
source_edit_plan_id = Column(String(36), nullable=True, index=True)
project_id = Column(String(36), nullable=False, default="", index=True)
created_by_user_id = Column(String(36), nullable=False, default="", index=True)
created_at = Column(DateTime, nullable=False, default=lambda: datetime.now(timezone.utc))
updated_at = Column(DateTime, nullable=False, default=lambda: datetime.now(timezone.utc))
@@ -163,8 +164,8 @@ class TemplateClipConfigModel(Base):
__tablename__ = "template_clip_configs"
id = Column(String(32), primary_key=True)
template_id = Column(String(32), nullable=False, index=True)
id = Column(String(36), primary_key=True)
template_id = Column(String(36), nullable=False, index=True)
clip_type = Column(String(20), nullable=False, index=True)
order = Column(Integer, nullable=False)
min_duration = Column(Float, nullable=False, default=0.0)
@@ -185,12 +186,12 @@ class EditPlanClipModel(Base):
__tablename__ = "edit_plan_clips"
id = Column(String(32), primary_key=True)
plan_id = Column(String(32), nullable=False, index=True)
id = Column(String(36), primary_key=True)
plan_id = Column(String(36), nullable=False, index=True)
clip_type = Column(String(20), nullable=False, index=True)
order = Column(Integer, nullable=False)
template_clip_config_id = Column(String(32), nullable=False, default="", index=True)
asset_id = Column(String(32), nullable=False, default="", index=True)
template_clip_config_id = Column(String(36), nullable=False, default="", index=True)
asset_id = Column(String(36), nullable=False, default="", index=True)
text_content = Column(Text, nullable=False, default="")
start_time = Column(Float, nullable=False, default=0.0)
duration = Column(Float, nullable=False, default=0.0)
@@ -204,13 +205,13 @@ class EditPlanClipModel(Base):
class IngestJobModel(Base):
__tablename__ = "ingest_jobs"
id = Column(String(32), primary_key=True)
project_id = Column(String(32), nullable=False, index=True)
library_id = Column(String(32), nullable=False, index=True)
id = Column(String(36), primary_key=True)
project_id = Column(String(36), nullable=False, index=True)
library_id = Column(String(36), nullable=False, index=True)
storage_key = Column(String(255), nullable=False)
status = Column(String(20), nullable=False, default="pending")
error_message = Column(Text, nullable=False, default="")
result_asset_id = Column(String(32), nullable=False, default="")
result_asset_id = Column(String(36), nullable=False, default="")
file_hash = Column(String(64), nullable=True, index=True)
created_at = Column(DateTime, nullable=False, default=lambda: datetime.now(timezone.utc))
updated_at = Column(DateTime, nullable=False, default=lambda: datetime.now(timezone.utc))
@@ -219,9 +220,9 @@ class IngestJobModel(Base):
class ClassificationJobModel(Base):
__tablename__ = "classification_jobs"
id = Column(String(32), primary_key=True)
project_id = Column(String(32), nullable=False, index=True)
asset_id = Column(String(32), nullable=False, index=True)
id = Column(String(36), primary_key=True)
project_id = Column(String(36), nullable=False, index=True)
asset_id = Column(String(36), nullable=False, index=True)
status = Column(String(20), nullable=False, default="pending")
classification = Column(String(50), nullable=False, default="")
confidence = Column(Float, nullable=False, default=0.0)
@@ -233,11 +234,11 @@ class ClassificationJobModel(Base):
class GenerationTaskModel(Base):
__tablename__ = "generation_tasks"
id = Column(String(32), primary_key=True)
project_id = Column(String(32), nullable=False, default="", index=True)
strategy_id = Column(String(32), nullable=False, default="")
asset_library_id = Column(String(32), nullable=False, default="", index=True)
voice_library_id = Column(String(32), nullable=False, default="")
id = Column(String(36), primary_key=True)
project_id = Column(String(36), nullable=False, default="", index=True)
strategy_id = Column(String(36), nullable=False, default="")
asset_library_id = Column(String(36), nullable=False, default="", index=True)
voice_library_id = Column(String(36), nullable=False, default="")
template_id = Column(String(36), nullable=False, default="", index=True)
asset_ids = Column(JSON, nullable=False, default=list)
title_ids = Column(JSON, nullable=False, default=list)
@@ -251,10 +252,10 @@ class GenerationTaskModel(Base):
error_message = Column(Text, nullable=False, default="")
started_at = Column(DateTime, nullable=True)
completed_at = Column(DateTime, nullable=True)
created_by_user_id = Column(String(32), nullable=False, default="", index=True)
source_edit_plan_id = Column(String(32), nullable=True, index=True)
created_by_user_id = Column(String(36), nullable=False, default="", index=True)
source_edit_plan_id = Column(String(36), nullable=True, index=True)
asset_select_mode = Column(String(20), nullable=False, default="")
batch_id = Column(String(32), nullable=False, default="", index=True)
batch_id = Column(String(36), nullable=False, default="", index=True)
extra_meta = Column("metadata", JSON, nullable=False, default=dict)
created_at = Column(DateTime, nullable=False, default=lambda: datetime.now(timezone.utc))
@@ -262,9 +263,9 @@ class GenerationTaskModel(Base):
class GeneratedVideoModel(Base):
__tablename__ = "generated_videos"
id = Column(String(32), primary_key=True)
project_id = Column(String(32), nullable=False, index=True)
generation_task_id = Column(String(32), nullable=False, index=True)
id = Column(String(36), primary_key=True)
project_id = Column(String(36), nullable=False, index=True)
generation_task_id = Column(String(36), nullable=False, index=True)
name = Column(String(255), nullable=False)
# file_url: 完整可访问的 URL,用于客户端直接访问视频
file_url = Column(String(1000), nullable=False)
@@ -283,7 +284,7 @@ class GeneratedVideoModel(Base):
updated_at = Column(DateTime, nullable=True)
video_fingerprint = Column(Text, nullable=True)
is_duplicate = Column(Boolean, nullable=False, default=False)
duplicate_of = Column(String(32), nullable=True)
duplicate_of = Column(String(36), nullable=True)
class TitleLibraryModel(Base):
@@ -450,8 +451,8 @@ class JobModel(Base):
__tablename__ = "jobs"
id = Column(String(32), primary_key=True)
project_id = Column(String(32), nullable=False, index=True)
id = Column(String(36), primary_key=True)
project_id = Column(String(36), nullable=False, index=True)
job_type = Column(String(30), nullable=False, index=True)
status = Column(String(20), nullable=False, default="pending", index=True)
progress = Column(Float, nullable=False, default=0.0)
@@ -462,8 +463,8 @@ class JobModel(Base):
retry_count = Column(Integer, nullable=False, default=0)
max_retries = Column(Integer, nullable=False, default=3)
celery_task_id = Column(String(100), nullable=False, default="")
source_id = Column(String(32), nullable=False, default="", index=True)
created_by_user_id = Column(String(32), nullable=False, default="", index=True)
source_id = Column(String(36), nullable=False, default="", index=True)
created_by_user_id = Column(String(36), nullable=False, default="", index=True)
started_at = Column(DateTime, nullable=True)
completed_at = Column(DateTime, nullable=True)
created_at = Column(DateTime, nullable=False, default=lambda: datetime.now(timezone.utc))
-1
View File
@@ -27,7 +27,6 @@ from .generated_videos import (
from .generation_tasks import (
CreateGenerationTaskCommand,
CreateGenerationTaskUseCase,
GetGenerationTaskUseCase,
)
from .ingest_jobs import SubmitIngestJobCommand, SubmitIngestJobUseCase
from .jobs import (
+1 -2
View File
@@ -12,10 +12,9 @@ JWT 处理器委托层
payload = jwt_handler.verify_access_token(token)
"""
from datetime import datetime, timedelta
from typing import Any, Dict, Optional
from packages.application.auth.jwt_service import JWTConfig, JWTService, TokenType
from packages.application.auth.jwt_service import JWTConfig, JWTService
class JWTHandler:
+4 -4
View File
@@ -208,10 +208,10 @@ def _get_jwt_service():
kw = dict(secret_key=settings.JWT_SECRET_KEY)
if hasattr(settings, "JWT_ALGORITHM"):
kw["algorithm"] = settings.JWT_ALGORITHM
if hasattr(settings, "ACCESS_TOKEN_EXPIRE_MINUTES"):
kw["access_token_expire_minutes"] = settings.ACCESS_TOKEN_EXPIRE_MINUTES
if hasattr(settings, "REFRESH_TOKEN_EXPIRE_DAYS"):
kw["refresh_token_expire_days"] = settings.REFRESH_TOKEN_EXPIRE_DAYS
if hasattr(settings, "JWT_ACCESS_TOKEN_EXPIRE_MINUTES"):
kw["access_token_expire_minutes"] = settings.JWT_ACCESS_TOKEN_EXPIRE_MINUTES
if hasattr(settings, "JWT_REFRESH_TOKEN_EXPIRE_DAYS"):
kw["refresh_token_expire_days"] = settings.JWT_REFRESH_TOKEN_EXPIRE_DAYS
_jwt_service_instance = JWTService(JWTConfig(**kw))
return _jwt_service_instance
+1 -1
View File
@@ -293,7 +293,7 @@ class LogoutUseCase:
try:
if request.logout_all_devices:
# 删除所有设备的 session
count = self.session_store.delete_all_user_sessions(request.user_id)
self.session_store.delete_all_user_sessions(request.user_id)
return True, None
else:
# 删除当前 session
@@ -85,8 +85,6 @@ class PasswordHasher:
True 如果需要重新哈希
"""
try:
hashed_bytes = hashed_password.encode("utf-8")
current_rounds = bcrypt.getsalt(hashed_bytes)
# 提取当前的 cost factor
# bcrypt hash 格式: $2b$rounds$salt+hash
@@ -3,7 +3,7 @@
"""
import secrets
from datetime import datetime, timedelta, timezone
from datetime import datetime, timezone
from typing import Optional
from uuid import uuid4
+1 -1
View File
@@ -3,7 +3,7 @@
"""
from math import ceil
from typing import Generic, List, Optional, TypeVar
from typing import Generic, List, TypeVar
from pydantic import BaseModel, Field
-2
View File
@@ -7,9 +7,7 @@ from __future__ import annotations
import logging
from dataclasses import dataclass, field
from datetime import datetime, timezone
from typing import Any
from uuid import uuid4
from packages.domain.job import Job, JobStatus, JobType
from packages.ports.job_repository import JobRepository
-1
View File
@@ -9,7 +9,6 @@ from typing import List, Optional
from packages.adapters.sqlalchemy_impl.recipe_repository import SQLAlchemyRecipeRepository
from packages.application.recipe.commands import (
CreateRecipeCommand,
RecipeItemCommand,
UpdateRecipeCommand,
)
from packages.domain.recipe import Recipe, RecipeItem
-1
View File
@@ -1 +0,0 @@
"""TTS Job application layer."""
@@ -148,7 +148,6 @@ class TTSStreamingService:
# 并发合成所有分段,按顺序流式推送
queue: asyncio.Queue[tuple[int, Optional[bytes], Optional[str]]] = asyncio.Queue()
completed_count = 0
async def _synthesize_one(idx: int, seg_text: str) -> None:
"""合成单个分段并放入队列。"""
+2 -2
View File
@@ -25,9 +25,9 @@ from packages.application.cosyvoice_service import (
CosyVoiceError,
CosyVoiceService,
)
from packages.application.tts_job.audio_merger import AudioMergeError, AudioMerger
from packages.application.tts_job.audio_merger import AudioMerger
from packages.application.tts_job.text_splitter import split_text
from packages.domain.tts_job import TTSJob, TTSJobStatus
from packages.domain.tts_job import TTSJob
from packages.ports.tts_job_repository import TTSJobRepository
from packages.shared.storage import SharedStorageService, get_shared_storage_service
@@ -2,7 +2,6 @@
from __future__ import annotations
import uuid
from typing import List, Optional
from packages.domain.voice_clone_profile import VoiceCloneProfile
+2 -3
View File
@@ -10,7 +10,7 @@
from __future__ import annotations
import logging
from typing import Any, Optional
from typing import Optional
from packages.application.cosyvoice_service import (
CosyVoiceAuthError,
@@ -21,9 +21,8 @@ from packages.application.voice_clone.use_cases import (
CreateVoiceCloneUseCase,
RetryVoiceCloneUseCase,
VoiceCloneNotFoundError,
VoiceCloneNotRetryableError,
)
from packages.domain.voice_clone_profile import VoiceCloneProfile, VoiceCloneStatus
from packages.domain.voice_clone_profile import VoiceCloneProfile
from packages.ports.voice_clone_profile_repository import VoiceCloneProfileRepository
logger = logging.getLogger(__name__)
-1
View File
@@ -13,7 +13,6 @@ else:
pass
from typing import Any
from uuid import uuid4
+45 -8
View File
@@ -8,8 +8,9 @@
from __future__ import annotations
import copy
from enum import Enum
from typing import List, Optional
from typing import Optional
from pydantic import BaseModel, Field
@@ -129,24 +130,29 @@ class EditPlanConfigSchema(BaseModel):
用于 API 层校验和默认值填充。所有子结构均可选,
未传入时使用各自默认值。
editing_mode 记录计划使用的剪辑模式。
"""
cover: CoverConfig = Field(default_factory=CoverConfig, description="封面配置")
title: TitleConfig = Field(default_factory=TitleConfig, description="标题配置")
subtitle: SubtitleConfig = Field(default_factory=SubtitleConfig, description="字幕配置")
bgm: BGMConfig = Field(default_factory=BGMConfig, description="BGM 配置")
editing_mode: str = Field(default="one_take", description="剪辑模式")
class EditTemplateConfigSchema(BaseModel):
"""EditTemplate.config 完整结构
模板级别的默认配置,创建计划时可作为初始值继承。
editing_mode 指定模板对应的剪辑模式,transition_enabled 控制是否启用转场。
"""
cover: CoverConfig = Field(default_factory=CoverConfig, description="封面默认配置")
title: TitleConfig = Field(default_factory=TitleConfig, description="标题默认配置")
subtitle: SubtitleConfig = Field(default_factory=SubtitleConfig, description="字幕默认配置")
bgm: BGMConfig = Field(default_factory=BGMConfig, description="BGM 默认配置")
editing_mode: str = Field(default="one_take", description="剪辑模式")
transition_enabled: bool = Field(default=True, description="是否启用转场")
# ── 默认值常量 ────────────────────────────────────────────────────────────────
@@ -183,9 +189,13 @@ DEFAULT_EDIT_PLAN_CONFIG: dict = {
"asset_id": "",
"volume": 0.3,
},
"editing_mode": "one_take",
}
DEFAULT_EDIT_TEMPLATE_CONFIG: dict = DEFAULT_EDIT_PLAN_CONFIG.copy()
DEFAULT_EDIT_TEMPLATE_CONFIG: dict = {
**DEFAULT_EDIT_PLAN_CONFIG,
"transition_enabled": True,
}
# ── 工具函数 ──────────────────────────────────────────────────────────────────
@@ -197,9 +207,7 @@ def normalize_plan_config(raw: dict | None) -> dict:
用于创建/更新计划时确保 config 结构完整。
"""
if raw is None:
return DEFAULT_EDIT_PLAN_CONFIG.copy()
import copy
return copy.deepcopy(DEFAULT_EDIT_PLAN_CONFIG)
base = copy.deepcopy(DEFAULT_EDIT_PLAN_CONFIG)
@@ -209,14 +217,43 @@ def normalize_plan_config(raw: dict | None) -> dict:
base[section_key] = {}
base[section_key].update(raw[section_key])
# editing_mode 顶层字段
if "editing_mode" in raw and isinstance(raw["editing_mode"], str):
base["editing_mode"] = raw["editing_mode"]
# 保留非标准字段(如 generation_task_id
for key, value in raw.items():
if key not in ("cover", "title", "subtitle", "bgm"):
if key not in ("cover", "title", "subtitle", "bgm", "editing_mode"):
base[key] = value
return base
def normalize_template_config(raw: dict | None) -> dict:
"""将模板原始 config dict 标准化。逻辑同 normalize_plan_config。"""
return normalize_plan_config(raw)
"""将模板原始 config dict 标准化。
在 plan config 基础上额外支持 transition_enabled 字段。
"""
if raw is None:
return copy.deepcopy(DEFAULT_EDIT_TEMPLATE_CONFIG)
base = copy.deepcopy(DEFAULT_EDIT_TEMPLATE_CONFIG)
for section_key in ("cover", "title", "subtitle", "bgm"):
if section_key in raw and isinstance(raw[section_key], dict):
if section_key not in base:
base[section_key] = {}
base[section_key].update(raw[section_key])
# 顶层字段
if "editing_mode" in raw and isinstance(raw["editing_mode"], str):
base["editing_mode"] = raw["editing_mode"]
if "transition_enabled" in raw and isinstance(raw["transition_enabled"], bool):
base["transition_enabled"] = raw["transition_enabled"]
# 保留非标准字段
for key, value in raw.items():
if key not in ("cover", "title", "subtitle", "bgm", "editing_mode", "transition_enabled"):
base[key] = value
return base
+16
View File
@@ -18,6 +18,8 @@ else:
from typing import Any
from uuid import uuid4
from .editing_mode import EditingMode
class EditTemplateStatus(StrEnum):
"""模板状态"""
@@ -26,18 +28,25 @@ class EditTemplateStatus(StrEnum):
INACTIVE = "inactive"
_VALID_EDITING_MODES = {m.value for m in EditingMode}
@dataclass(slots=True)
class EditTemplate:
"""Phase 8 剪辑模板实体
全局模板库中的模板,定义剪辑风格、配置参数和预览信息。
不绑定到具体项目,可被多个 EditPlan 引用。
editing_mode 指定模板对应的剪辑模式(one_take / pip / voice_over / voice_pip),
决定剪辑计划生成时的片段结构。
"""
id: str
name: str
description: str = ""
template_type: str = "default"
editing_mode: str = EditingMode.ONE_TAKE.value
config: dict[str, Any] = field(default_factory=dict)
preview_url: str = ""
sort_weight: int = 0
@@ -52,6 +61,7 @@ class EditTemplate:
*,
description: str = "",
template_type: str = "default",
editing_mode: str = EditingMode.ONE_TAKE.value,
config: dict[str, Any] | None = None,
preview_url: str = "",
sort_weight: int = 0,
@@ -61,11 +71,17 @@ class EditTemplate:
clean_name = name.strip()
if not clean_name:
raise ValueError("模板名称不能为空")
clean_mode = editing_mode.strip() or EditingMode.ONE_TAKE.value
if clean_mode not in _VALID_EDITING_MODES:
raise ValueError(
f"无效的 editing_mode: {clean_mode}" f"允许值: {', '.join(sorted(_VALID_EDITING_MODES))}"
)
return cls(
id=uuid4().hex,
name=clean_name,
description=description.strip(),
template_type=template_type.strip() or "default",
editing_mode=clean_mode,
config=config or {},
preview_url=preview_url.strip(),
sort_weight=sort_weight,
+160
View File
@@ -1,3 +1,11 @@
"""GenerationTask 领域模型 — 视频生成任务.
状态机:
pending → running → completed
↘ failed → pending (重试)
↘ cancelled
"""
from __future__ import annotations
import sys
@@ -17,11 +25,43 @@ from uuid import uuid4
class GenerationTaskStatus(StrEnum):
"""生成任务状态枚举。"""
PENDING = "pending"
"""待处理(任务已创建,等待执行)"""
RUNNING = "running"
"""运行中(正在生成视频)"""
COMPLETED = "completed"
"""已完成(视频生成成功)"""
FAILED = "failed"
"""失败(生成失败)"""
CANCELLED = "cancelled"
"""已取消(用户取消或系统取消)"""
# 终态集合
TERMINAL_STATUSES = frozenset(
{GenerationTaskStatus.COMPLETED, GenerationTaskStatus.FAILED, GenerationTaskStatus.CANCELLED}
)
# 合法状态转换
_VALID_TRANSITIONS: dict[GenerationTaskStatus, set[GenerationTaskStatus]] = {
GenerationTaskStatus.PENDING: {
GenerationTaskStatus.RUNNING,
GenerationTaskStatus.FAILED,
GenerationTaskStatus.CANCELLED,
},
GenerationTaskStatus.RUNNING: {
GenerationTaskStatus.COMPLETED,
GenerationTaskStatus.FAILED,
GenerationTaskStatus.CANCELLED,
},
GenerationTaskStatus.FAILED: {GenerationTaskStatus.PENDING}, # 重试回到 pending
}
@dataclass(slots=True)
@@ -83,3 +123,123 @@ class GenerationTask:
asset_select_mode=asset_select_mode,
batch_id=batch_id,
)
# ── 状态查询 ────────────────────────────────────────────────────────────
@property
def is_terminal(self) -> bool:
"""是否处于终态(completed / failed / cancelled)。"""
return self.status in TERMINAL_STATUSES
@property
def is_completed(self) -> bool:
"""是否已完成。"""
return self.status == GenerationTaskStatus.COMPLETED
@property
def is_failed(self) -> bool:
"""是否失败。"""
return self.status == GenerationTaskStatus.FAILED
@property
def is_running(self) -> bool:
"""是否运行中。"""
return self.status == GenerationTaskStatus.RUNNING
# ── 状态转换 ────────────────────────────────────────────────────────────
def transition_to(self, new_status: GenerationTaskStatus | str) -> None:
"""执行状态转换。
Args:
new_status: 目标状态
Raises:
ValueError: 非法状态转换
"""
if isinstance(new_status, str):
try:
new_status = GenerationTaskStatus(new_status)
except ValueError:
raise ValueError(f"无效状态: {new_status}")
allowed = _VALID_TRANSITIONS.get(self.status, set())
if new_status not in allowed:
raise ValueError(
f"非法状态转换: {self.status.value}{new_status.value}"
f"允许: {{{', '.join(sorted(s.value for s in allowed))}}}"
)
self.status = new_status
def mark_processing(self) -> None:
"""标记为处理中(pending → running)。
设置 started_at,清除 error_message。
Raises:
ValueError: 当前状态不允许转换到 running
"""
self.transition_to(GenerationTaskStatus.RUNNING)
self.started_at = datetime.now(timezone.utc)
self.error_message = ""
def mark_completed(self, result_count: int = 1) -> None:
"""标记为已完成(running → completed)。
设置 completed_at、progress=100.0、result_count,清除 error_message。
Args:
result_count: 生成的视频数量,默认为 1
Raises:
ValueError: 当前状态不允许转换到 completed
"""
self.transition_to(GenerationTaskStatus.COMPLETED)
self.completed_at = datetime.now(timezone.utc)
self.progress = 100.0
self.result_count = result_count
self.error_message = ""
def mark_failed(self, error_message: str) -> None:
"""标记为失败(pending / running → failed)。
设置 error_message、completed_at。
Args:
error_message: 错误信息
Raises:
ValueError: 当前状态不允许转换到 failed
"""
self.transition_to(GenerationTaskStatus.FAILED)
self.error_message = error_message
self.completed_at = datetime.now(timezone.utc)
def mark_cancelled(self) -> None:
"""标记为已取消(pending / running → cancelled)。
设置 completed_at。
Raises:
ValueError: 当前状态不允许转换到 cancelled
"""
self.transition_to(GenerationTaskStatus.CANCELLED)
self.completed_at = datetime.now(timezone.utc)
def mark_pending_from_failed(self) -> None:
"""从失败状态重置为待处理(用于重试)。
清除 error_message、started_at、completed_at、progress。
Raises:
ValueError: 当前状态不是 failed
"""
if self.status != GenerationTaskStatus.FAILED:
raise ValueError(f"只有 failed 状态的任务可以重置为 pending,当前状态: {self.status.value}")
self.transition_to(GenerationTaskStatus.PENDING)
self.error_message = ""
self.started_at = None
self.completed_at = None
self.progress = 0.0
self.result_count = 0
+1 -1
View File
@@ -24,7 +24,7 @@ from __future__ import annotations
import logging
from dataclasses import dataclass, field
from typing import Any, Dict, Optional, Set
from typing import Dict, Optional
logger = logging.getLogger(__name__)
+1 -1
View File
@@ -14,7 +14,7 @@ from __future__ import annotations
import logging
from dataclasses import dataclass, field
from enum import Enum
from typing import Any, Callable, Dict, List, Optional, Set
from typing import Any, Callable, Dict, List, Optional
logger = logging.getLogger(__name__)
+1 -1
View File
@@ -2,7 +2,7 @@
from abc import ABC, abstractmethod
from packages.domain import AssetLibrary, AssetLibraryKind
from packages.domain import AssetLibrary
class AssetLibraryRepository(ABC):
+1 -1
View File
@@ -24,7 +24,7 @@ class SharedSettings(BaseSettings):
celery_result_backend: str = "redis://localhost:6379/1"
# OSS Aliyun
oss_endpoint: str = "oss-cn-hangzhou.aliiyuncs.com"
oss_endpoint: str = "oss-cn-hangzhou.aliyuncs.com"
oss_access_key_id: str = ""
oss_access_key_secret: str = ""
oss_bucket_name: str = "xiaoxia-autocut"
+5
View File
@@ -1,3 +1,8 @@
[pytest]
pythonpath = . apps/api apps/worker
testpaths = tests
# ===== 覆盖率配置 =====
# 覆盖率统计范围(供 --cov 使用时的默认源)
# 注意:addopts 不默认开启 --cov,避免影响本地开发调试
# CI 中通过命令行参数显式开启:--cov=apps --cov-report=term --cov-report=xml --cov-fail-under=50
+3
View File
@@ -24,6 +24,9 @@ celery==5.4.0
# 对象存储
oss2==2.18.4
cryptography==46.0.5
# 覆盖系统预装的旧版pyOpenSSL,与cryptography 46.0.5兼容
pyOpenSSL==26.2.0
# HTTP 客户端
httpx==0.27.2
+1
View File
@@ -247,3 +247,4 @@ def main() -> int:
if __name__ == "__main__":
sys.exit(main())
@@ -0,0 +1,236 @@
"""四模式渲染集成测试.
验证 4 种剪辑模式(ONE_TAKE / PIP / VOICE_OVER / VOICE_PIP)通过
_build_plan_and_clips_from_task + UnifiedRenderService 的完整渲染流程。
需要 ffmpeg 可用;CI 无 ffmpeg 时自动跳过。
"""
from __future__ import annotations
import shutil
import subprocess
import tempfile
from pathlib import Path
import pytest
from video_processing.unified_render_service import (
RenderResult,
UnifiedRenderService,
_resolve_layer_role,
)
from worker_app.tasks.generation import _build_plan_and_clips_from_task
pytestmark = pytest.mark.skipif(
not shutil.which("ffmpeg"),
reason="ffmpeg not available",
)
# ── 辅助函数 ──────────────────────────────────────────────────────────────────
def _generate_test_video(path: Path, duration: float = 3.0, color: str = "red") -> None:
"""生成一个纯色测试视频。"""
cmd = [
"ffmpeg",
"-y",
"-f",
"lavfi",
"-i",
f"color=c={color}:s=640x360:d={duration}:r=25",
"-c:v",
"libx264",
"-pix_fmt",
"yuv420p",
"-movflags",
"+faststart",
str(path),
]
subprocess.run(cmd, check=True, capture_output=True, timeout=30)
def _render_with_mode(
mode: str,
num_clips: int = 3,
duration: float = 2.0,
) -> tuple[RenderResult, Path]:
"""用指定模式生成测试视频并渲染,返回 (result, work_dir)。
调用方负责清理 work_dir。
"""
work_dir = Path(tempfile.mkdtemp(prefix="test_4mode_"))
# 生成测试视频素材
colors = ["red", "green", "blue", "yellow", "purple"]
downloaded_paths: list[Path] = []
for i in range(num_clips):
p = work_dir / f"test_{i:03d}.mp4"
_generate_test_video(p, duration=duration, color=colors[i % len(colors)])
downloaded_paths.append(p)
# 构建虚拟 plan + clips
task_id = f"test_task_{mode}"
plan, clips, asset_path_map = _build_plan_and_clips_from_task(
task_id=task_id,
downloaded_paths=downloaded_paths,
mode=mode,
)
# 渲染
service = UnifiedRenderService(
plan=plan,
clips=clips,
asset_path_map=asset_path_map,
work_dir=work_dir,
output_width=640,
output_height=360,
output_fps=25,
)
result = service.render()
return result, work_dir
# ── 测试 _build_plan_and_clips_from_task ──────────────────────────────────────
class TestBuildPlanAndClips:
"""测试 4 种模式的虚拟 plan 构建。"""
def _make_paths(self, n: int) -> list[Path]:
return [Path(f"/tmp/test_{i}.mp4") for i in range(n)]
def test_one_take_mode(self):
paths = self._make_paths(3)
plan, clips, asset_map = _build_plan_and_clips_from_task("t1", paths, "one_take")
assert plan.id == "t1"
assert len(clips) == 3
assert all(c.clip_type == "main" for c in clips)
assert len(asset_map) == 3
def test_pip_mode(self):
paths = self._make_paths(3)
plan, clips, asset_map = _build_plan_and_clips_from_task("t2", paths, "pip")
assert len(clips) == 3
assert clips[0].clip_type == "main"
assert clips[1].clip_type == "overlay"
assert clips[2].clip_type == "overlay"
def test_voice_over_mode(self):
paths = self._make_paths(3)
plan, clips, asset_map = _build_plan_and_clips_from_task("t3", paths, "voice_over")
assert len(clips) == 3
assert all(c.clip_type == "main" for c in clips)
assert all(c.config.get("role") == "b_roll" for c in clips)
def test_voice_pip_mode(self):
paths = self._make_paths(4)
plan, clips, asset_map = _build_plan_and_clips_from_task("t4", paths, "voice_pip")
assert len(clips) == 4
assert clips[0].clip_type == "background"
assert clips[1].clip_type == "corner_voice"
assert clips[2].clip_type == "b_roll"
assert clips[3].clip_type == "b_roll"
def test_unknown_mode_defaults_to_one_take(self):
paths = self._make_paths(2)
plan, clips, asset_map = _build_plan_and_clips_from_task("t5", paths, "unknown_mode")
assert len(clips) == 2
assert all(c.clip_type == "main" for c in clips)
def test_asset_path_map_keys_match_clip_asset_ids(self):
paths = self._make_paths(3)
_, clips, asset_map = _build_plan_and_clips_from_task("t6", paths, "one_take")
clip_asset_ids = {c.asset_id for c in clips}
map_keys = set(asset_map.keys())
assert clip_asset_ids == map_keys
# ── 测试图层分组(4 模式) ────────────────────────────────────────────────────
class TestFourModeLayerGrouping:
"""验证 4 种模式的 clip_type 分布经 _resolve_layer_role 后产生正确的图层。"""
def test_one_take_layers(self):
"""ONE_TAKE: 3 main → 1 main layer。"""
paths = [Path(f"/tmp/ot_{i}.mp4") for i in range(3)]
_, clips, _ = _build_plan_and_clips_from_task("ot", paths, "one_take")
roles = {_resolve_layer_role(c.clip_type, c.config) for c in clips}
assert roles == {"main"}
def test_pip_layers(self):
"""PIP: 1 main + 2 overlay → main + overlay。"""
paths = [Path(f"/tmp/pip_{i}.mp4") for i in range(3)]
_, clips, _ = _build_plan_and_clips_from_task("pip", paths, "pip")
roles = {_resolve_layer_role(c.clip_type, c.config) for c in clips}
assert roles == {"main", "overlay"}
def test_voice_over_layers(self):
"""VOICE_OVER: 3 main(b_roll) → broll。"""
paths = [Path(f"/tmp/vo_{i}.mp4") for i in range(3)]
_, clips, _ = _build_plan_and_clips_from_task("vo", paths, "voice_over")
roles = {_resolve_layer_role(c.clip_type, c.config) for c in clips}
assert roles == {"broll"}
def test_voice_pip_layers(self):
"""VOICE_PIP: 1 bg + 1 corner_voice + 2 b_roll → 3 个图层。"""
paths = [Path(f"/tmp/vpip_{i}.mp4") for i in range(4)]
_, clips, _ = _build_plan_and_clips_from_task("vpip", paths, "voice_pip")
roles = {_resolve_layer_role(c.clip_type, c.config) for c in clips}
assert roles == {"background", "corner_voice", "broll"}
# ── 端到端渲染测试(需要 ffmpeg) ─────────────────────────────────────────────
class TestEndToEndRendering:
"""4 种模式的完整渲染测试,验证输出文件存在且时长合理。"""
def test_one_take_render(self):
result, work_dir = _render_with_mode("one_take", num_clips=2, duration=2.0)
try:
assert result.output_path.exists()
assert result.file_size > 0
assert result.duration > 0
assert result.width == 640
assert result.height == 360
finally:
shutil.rmtree(work_dir, ignore_errors=True)
def test_pip_render(self):
result, work_dir = _render_with_mode("pip", num_clips=2, duration=2.0)
try:
assert result.output_path.exists()
assert result.file_size > 0
assert result.duration > 0
finally:
shutil.rmtree(work_dir, ignore_errors=True)
def test_voice_over_render(self):
result, work_dir = _render_with_mode("voice_over", num_clips=2, duration=2.0)
try:
assert result.output_path.exists()
assert result.file_size > 0
assert result.duration > 0
finally:
shutil.rmtree(work_dir, ignore_errors=True)
def test_voice_pip_render(self):
result, work_dir = _render_with_mode("voice_pip", num_clips=3, duration=2.0)
try:
assert result.output_path.exists()
assert result.file_size > 0
assert result.duration > 0
finally:
shutil.rmtree(work_dir, ignore_errors=True)
+238
View File
@@ -0,0 +1,238 @@
"""全链路集成测试.
验证 PlanGeneratorService → UnifiedRenderService → 查重 的端到端流程。
"""
from __future__ import annotations
import shutil
import subprocess
import tempfile
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
from video_processing.unified_render_service import (
RenderResult,
UnifiedRenderService,
)
from worker_app.tasks.generation import (
OUTPUT_HEIGHT,
OUTPUT_WIDTH,
_build_plan_and_clips_from_task,
_create_fallback_clip,
_mux_audio_track,
)
pytestmark = pytest.mark.skipif(
not shutil.which("ffmpeg"),
reason="ffmpeg not available",
)
def _generate_test_video(path: Path, duration: float = 3.0) -> None:
"""生成一个测试视频。"""
cmd = [
"ffmpeg",
"-y",
"-f",
"lavfi",
"-i",
f"color=c=blue:s=640x360:d={duration}:r=25",
"-c:v",
"libx264",
"-pix_fmt",
"yuv420p",
str(path),
]
subprocess.run(cmd, check=True, capture_output=True, timeout=30)
def _generate_test_audio(path: Path, duration: float = 5.0) -> None:
"""生成一个测试音频文件。"""
cmd = [
"ffmpeg",
"-y",
"-f",
"lavfi",
"-i",
f"sine=frequency=440:duration={duration}",
"-c:a",
"aac",
"-b:a",
"128k",
str(path),
]
subprocess.run(cmd, check=True, capture_output=True, timeout=30)
# ── 测试 _create_fallback_clip ────────────────────────────────────────────────
class TestFallbackClip:
"""测试 fallback 视频生成。"""
def test_fallback_clip_creates_video(self):
with tempfile.TemporaryDirectory() as tmpdir:
output = Path(tmpdir) / "fallback.mp4"
_create_fallback_clip(output, "Test Fallback")
assert output.exists()
assert output.stat().st_size > 0
# ── 测试 _mux_audio_track ────────────────────────────────────────────────────
class TestMuxAudioTrack:
"""测试视频+音频混合。"""
def test_mux_audio_into_video(self):
with tempfile.TemporaryDirectory() as tmpdir:
video_path = Path(tmpdir) / "video.mp4"
audio_path = Path(tmpdir) / "audio.aac"
output_path = Path(tmpdir) / "output.mp4"
_generate_test_video(video_path, duration=3.0)
_generate_test_audio(audio_path, duration=5.0)
_mux_audio_track(video_path, str(audio_path), output_path)
assert output_path.exists()
assert output_path.stat().st_size > 0
# 验证输出文件包含音频轨
probe_cmd = [
"ffprobe",
"-v",
"quiet",
"-show_streams",
"-select_streams",
"a",
"-of",
"csv=p=0",
str(output_path),
]
result = subprocess.run(probe_cmd, capture_output=True, text=True, timeout=10)
# 如果有音频流,输出非空
assert result.stdout.strip() != "" or result.returncode == 0
# ── 测试 PlanGenerator → UnifiedRenderService 全链路 ─────────────────────────
class TestFullPipeline:
"""验证从虚拟 plan 构建到渲染输出的完整流程。"""
def test_one_take_pipeline(self):
"""ONE_TAKE 模式完整流程。"""
with tempfile.TemporaryDirectory() as tmpdir:
work_dir = Path(tmpdir)
# 生成测试素材
paths = []
for i in range(3):
p = work_dir / f"clip_{i}.mp4"
_generate_test_video(p, duration=2.0)
paths.append(p)
# 构建虚拟 plan
plan, clips, asset_map = _build_plan_and_clips_from_task("pipeline_test", paths, "one_take")
# 渲染
service = UnifiedRenderService(
plan=plan,
clips=clips,
asset_path_map=asset_map,
work_dir=work_dir,
output_width=640,
output_height=360,
)
result = service.render()
assert result.output_path.exists()
assert result.duration > 0
assert result.file_size > 0
assert result.width == 640
assert result.height == 360
def test_pipeline_with_audio_mux(self):
"""渲染 + 混音后处理。"""
with tempfile.TemporaryDirectory() as tmpdir:
work_dir = Path(tmpdir)
# 生成测试素材
video_path = work_dir / "clip_0.mp4"
_generate_test_video(video_path, duration=3.0)
# 构建虚拟 plan
plan, clips, asset_map = _build_plan_and_clips_from_task("audio_test", [video_path], "one_take")
# 渲染
service = UnifiedRenderService(
plan=plan,
clips=clips,
asset_path_map=asset_map,
work_dir=work_dir,
output_width=640,
output_height=360,
)
render_result = service.render()
# 混音
audio_path = work_dir / "voice.aac"
_generate_test_audio(audio_path, duration=5.0)
final_path = work_dir / "final.mp4"
_mux_audio_track(render_result.output_path, str(audio_path), final_path)
assert final_path.exists()
assert final_path.stat().st_size > 0
def test_single_clip_pipeline(self):
"""单 clip 渲染(无转场)。"""
with tempfile.TemporaryDirectory() as tmpdir:
work_dir = Path(tmpdir)
video_path = work_dir / "single.mp4"
_generate_test_video(video_path, duration=5.0)
plan, clips, asset_map = _build_plan_and_clips_from_task("single_test", [video_path], "one_take")
service = UnifiedRenderService(
plan=plan,
clips=clips,
asset_path_map=asset_map,
work_dir=work_dir,
output_width=640,
output_height=360,
)
result = service.render()
assert result.output_path.exists()
assert result.duration > 0
def test_dedup_helper_integration(self):
"""验证 dedup_helpers.create_video_record_and_dedup 的导入和签名。"""
# 只验证函数存在且签名正确(不实际调用,需要数据库)
import inspect
from video_processing.dedup_helpers import create_video_record_and_dedup
sig = inspect.signature(create_video_record_and_dedup)
params = set(sig.parameters.keys())
expected = {
"generation_task_id",
"project_id",
"batch_id",
"file_url",
"file_size",
"duration",
"video_path",
"mode",
"session",
"width",
"height",
"fps",
}
assert expected.issubset(params), f"Missing params: {expected - params}"
@@ -105,11 +105,19 @@ class TestNormalizePlanConfig:
class TestNormalizeTemplateConfig:
def test_same_as_plan_config(self):
def test_same_as_plan_config_plus_template_fields(self):
"""template config 包含 plan config 的所有字段,外加 transition_enabled"""
from packages.domain.config_schemas import normalize_plan_config, normalize_template_config
raw = {"title": {"text": "模板标题"}}
assert normalize_template_config(raw) == normalize_plan_config(raw)
plan_cfg = normalize_plan_config(raw)
tpl_cfg = normalize_template_config(raw)
# plan config 的字段在 template config 中应一致
for key in plan_cfg:
assert tpl_cfg[key] == plan_cfg[key]
# template config 额外包含 transition_enabled
assert "transition_enabled" in tpl_cfg
assert tpl_cfg["transition_enabled"] is True
def test_none_returns_defaults(self):
from packages.domain.config_schemas import DEFAULT_EDIT_TEMPLATE_CONFIG, normalize_template_config
+1
View File
@@ -407,6 +407,7 @@ class TestResponseSchema:
"name",
"description",
"template_type",
"editing_mode",
"config",
"preview_url",
"sort_weight",
@@ -0,0 +1,380 @@
"""P3 优化单元测试 — generation.py 三项优化.
覆盖:
P3-1: _download_library_assets strict 模式
P3-2: 归属校验合并到同一 DB session
P3-3: _verify_url_accessible HEAD 重试
"""
from __future__ import annotations
import sys
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import MagicMock, patch
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[2] / "apps" / "worker"))
# ── P3-3: _verify_url_accessible 重试 ───────────────────────────────────────
class TestVerifyUrlAccessibleRetry:
"""_verify_url_accessible 重试逻辑."""
@patch("time.sleep")
@patch("urllib.request.urlopen")
def test_first_attempt_success(self, mock_urlopen, mock_sleep):
"""首次成功,不重试."""
from worker_app.tasks.generation import _verify_url_accessible
mock_resp = MagicMock()
mock_resp.status = 200
mock_resp.__enter__ = MagicMock(return_value=mock_resp)
mock_resp.__exit__ = MagicMock(return_value=False)
mock_urlopen.return_value = mock_resp
assert _verify_url_accessible("https://example.com/file.mp4") is True
assert mock_urlopen.call_count == 1
mock_sleep.assert_not_called()
@patch("time.sleep")
@patch("urllib.request.urlopen")
def test_retry_then_success(self, mock_urlopen, mock_sleep):
"""首次失败,重试后成功."""
from worker_app.tasks.generation import _verify_url_accessible
# 第一次失败(网络异常),第二次成功
mock_resp_ok = MagicMock()
mock_resp_ok.status = 200
mock_resp_ok.__enter__ = MagicMock(return_value=mock_resp_ok)
mock_resp_ok.__exit__ = MagicMock(return_value=False)
mock_urlopen.side_effect = [
OSError("connection reset"),
mock_resp_ok,
]
assert _verify_url_accessible("https://example.com/file.mp4") is True
assert mock_urlopen.call_count == 2
mock_sleep.assert_called_once_with(1)
@patch("time.sleep")
@patch("urllib.request.urlopen")
def test_all_retries_exhausted(self, mock_urlopen, mock_sleep):
"""全部重试耗尽,返回 False."""
from worker_app.tasks.generation import _verify_url_accessible
mock_urlopen.side_effect = OSError("connection refused")
assert _verify_url_accessible("https://example.com/file.mp4") is False
# 1 首次 + 2 重试 = 3 次
assert mock_urlopen.call_count == 3
assert mock_sleep.call_count == 2
@patch("time.sleep")
@patch("urllib.request.urlopen")
def test_http_500_then_success(self, mock_urlopen, mock_sleep):
"""HTTP 500 后重试成功."""
from worker_app.tasks.generation import _verify_url_accessible
mock_resp_500 = MagicMock()
mock_resp_500.status = 500
mock_resp_500.__enter__ = MagicMock(return_value=mock_resp_500)
mock_resp_500.__exit__ = MagicMock(return_value=False)
mock_resp_200 = MagicMock()
mock_resp_200.status = 200
mock_resp_200.__enter__ = MagicMock(return_value=mock_resp_200)
mock_resp_200.__exit__ = MagicMock(return_value=False)
mock_urlopen.side_effect = [mock_resp_500, mock_resp_200]
assert _verify_url_accessible("https://example.com/file.mp4") is True
assert mock_urlopen.call_count == 2
@patch("time.sleep")
@patch("urllib.request.urlopen")
def test_custom_retries_zero(self, mock_urlopen, mock_sleep):
"""retries=0 时不重试."""
from worker_app.tasks.generation import _verify_url_accessible
mock_urlopen.side_effect = OSError("timeout")
assert _verify_url_accessible("https://example.com/file.mp4", retries=0) is False
assert mock_urlopen.call_count == 1
mock_sleep.assert_not_called()
# ── P3-1: _download_library_assets strict 模式 ──────────────────────────────
def _make_mock_asset(
asset_id: str,
name: str,
file_url: str | None,
asset_library_id: str = "lib-1",
project_id: str = "",
):
"""构造 mock AssetModel 实例."""
return SimpleNamespace(
id=asset_id,
name=name,
file_url=file_url,
asset_library_id=asset_library_id,
project_id=project_id,
status="ready",
file_type="video",
created_at="2026-01-01",
)
def _setup_mock_session(assets):
"""构造 mock session,返回 (mock_session, mock_query_chain)."""
mock_session = MagicMock()
mock_query = MagicMock()
# chain: session.query().filter().filter().order_by().all()
mock_session.query.return_value = mock_query
mock_query.filter.return_value = mock_query
mock_query.order_by.return_value = mock_query
mock_query.all.return_value = assets
return mock_session
class TestDownloadLibraryAssetsStrictMode:
"""_download_library_assets strict 模式."""
@patch("worker_app.tasks.generation.SessionLocal")
@patch("worker_app.tasks.generation.download_asset")
def test_strict_mode_raises_on_download_failure(self, mock_download, mock_session_factory):
"""strict=True 时,单个素材下载失败立即抛 RuntimeError."""
from worker_app.tasks.generation import _download_library_assets
assets = [
_make_mock_asset("a1", "video1.mp4", "uploads/video1.mp4"),
_make_mock_asset("a2", "video2.mp4", "uploads/video2.mp4"),
]
mock_session = _setup_mock_session(assets)
mock_session_factory.return_value = mock_session
# 第一个成功,第二个失败
mock_download.side_effect = [True, False]
with pytest.raises(RuntimeError, match="素材下载失败"):
_download_library_assets(
Path("/tmp"),
asset_library_id="lib-1",
asset_ids=["a1", "a2"],
strict=True,
)
@patch("worker_app.tasks.generation.SessionLocal")
@patch("worker_app.tasks.generation.download_asset")
def test_non_strict_mode_returns_partial_results(self, mock_download, mock_session_factory):
"""strict=False 时,跳过失败素材,返回成功列表."""
from worker_app.tasks.generation import _download_library_assets
assets = [
_make_mock_asset("a1", "video1.mp4", "uploads/video1.mp4"),
_make_mock_asset("a2", "video2.mp4", "uploads/video2.mp4"),
_make_mock_asset("a3", "video3.mp4", "uploads/video3.mp4"),
]
mock_session = _setup_mock_session(assets)
mock_session_factory.return_value = mock_session
# 第一个成功,第二个失败,第三个成功
mock_download.side_effect = [True, False, True]
result = _download_library_assets(
Path("/tmp"),
asset_library_id="lib-1",
asset_ids=["a1", "a2", "a3"],
strict=False,
)
assert len(result) == 2
@patch("worker_app.tasks.generation.SessionLocal")
@patch("worker_app.tasks.generation.download_asset")
def test_non_strict_all_fail_raises(self, mock_download, mock_session_factory):
"""strict=False 但全部失败时仍抛 RuntimeError."""
from worker_app.tasks.generation import _download_library_assets
assets = [
_make_mock_asset("a1", "video1.mp4", "uploads/video1.mp4"),
]
mock_session = _setup_mock_session(assets)
mock_session_factory.return_value = mock_session
mock_download.return_value = False
with pytest.raises(RuntimeError, match="全部下载失败"):
_download_library_assets(
Path("/tmp"),
asset_library_id="lib-1",
asset_ids=["a1"],
strict=False,
)
@patch("worker_app.tasks.generation.SessionLocal")
@patch("worker_app.tasks.generation.download_asset")
def test_strict_mode_raises_on_missing_file_url(self, mock_download, mock_session_factory):
"""strict=True 时,素材缺少 file_url 立即抛异常."""
from worker_app.tasks.generation import _download_library_assets
assets = [
_make_mock_asset("a1", "video1.mp4", None), # file_url 为空
]
mock_session = _setup_mock_session(assets)
mock_session_factory.return_value = mock_session
with pytest.raises(RuntimeError, match="素材缺少 file_url"):
_download_library_assets(
Path("/tmp"),
asset_library_id="lib-1",
strict=True,
)
# download_asset 不应被调用
mock_download.assert_not_called()
@patch("worker_app.tasks.generation.SessionLocal")
@patch("worker_app.tasks.generation.download_asset")
def test_default_is_strict(self, mock_download, mock_session_factory):
"""默认 strict=True."""
from worker_app.tasks.generation import _download_library_assets
assets = [
_make_mock_asset("a1", "video1.mp4", "uploads/video1.mp4"),
]
mock_session = _setup_mock_session(assets)
mock_session_factory.return_value = mock_session
mock_download.return_value = False
# 不传 strict 参数,默认严格模式
with pytest.raises(RuntimeError, match="素材下载失败"):
_download_library_assets(
Path("/tmp"),
asset_library_id="lib-1",
)
# ── P3-2: 归属校验合并到同一 session ────────────────────────────────────────
class TestDownloadLibraryAssetsOwnershipValidation:
"""归属校验合并到 _download_library_assets 同一 session."""
@patch("worker_app.tasks.generation.SessionLocal")
@patch("worker_app.tasks.generation.download_asset")
def test_ownership_mismatch_raises_value_error(self, mock_download, mock_session_factory):
"""asset_ids 不属于指定素材库时抛 ValueError."""
from worker_app.tasks.generation import _download_library_assets
# asset 属于 lib-2,但请求的是 lib-1
assets = [
_make_mock_asset("a1", "video1.mp4", "uploads/video1.mp4", asset_library_id="lib-2"),
]
mock_session = _setup_mock_session(assets)
mock_session_factory.return_value = mock_session
with pytest.raises(ValueError, match="素材不属于指定素材库"):
_download_library_assets(
Path("/tmp"),
asset_library_id="lib-1",
asset_ids=["a1"],
)
# 不应调用 download_asset(校验在下载前)
mock_download.assert_not_called()
@patch("worker_app.tasks.generation.SessionLocal")
@patch("worker_app.tasks.generation.download_asset")
def test_missing_asset_ids_raises_value_error(self, mock_download, mock_session_factory):
"""指定的 asset_ids 不存在时抛 ValueError."""
from worker_app.tasks.generation import _download_library_assets
# DB 返回空(asset_ids 不存在,query 过滤后无结果)
mock_session = _setup_mock_session([])
mock_session_factory.return_value = mock_session
with pytest.raises(RuntimeError, match="未找到视频素材"):
_download_library_assets(
Path("/tmp"),
asset_library_id="lib-1",
asset_ids=["nonexistent-id"],
)
@patch("worker_app.tasks.generation.SessionLocal")
@patch("worker_app.tasks.generation.download_asset")
def test_project_ownership_mismatch_raises(self, mock_download, mock_session_factory):
"""项目级模式下归属不匹配抛 ValueError."""
from worker_app.tasks.generation import _download_library_assets
assets = [
_make_mock_asset(
"a1",
"video1.mp4",
"uploads/video1.mp4",
asset_library_id="",
project_id="proj-2",
),
]
mock_session = _setup_mock_session(assets)
mock_session_factory.return_value = mock_session
with pytest.raises(ValueError, match="素材不属于指定项目"):
_download_library_assets(
Path("/tmp"),
project_id="proj-1",
asset_ids=["a1"],
)
mock_download.assert_not_called()
@patch("worker_app.tasks.generation.SessionLocal")
@patch("worker_app.tasks.generation.download_asset")
def test_ownership_pass_then_download(self, mock_download, mock_session_factory):
"""归属校验通过后正常下载."""
from worker_app.tasks.generation import _download_library_assets
assets = [
_make_mock_asset("a1", "video1.mp4", "uploads/video1.mp4", asset_library_id="lib-1"),
]
mock_session = _setup_mock_session(assets)
mock_session_factory.return_value = mock_session
mock_download.return_value = True
result = _download_library_assets(
Path("/tmp"),
asset_library_id="lib-1",
asset_ids=["a1"],
)
assert len(result) == 1
mock_download.assert_called_once()
@patch("worker_app.tasks.generation.SessionLocal")
@patch("worker_app.tasks.generation.download_asset")
def test_single_session_used(self, mock_download, mock_session_factory):
"""验证只创建了一个 DB sessionP3-2 核心)."""
from worker_app.tasks.generation import _download_library_assets
assets = [
_make_mock_asset("a1", "video1.mp4", "uploads/video1.mp4", asset_library_id="lib-1"),
]
mock_session = _setup_mock_session(assets)
mock_session_factory.return_value = mock_session
mock_download.return_value = True
_download_library_assets(
Path("/tmp"),
asset_library_id="lib-1",
asset_ids=["a1"],
)
# SessionLocal 只调用一次(合并前会调用两次:校验 + 下载)
assert mock_session_factory.call_count == 1
+179
View File
@@ -0,0 +1,179 @@
"""
P0-2 修复测试:generation_tasks results 端点返回 OSS 预签名 URL
验证 list_generation_results 端点:
- 对每个生成视频的 file_url 调用 storage_service.get_download_url()
- 返回的 download_url 是预签名临时 URL24h 有效期)
- 与 generated_videos.py 中的模式一致
"""
from __future__ import annotations
import os
import sys
from pathlib import Path
from unittest.mock import MagicMock, patch
os.environ.setdefault("JWT_SECRET_KEY", "unit-test-secret-key-for-testing")
os.environ.setdefault("DATABASE_URL", "sqlite:///test.db")
sys.path.insert(0, str(Path(__file__).resolve().parents[2] / "apps" / "api"))
from app.api.routes.generation_tasks import router
from app.auth import AuthenticatedUser, get_current_user
from app.core.storage import OSSStorageService, get_storage_service
from app.dependencies import (
get_generated_video_repository,
get_generation_task_repository,
get_project_repository,
)
from fastapi import FastAPI
from fastapi.testclient import TestClient
from packages.domain import (
GeneratedVideo,
GenerationTask,
GenerationTaskStatus,
Project,
User,
)
# ---------------------------------------------------------------------------
# Stub repositories
# ---------------------------------------------------------------------------
class StubProjectRepository:
def __init__(self, projects=None):
self._projects = projects or {}
def find_by_id(self, project_id):
return self._projects.get(project_id)
class StubGenerationTaskRepository:
def __init__(self, tasks=None):
self._tasks = tasks or {}
def get(self, task_id):
return self._tasks.get(task_id)
class StubGeneratedVideoRepository:
def __init__(self, videos=None):
self._videos = videos or {}
def list_by_generation_task(self, task_id):
return [v for v in self._videos.values() if v.generation_task_id == task_id]
# ---------------------------------------------------------------------------
# Fixtures
# ---------------------------------------------------------------------------
def _make_app(task, videos, storage_mock):
app = FastAPI()
app.include_router(router, prefix="/api/v1/generation")
user = User(id="user-1", email="test@test.com", display_name="Test", username="testuser")
auth = AuthenticatedUser(user=user, token_type="access")
project = Project(id="project-1", name="Test", description="", owner_user_id="user-1")
app.dependency_overrides[get_current_user] = lambda: auth
app.dependency_overrides[get_project_repository] = lambda: StubProjectRepository({"project-1": project})
app.dependency_overrides[get_generation_task_repository] = lambda: StubGenerationTaskRepository({"task-1": task})
app.dependency_overrides[get_generated_video_repository] = lambda: StubGeneratedVideoRepository(videos)
app.dependency_overrides[get_storage_service] = lambda: storage_mock
return app
def test_results_endpoint_generates_presigned_urls():
"""验证 list_generation_results 为每个视频生成预签名 download_url"""
task = GenerationTask(
id="task-1",
project_id="project-1",
asset_library_id="lib-1",
strategy_id="s1",
status=GenerationTaskStatus.COMPLETED,
progress=100,
result_count=2,
created_by_user_id="user-1",
)
videos = {
"v1": GeneratedVideo.create(
project_id="project-1",
generation_task_id="task-1",
name="video1.mp4",
file_url="https://bucket.oss-cn-hangzhou.aliyuncs.com/generated/v1.mp4",
file_size=1024,
duration=5.0,
width=1280,
height=720,
fps=25.0,
),
"v2": GeneratedVideo.create(
project_id="project-1",
generation_task_id="task-1",
name="video2.mp4",
file_url="https://bucket.oss-cn-hangzhou.aliyuncs.com/generated/v2.mp4",
file_size=2048,
duration=10.0,
width=1920,
height=1080,
fps=30.0,
),
}
storage_mock = MagicMock(spec=OSSStorageService)
storage_mock.get_download_url.side_effect = (
lambda url, expires_seconds=3600: f"{url}?signature=presigned&expires={expires_seconds}"
)
app = _make_app(task, videos, storage_mock)
client = TestClient(app)
response = client.get("/api/v1/generation/tasks/task-1/results")
assert response.status_code == 200
data = response.json()
assert len(data["items"]) == 2
# Verify presigned URLs were generated with 24h expiry
assert storage_mock.get_download_url.call_count == 2
for call in storage_mock.get_download_url.call_args_list:
assert call.kwargs["expires_seconds"] == 86400
# Verify download_url is present in response
for item in data["items"]:
assert item["download_url"] is not None
assert "signature=presigned" in item["download_url"]
assert "expires=86400" in item["download_url"]
# Verify file_url is still the original (raw) URL
assert data["items"][0]["file_url"] == videos["v1"].file_url
def test_results_endpoint_handles_empty_videos():
"""验证无视频时正常返回空列表"""
task = GenerationTask(
id="task-1",
project_id="project-1",
asset_library_id="lib-1",
strategy_id="s1",
status=GenerationTaskStatus.RUNNING,
progress=50,
created_by_user_id="user-1",
)
storage_mock = MagicMock(spec=OSSStorageService)
app = _make_app(task, {}, storage_mock)
client = TestClient(app)
response = client.get("/api/v1/generation/tasks/task-1/results")
assert response.status_code == 200
assert response.json()["items"] == []
storage_mock.get_download_url.assert_not_called()
+455
View File
@@ -0,0 +1,455 @@
"""GenerationTask 领域模型状态机单元测试.
覆盖:
- 初始状态为 pending
- mark_processing: pending → running
- mark_completed: running → completed
- mark_failed: pending/running → failed
- mark_cancelled: pending/running → cancelled
- mark_pending_from_failed: failed → pending(重试)
- 非法状态转换抛出 ValueError
- is_terminal / is_completed / is_failed / is_running 属性
- 状态转换时的时间戳设置
"""
from __future__ import annotations
import pytest
from packages.domain.generation_task import (
TERMINAL_STATUSES,
GenerationTask,
GenerationTaskStatus,
)
def _make_task(**overrides) -> GenerationTask:
"""创建一个测试用的 GenerationTask。"""
defaults = dict(
id="task-test-001",
project_id="proj-1",
asset_library_id="lib-1",
)
defaults.update(overrides)
return GenerationTask(**defaults)
# ── 初始状态 ──────────────────────────────────────────────────────────────────
class TestInitialState:
"""测试初始状态。"""
def test_default_status_is_pending(self) -> None:
"""新创建的任务默认状态为 pending。"""
task = _make_task()
assert task.status == GenerationTaskStatus.PENDING
assert task.progress == 0.0
assert task.result_count == 0
assert task.error_message == ""
assert task.started_at is None
assert task.completed_at is None
def test_create_factory_returns_pending(self) -> None:
"""GenerationTask.create() 返回的任务状态为 pending。"""
task = GenerationTask.create(
project_id="proj-1",
asset_library_id="lib-1",
created_by_user_id="user-1",
)
assert task.status == GenerationTaskStatus.PENDING
assert task.progress == 0.0
assert task.result_count == 0
def test_is_not_terminal_initially(self) -> None:
"""初始状态不是终态。"""
task = _make_task()
assert not task.is_terminal
assert not task.is_completed
assert not task.is_failed
assert not task.is_running
def test_terminal_statuses_constant(self) -> None:
"""终态集合包含 completed / failed / cancelled。"""
assert GenerationTaskStatus.COMPLETED in TERMINAL_STATUSES
assert GenerationTaskStatus.FAILED in TERMINAL_STATUSES
assert GenerationTaskStatus.CANCELLED in TERMINAL_STATUSES
assert GenerationTaskStatus.PENDING not in TERMINAL_STATUSES
assert GenerationTaskStatus.RUNNING not in TERMINAL_STATUSES
# ── mark_processing ──────────────────────────────────────────────────────────
class TestMarkProcessing:
"""测试 pending → running 转换。"""
def test_pending_to_running_success(self) -> None:
"""pending 状态的任务可以标记为 running。"""
task = _make_task()
task.mark_processing()
assert task.status == GenerationTaskStatus.RUNNING
assert task.is_running
assert task.started_at is not None
assert task.error_message == ""
def test_started_at_is_set(self) -> None:
"""mark_processing 设置 started_at 时间戳。"""
task = _make_task()
assert task.started_at is None
task.mark_processing()
assert task.started_at is not None
def test_error_message_cleared(self) -> None:
"""mark_processing 清除 error_message(如果有的话)。"""
task = _make_task()
# 注意:pending 状态通常没有 error_message,这里验证确保被清除
task.error_message = "some old error"
# 直接设置状态绕过校验(模拟异常场景)
task.status = GenerationTaskStatus.PENDING
task.mark_processing()
assert task.error_message == ""
def test_running_to_running_raises(self) -> None:
"""running 状态不能再次 mark_processing。"""
task = _make_task()
task.mark_processing()
with pytest.raises(ValueError, match="非法状态转换"):
task.mark_processing()
def test_completed_to_running_raises(self) -> None:
"""completed 状态不能回到 running。"""
task = _make_task()
task.mark_processing()
task.mark_completed()
with pytest.raises(ValueError, match="非法状态转换"):
task.mark_processing()
def test_failed_to_running_raises(self) -> None:
"""failed 状态不能直接到 running(应先重置为 pending)。"""
task = _make_task()
task.mark_processing()
task.mark_failed("some error")
with pytest.raises(ValueError, match="非法状态转换"):
task.mark_processing()
# ── mark_completed ───────────────────────────────────────────────────────────
class TestMarkCompleted:
"""测试 running → completed 转换。"""
def test_running_to_completed_success(self) -> None:
"""running 状态的任务可以标记为 completed。"""
task = _make_task()
task.mark_processing()
task.mark_completed()
assert task.status == GenerationTaskStatus.COMPLETED
assert task.is_completed
assert task.is_terminal
assert task.completed_at is not None
def test_progress_set_to_100(self) -> None:
"""mark_completed 设置 progress 为 100.0。"""
task = _make_task()
task.mark_processing()
task.progress = 50.0 # 模拟中间进度
task.mark_completed()
assert task.progress == 100.0
def test_default_result_count_is_1(self) -> None:
"""默认 result_count 为 1。"""
task = _make_task()
task.mark_processing()
task.mark_completed()
assert task.result_count == 1
def test_custom_result_count(self) -> None:
"""可以指定 result_count。"""
task = _make_task()
task.mark_processing()
task.mark_completed(result_count=5)
assert task.result_count == 5
def test_error_message_cleared(self) -> None:
"""mark_completed 清除 error_message。"""
task = _make_task()
task.mark_processing()
task.error_message = "temporary error"
task.mark_completed()
assert task.error_message == ""
def test_completed_at_is_set(self) -> None:
"""mark_completed 设置 completed_at。"""
task = _make_task()
task.mark_processing()
assert task.completed_at is None
task.mark_completed()
assert task.completed_at is not None
def test_pending_to_completed_raises(self) -> None:
"""pending 状态不能直接到 completed。"""
task = _make_task()
with pytest.raises(ValueError, match="非法状态转换"):
task.mark_completed()
def test_completed_to_completed_raises(self) -> None:
"""completed 状态不能再次 mark_completed。"""
task = _make_task()
task.mark_processing()
task.mark_completed()
with pytest.raises(ValueError, match="非法状态转换"):
task.mark_completed()
def test_failed_to_completed_raises(self) -> None:
"""failed 状态不能直接到 completed。"""
task = _make_task()
task.mark_processing()
task.mark_failed("error")
with pytest.raises(ValueError, match="非法状态转换"):
task.mark_completed()
# ── mark_failed ──────────────────────────────────────────────────────────────
class TestMarkFailed:
"""测试 pending/running → failed 转换。"""
def test_pending_to_failed_success(self) -> None:
"""pending 状态可以直接标记为 failed。"""
task = _make_task()
task.mark_failed("资源不足")
assert task.status == GenerationTaskStatus.FAILED
assert task.is_failed
assert task.is_terminal
assert task.error_message == "资源不足"
assert task.completed_at is not None
def test_running_to_failed_success(self) -> None:
"""running 状态可以标记为 failed。"""
task = _make_task()
task.mark_processing()
task.mark_failed("生成失败:FFmpeg 错误")
assert task.status == GenerationTaskStatus.FAILED
assert task.is_failed
assert task.is_terminal
assert task.error_message == "生成失败:FFmpeg 错误"
assert task.completed_at is not None
def test_completed_to_failed_raises(self) -> None:
"""completed 状态不能标记为 failed。"""
task = _make_task()
task.mark_processing()
task.mark_completed()
with pytest.raises(ValueError, match="非法状态转换"):
task.mark_failed("late error")
def test_failed_to_failed_raises(self) -> None:
"""failed 状态不能再次 mark_failed。"""
task = _make_task()
task.mark_failed("first error")
with pytest.raises(ValueError, match="非法状态转换"):
task.mark_failed("second error")
def test_error_message_preserved(self) -> None:
"""错误信息被正确保存。"""
task = _make_task()
error_msg = "FFmpeg returned non-zero exit status 1"
task.mark_failed(error_msg)
assert task.error_message == error_msg
# ── mark_cancelled ───────────────────────────────────────────────────────────
class TestMarkCancelled:
"""测试 pending/running → cancelled 转换。"""
def test_pending_to_cancelled_success(self) -> None:
"""pending 状态可以取消。"""
task = _make_task()
task.mark_cancelled()
assert task.status == GenerationTaskStatus.CANCELLED
assert task.is_terminal
assert task.completed_at is not None
def test_running_to_cancelled_success(self) -> None:
"""running 状态可以取消。"""
task = _make_task()
task.mark_processing()
task.mark_cancelled()
assert task.status == GenerationTaskStatus.CANCELLED
assert task.is_terminal
def test_completed_to_cancelled_raises(self) -> None:
"""completed 状态不能取消。"""
task = _make_task()
task.mark_processing()
task.mark_completed()
with pytest.raises(ValueError, match="非法状态转换"):
task.mark_cancelled()
def test_failed_to_cancelled_raises(self) -> None:
"""failed 状态不能取消。"""
task = _make_task()
task.mark_failed("some error")
with pytest.raises(ValueError, match="非法状态转换"):
task.mark_cancelled()
# ── mark_pending_from_failed (重试) ─────────────────────────────────────────
class TestMarkPendingFromFailed:
"""测试 failed → pending(重试)转换。"""
def test_failed_to_pending_success(self) -> None:
"""failed 状态可以重置为 pending(用于重试)。"""
task = _make_task()
task.mark_processing()
task.mark_failed("临时错误")
task.mark_pending_from_failed()
assert task.status == GenerationTaskStatus.PENDING
assert not task.is_terminal
assert task.error_message == ""
assert task.started_at is None
assert task.completed_at is None
assert task.progress == 0.0
assert task.result_count == 0
def test_pending_to_pending_raises(self) -> None:
"""pending 状态不能调用 mark_pending_from_failed。"""
task = _make_task()
with pytest.raises(ValueError, match="只有 failed 状态"):
task.mark_pending_from_failed()
def test_running_to_pending_raises(self) -> None:
"""running 状态不能调用 mark_pending_from_failed。"""
task = _make_task()
task.mark_processing()
with pytest.raises(ValueError, match="只有 failed 状态"):
task.mark_pending_from_failed()
def test_completed_to_pending_raises(self) -> None:
"""completed 状态不能调用 mark_pending_from_failed。"""
task = _make_task()
task.mark_processing()
task.mark_completed()
with pytest.raises(ValueError, match="只有 failed 状态"):
task.mark_pending_from_failed()
# ── transition_to 通用方法 ───────────────────────────────────────────────────
class TestTransitionTo:
"""测试通用的 transition_to 方法。"""
def test_string_status_conversion(self) -> None:
"""可以传入字符串形式的状态。"""
task = _make_task()
task.transition_to("running")
assert task.status == GenerationTaskStatus.RUNNING
def test_invalid_string_raises(self) -> None:
"""无效的状态字符串抛出 ValueError。"""
task = _make_task()
with pytest.raises(ValueError, match="无效状态"):
task.transition_to("invalid_status")
def test_enum_status(self) -> None:
"""可以传入枚举形式的状态。"""
task = _make_task()
task.transition_to(GenerationTaskStatus.RUNNING)
assert task.status == GenerationTaskStatus.RUNNING
def test_error_message_includes_allowed_statuses(self) -> None:
"""错误信息包含允许的状态列表。"""
task = _make_task()
task.mark_processing()
task.mark_completed()
with pytest.raises(ValueError) as exc_info:
task.transition_to(GenerationTaskStatus.RUNNING)
assert "completed" in str(exc_info.value)
assert "running" in str(exc_info.value)
# ── 完整流转路径 ─────────────────────────────────────────────────────────────
class TestFullFlow:
"""测试完整的状态流转路径。"""
def test_happy_path(self) -> None:
"""正常路径:pending → running → completed。"""
task = _make_task()
assert task.status == GenerationTaskStatus.PENDING
assert not task.is_terminal
task.mark_processing()
assert task.status == GenerationTaskStatus.RUNNING
assert task.started_at is not None
assert not task.is_terminal
task.mark_completed(result_count=3)
assert task.status == GenerationTaskStatus.COMPLETED
assert task.is_completed
assert task.is_terminal
assert task.completed_at is not None
assert task.result_count == 3
assert task.progress == 100.0
def test_failure_path_from_running(self) -> None:
"""失败路径:pending → running → failed。"""
task = _make_task()
task.mark_processing()
assert task.is_running
task.mark_failed("网络超时")
assert task.is_failed
assert task.is_terminal
assert task.error_message == "网络超时"
assert task.completed_at is not None
def test_failure_path_from_pending(self) -> None:
"""失败路径:pending → failed(启动前校验失败等)。"""
task = _make_task()
task.mark_failed("参数校验失败")
assert task.is_failed
assert task.is_terminal
def test_retry_path(self) -> None:
"""重试路径:pending → running → failed → pending → running → completed。"""
task = _make_task()
# 第一次尝试失败
task.mark_processing()
task.mark_failed("临时错误")
assert task.is_failed
# 重试
task.mark_pending_from_failed()
assert task.status == GenerationTaskStatus.PENDING
assert task.error_message == ""
# 第二次成功
task.mark_processing()
task.mark_completed()
assert task.is_completed
def test_cancel_from_pending(self) -> None:
"""取消路径:pending → cancelled。"""
task = _make_task()
task.mark_cancelled()
assert task.status == GenerationTaskStatus.CANCELLED
assert task.is_terminal
def test_cancel_from_running(self) -> None:
"""取消路径:pending → running → cancelled。"""
task = _make_task()
task.mark_processing()
task.mark_cancelled()
assert task.status == GenerationTaskStatus.CANCELLED
assert task.is_terminal
+268
View File
@@ -0,0 +1,268 @@
"""P0/P1 修复单元测试 — 一键生成 P0 问题 + P1 校验.
覆盖:
P0-1: _download_library_assets 双模式查询(asset_library_id / project_id
P0-2: OSS 上传失败抛异常 + URL 可访问性校验
P0-3: FFmpeg 失败时完整 stderr 日志
P1: template_id 存在性校验 + asset_ids 归属校验
"""
from __future__ import annotations
import logging
import subprocess
import sys
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
# 添加 worker app 到 sys.path
_WORKER_ROOT = Path(__file__).resolve().parents[2] / "apps" / "worker"
if str(_WORKER_ROOT) not in sys.path:
sys.path.insert(0, str(_WORKER_ROOT))
# ── P0-1: _download_library_assets ────────────────────────────────────────────
class TestDownloadLibraryAssets:
"""P0-1: 素材下载双模式 + 错误处理."""
def _make_asset(self, id_: str, file_url: str, project_id: str = "p1", library_id: str = "lib1"):
mock = MagicMock()
mock.id = id_
mock.file_url = file_url
mock.name = f"asset_{id_}"
mock.project_id = project_id
mock.asset_library_id = library_id
return mock
@patch("worker_app.tasks.generation.SessionLocal")
@patch("worker_app.tasks.generation.download_asset")
def test_asset_library_mode(self, mock_download, mock_session_factory):
"""素材库模式:按 asset_library_id 查询."""
from worker_app.tasks.generation import _download_library_assets
session = MagicMock()
mock_session_factory.return_value = session
query = MagicMock()
session.query.return_value = query
filter_result = MagicMock()
query.filter.return_value = filter_result
in_filter = MagicMock()
filter_result.filter.return_value = in_filter
assets = [self._make_asset("a1", "video/a1.mp4")]
in_filter.order_by.return_value.all.return_value = assets
mock_download.return_value = True
with patch("worker_app.tasks.generation.AssetModel", create=True):
result = _download_library_assets(
Path("/tmp/test"),
asset_library_id="lib1",
)
assert len(result) == 1
mock_download.assert_called_once()
@patch("worker_app.tasks.generation.SessionLocal")
@patch("worker_app.tasks.generation.download_asset")
def test_project_mode(self, mock_download, mock_session_factory):
"""项目级模式:asset_library_id 为空时按 project_id 查询."""
from worker_app.tasks.generation import _download_library_assets
session = MagicMock()
mock_session_factory.return_value = session
query = MagicMock()
session.query.return_value = query
filter_result = MagicMock()
query.filter.return_value = filter_result
proj_filter = MagicMock()
filter_result.filter.return_value = proj_filter
assets = [self._make_asset("a1", "video/a1.mp4", project_id="proj1")]
proj_filter.order_by.return_value.all.return_value = assets
mock_download.return_value = True
result = _download_library_assets(
Path("/tmp/test"),
project_id="proj1",
)
assert len(result) == 1
def test_both_empty_raises(self):
"""asset_library_id 和 project_id 都为空时抛 ValueError."""
from worker_app.tasks.generation import _download_library_assets
with pytest.raises(ValueError, match="至少需要提供一个"):
_download_library_assets(Path("/tmp/test"))
@patch("worker_app.tasks.generation.SessionLocal")
def test_no_assets_found_raises(self, mock_session_factory):
"""查不到素材时抛 RuntimeError."""
from worker_app.tasks.generation import _download_library_assets
session = MagicMock()
mock_session_factory.return_value = session
query = MagicMock()
session.query.return_value = query
filter_result = MagicMock()
query.filter.return_value = filter_result
in_filter = MagicMock()
filter_result.filter.return_value = in_filter
in_filter.order_by.return_value.all.return_value = []
with pytest.raises(RuntimeError, match="未找到视频素材"):
_download_library_assets(
Path("/tmp/test"),
asset_library_id="lib1",
)
@patch("worker_app.tasks.generation.SessionLocal")
@patch("worker_app.tasks.generation.download_asset")
def test_all_asset_ids_fail_raises(self, mock_download, mock_session_factory):
"""指定 asset_ids 但全部下载失败时抛 RuntimeError."""
from worker_app.tasks.generation import _download_library_assets
session = MagicMock()
mock_session_factory.return_value = session
query = MagicMock()
session.query.return_value = query
filter_result = MagicMock()
query.filter.return_value = filter_result
in_filter = MagicMock()
filter_result.filter.return_value = in_filter
id_filter = MagicMock()
in_filter.filter.return_value = id_filter
assets = [self._make_asset("a1", "video/a1.mp4")]
id_filter.order_by.return_value.all.return_value = assets
mock_download.return_value = False # 全部下载失败
with pytest.raises(RuntimeError, match="素材下载失败"):
_download_library_assets(
Path("/tmp/test"),
asset_library_id="lib1",
asset_ids=["a1"],
)
# ── P0-2: OSS 上传 + URL 校验 ────────────────────────────────────────────────
class TestOSSUploadAndVerify:
"""P0-2: OSS 上传失败抛异常 + URL 可访问性校验."""
def test_verify_url_accessible_success(self):
"""URL 可访问时返回 True."""
from worker_app.tasks.generation import _verify_url_accessible
mock_response = MagicMock()
mock_response.status = 200
mock_response.__enter__ = MagicMock(return_value=mock_response)
mock_response.__exit__ = MagicMock(return_value=False)
with patch("urllib.request.urlopen", return_value=mock_response):
assert _verify_url_accessible("https://example.com/test.mp4") is True
def test_verify_url_accessible_failure(self):
"""URL 不可访问时返回 False."""
from worker_app.tasks.generation import _verify_url_accessible
with patch("urllib.request.urlopen", side_effect=Exception("connection refused")):
assert _verify_url_accessible("https://example.com/test.mp4") is False
def test_verify_url_404(self):
"""URL 返回 404 时返回 False."""
from worker_app.tasks.generation import _verify_url_accessible
mock_response = MagicMock()
mock_response.status = 404
mock_response.__enter__ = MagicMock(return_value=mock_response)
mock_response.__exit__ = MagicMock(return_value=False)
with patch("urllib.request.urlopen", return_value=mock_response):
assert _verify_url_accessible("https://example.com/test.mp4") is False
# ── P0-3: FFmpeg stderr 日志 ─────────────────────────────────────────────────
class TestFFmpegStderrLogging:
"""P0-3: FFmpeg 失败时完整 stderr 打到日志."""
def test_run_ffmpeg_logs_stderr_on_failure(self, caplog):
"""run_ffmpeg 失败时记录 stderr 到日志."""
from video_processing.ffmpeg_utils import run_ffmpeg
error = subprocess.CalledProcessError(
returncode=183,
cmd=["ffmpeg", "-y", "-i", "input.mp4", "output.mp4"],
output="",
stderr="Error message from ffmpeg: filter graph error details here",
)
with patch("subprocess.run", side_effect=error):
with caplog.at_level(logging.ERROR):
with pytest.raises(subprocess.CalledProcessError):
run_ffmpeg(["ffmpeg", "-y", "-i", "input.mp4", "output.mp4"])
assert "FFmpeg 命令失败" in caplog.text
assert "exit_code=183" in caplog.text
assert "filter graph error" in caplog.text
def test_run_ffmpeg_success(self):
"""run_ffmpeg 成功时正常返回."""
from video_processing.ffmpeg_utils import run_ffmpeg
mock_result = MagicMock()
mock_result.stdout = "output"
mock_result.stderr = ""
with patch("subprocess.run", return_value=mock_result):
stdout, stderr = run_ffmpeg(["ffmpeg", "-version"])
assert stdout == "output"
assert stderr == ""
# ── P1: template_id + asset_ids 校验 ─────────────────────────────────────────
class TestP1Validations:
"""P1: template_id 存在性校验 + asset_ids 归属校验."""
def test_validate_template_exists_success(self):
"""模板存在时不抛异常."""
from worker_app.tasks.generation import _validate_template_exists
mock_template = MagicMock()
mock_template.id = "tmpl_001"
mock_template.name = "Test Template"
mock_template.is_active = True
session = MagicMock()
mock_session = MagicMock()
session.query.return_value = mock_session
filter_result = MagicMock()
mock_session.filter.return_value = filter_result
filter_result.first.return_value = mock_template
with patch("worker_app.tasks.generation.SessionLocal", return_value=session):
_validate_template_exists("tmpl_001") # 不抛异常
def test_validate_template_exists_not_found(self):
"""模板不存在时抛 ValueError."""
from worker_app.tasks.generation import _validate_template_exists
session = MagicMock()
mock_session = MagicMock()
session.query.return_value = mock_session
filter_result = MagicMock()
mock_session.filter.return_value = filter_result
filter_result.first.return_value = None
with patch("worker_app.tasks.generation.SessionLocal", return_value=session):
with pytest.raises(ValueError, match="模板不存在"):
_validate_template_exists("tmpl_nonexistent")
@@ -0,0 +1,158 @@
"""P0-2 修复:OSS 凭证验证 + 启动诊断。
验证:
1. 非开发环境 OSS_ACCESS_KEY_ID/SECRET 为空时启动失败
2. 开发环境允许空凭证
3. diagnose() 方法正确输出配置状态
"""
import logging
import os
import sys
from unittest.mock import MagicMock, patch
import pytest
def _fresh_settings(env: str):
"""清除 config 模块缓存,以指定 APP_ENV 重新导入 Settings。
为非开发环境预设 OSS 环境变量,确保模块级 get_settings() 能成功完成导入。
测试方法内可根据需要清除这些变量来测试验证器。
"""
for mod_name in [m for m in list(sys.modules) if "app.config" in m]:
del sys.modules[mod_name]
os.environ["APP_ENV"] = env
# 非开发环境下,为模块级导入提供有效凭证(避免导入时验证失败)
if env != "development":
os.environ.setdefault("OSS_ACCESS_KEY_ID", "test-key-for-import")
os.environ.setdefault("OSS_ACCESS_KEY_SECRET", "test-secret-for-import")
# 重置单例,让测试方法自行控制实例化
from apps.api.app import config as _cfg
from apps.api.app.config import Settings
_cfg._settings = None
return Settings
class TestOSSCredentialValidation:
"""测试 OSS 凭证验证器(直接调用验证器类方法)。"""
def test_empty_oss_key_id_rejected_in_staging(self):
"""非开发环境 OSS_ACCESS_KEY_ID 为空应报错。"""
Settings = _fresh_settings("staging")
with pytest.raises(Exception, match="OSS_ACCESS_KEY_ID"):
Settings.validate_oss_access_key_id("")
def test_empty_oss_key_secret_rejected_in_staging(self):
"""非开发环境 OSS_ACCESS_KEY_SECRET 为空应报错。"""
Settings = _fresh_settings("staging")
with pytest.raises(Exception, match="OSS_ACCESS_KEY_SECRET"):
Settings.validate_oss_access_key_secret("")
def test_empty_oss_credentials_allowed_in_development(self):
"""开发环境允许空 OSS 凭证。"""
Settings = _fresh_settings("development")
assert Settings.validate_oss_access_key_id("") == ""
assert Settings.validate_oss_access_key_secret("") == ""
def test_valid_credentials_pass_validation(self):
"""有效凭证应通过验证。"""
Settings = _fresh_settings("staging")
assert Settings.validate_oss_access_key_id("test-key-id") == "test-key-id"
assert Settings.validate_oss_access_key_secret("test-key-secret") == "test-key-secret"
def test_valid_credentials_instantiation_succeeds(self):
"""有效凭证应能成功创建 Settings 实例。"""
os.environ.pop("OSS_ACCESS_KEY_ID", None)
os.environ.pop("OSS_ACCESS_KEY_SECRET", None)
Settings = _fresh_settings("staging")
os.environ["OSS_ACCESS_KEY_ID"] = "test-key-id"
os.environ["OSS_ACCESS_KEY_SECRET"] = "test-key-secret"
s = Settings(_env_file=None)
assert s.OSS_ACCESS_KEY_ID == "test-key-id"
assert s.OSS_ACCESS_KEY_SECRET == "test-key-secret"
class TestOSSDiagnose:
"""测试 OSSStorageService.diagnose() 方法。"""
@patch("apps.api.app.core.storage.oss2", None)
@patch("apps.api.app.core.storage.get_settings")
def test_diagnose_logs_error_when_bucket_none(self, mock_settings, caplog):
"""bucket=None 时 diagnose 应输出 ERROR 日志。"""
from apps.api.app.core.storage import OSSStorageService
mock_settings.return_value.OSS_BUCKET_NAME = "test-bucket"
mock_settings.return_value.OSS_ENDPOINT = "oss-cn-test.com"
mock_settings.return_value.OSS_ACCESS_KEY_ID = ""
mock_settings.return_value.OSS_ACCESS_KEY_SECRET = ""
service = OSSStorageService()
assert service.bucket is None
with caplog.at_level(logging.ERROR, logger="apps.api.app.core.storage"):
service.diagnose()
assert any("" in record.message for record in caplog.records)
@patch("apps.api.app.core.storage.oss2")
@patch("apps.api.app.core.storage.get_settings")
def test_diagnose_logs_success_when_bucket_configured(self, mock_settings, mock_oss2, caplog):
"""bucket 已配置时 diagnose 应输出成功日志。"""
from apps.api.app.core.storage import OSSStorageService
mock_settings.return_value.OSS_BUCKET_NAME = "test-bucket"
mock_settings.return_value.OSS_ENDPOINT = "oss-cn-test.com"
mock_settings.return_value.OSS_ACCESS_KEY_ID = "test-key-id"
mock_settings.return_value.OSS_ACCESS_KEY_SECRET = "test-key-secret"
mock_oss2.Bucket.return_value = MagicMock()
service = OSSStorageService()
assert service.bucket is not None
with caplog.at_level(logging.INFO, logger="apps.api.app.core.storage"):
service.diagnose()
assert any("OSS诊断" in record.message for record in caplog.records)
class TestOSSHTTPSEndpoint:
"""测试 P0-2 真正根因:sign_url 必须返回 HTTPS URL。"""
@patch("apps.api.app.core.storage.oss2")
@patch("apps.api.app.core.storage.get_settings")
def test_endpoint_without_scheme_gets_https_prefix(self, mock_settings, mock_oss2):
"""endpoint 无 scheme 时应自动加 https://,确保 sign_url 生成 HTTPS URL。"""
from apps.api.app.core.storage import OSSStorageService
mock_settings.return_value.OSS_BUCKET_NAME = "test-bucket"
mock_settings.return_value.OSS_ENDPOINT = "oss-cn-hangzhou.aliyuncs.com"
mock_settings.return_value.OSS_ACCESS_KEY_ID = "test-key-id"
mock_settings.return_value.OSS_ACCESS_KEY_SECRET = "test-key-secret"
mock_oss2.Bucket.return_value = MagicMock()
OSSStorageService()
# 验证传给 oss2.Bucket 的 endpoint 带了 https://
call_args = mock_oss2.Bucket.call_args
endpoint_passed = call_args[0][1] # 第二个位置参数
assert endpoint_passed == "https://oss-cn-hangzhou.aliyuncs.com"
@patch("apps.api.app.core.storage.oss2")
@patch("apps.api.app.core.storage.get_settings")
def test_endpoint_with_existing_https_not_doubled(self, mock_settings, mock_oss2):
"""endpoint 已有 https:// 时不应重复添加。"""
from apps.api.app.core.storage import OSSStorageService
mock_settings.return_value.OSS_BUCKET_NAME = "test-bucket"
mock_settings.return_value.OSS_ENDPOINT = "https://oss-cn-hangzhou.aliyuncs.com"
mock_settings.return_value.OSS_ACCESS_KEY_ID = "test-key-id"
mock_settings.return_value.OSS_ACCESS_KEY_SECRET = "test-key-secret"
mock_oss2.Bucket.return_value = MagicMock()
OSSStorageService()
call_args = mock_oss2.Bucket.call_args
endpoint_passed = call_args[0][1]
assert endpoint_passed == "https://oss-cn-hangzhou.aliyuncs.com"
+379
View File
@@ -0,0 +1,379 @@
"""P0-2 / P0-3 修复验证测试.
P0-2: OSSStorageService.get_download_url 预签名 URL 逻辑验证
P0-3: FFmpeg xfade exit 234 — build_xfade_filter_chain 安全钳制 + effective_duration trim
"""
from __future__ import annotations
import logging
from unittest.mock import MagicMock, patch
import pytest
from video_processing.ffmpeg_utils import build_xfade_filter_chain
# ── P0-3: build_xfade_filter_chain 安全钳制 ──────────────────────────────────
class TestBuildXfadeFilterChainSafetyClamp:
"""验证 xfade 滤镜链的安全钳制逻辑,防止 exit 234。"""
def test_empty_clips(self):
"""空片段列表返回空字符串和 0 时长。"""
f, dur = build_xfade_filter_chain([], [], [])
assert f == ""
assert dur == 0.0
def test_single_clip(self):
"""单片段用 copy,不做 xfade。"""
f, dur = build_xfade_filter_chain([3.0], ["v0"], ["cut"])
assert "copy" in f
assert dur == 3.0
def test_two_clips_normal(self):
"""两个正常时长片段 — offset + td ≤ first_input_duration。"""
f, dur = build_xfade_filter_chain(
clip_durations=[5.0, 5.0],
clip_video_labels=["v0", "v1"],
transitions=["cut", "fade"],
transition_duration=0.5,
)
assert "xfade" in f
assert "duration=0.500" in f
# 总时长 = 5 + 5 - 0.5 = 9.5
assert abs(dur - 9.5) < 0.01
def test_short_clip_safety_clamp(self):
"""片段短于 transition_duration 时,td 被钳制,不会导致 exit 234。
这是 P0-3 的核心场景:视频只有 1std=0.5soffset 计算后
offset + td 可能超过 first_input_duration。
"""
# 两个 1s 片段,td=0.5s
# 原始 offset = max(0, 1.0 - 0.5*1) = 0.5
# first_input_dur = 1.0, available = 1.0 - 0.5 = 0.5
# safe_td = min(0.5, 0.5) = 0.5 — 刚好 OK
f, dur = build_xfade_filter_chain(
clip_durations=[1.0, 1.0],
clip_video_labels=["v0", "v1"],
transitions=["cut", "fade"],
transition_duration=0.5,
)
assert "xfade" in f
# offset + td 必须 ≤ first_input_duration
# offset=0.5, td=0.5 → 0.5+0.5=1.0 ≤ 1.0 ✓
assert dur > 0
def test_very_short_clip_clamped(self):
"""片段极短(0.3s),td 被钳制到 available 以内。"""
# clip1=0.3s, clip2=5.0s, td=0.5s
# offset = max(0, 0.3 - 0.5) = 0.0
# first_input_dur = 0.3
# available = 0.3 - 0.0 = 0.3
# safe_td = min(0.5, 0.3) = 0.3
f, dur = build_xfade_filter_chain(
clip_durations=[0.3, 5.0],
clip_video_labels=["v0", "v1"],
transitions=["cut", "fade"],
transition_duration=0.5,
)
assert "duration=0.300" in f # td 被钳制到 0.3
def test_multi_clip_chain_clamp(self):
"""多片段链式 xfade,每步都钳制。"""
# 3 个 0.5s 片段,td=0.5s
# Step 1: offset=0, first_input_dur=0.5, available=0.5, safe_td=0.5
# total_transition=0.5
# Step 2: cumulative=1.0, first_input_dur=1.0-0.5=0.5
# offset=max(0, 1.0-0.5*2)=0.0, available=0.5, safe_td=0.5
f, dur = build_xfade_filter_chain(
clip_durations=[0.5, 0.5, 0.5],
clip_video_labels=["v0", "v1", "v2"],
transitions=["cut", "fade", "fade"],
transition_duration=0.5,
)
assert "xfade" in f
assert f.count("xfade") == 2
assert dur > 0
def test_middle_clip_shorter_than_td(self):
"""P1 修复验证:中间片段短于 td 时,td 被钳制到该片段时长。
[5.0, 0.3, 5.0] + td=0.5s:
- Step 1: second input = 0.3s, safe_td 必须 ≤ 0.3
- Step 2: second input = 5.0s, safe_td 可以 = 0.5
"""
import re
f, dur = build_xfade_filter_chain(
clip_durations=[5.0, 0.3, 5.0],
clip_video_labels=["v0", "v1", "v2"],
transitions=["cut", "fade", "fade"],
transition_duration=0.5,
)
assert f.count("xfade") == 2
# 解析每个 xfade 的 duration
durations_found = []
for part in f.split(";"):
if "xfade=" not in part:
continue
m = re.search(r"duration=([\d.]+)", part)
assert m, f"无法解析: {part}"
durations_found.append(float(m.group(1)))
# 第一个 xfade: td 必须 ≤ 0.3 (第二个输入 clip_durations[1]=0.3)
assert durations_found[0] <= 0.3 + 0.001, (
f"第一个 xfade td={durations_found[0]} 超过 clip_durations[1]=0.3"
)
# 第二个 xfade: td 可以 = 0.5 (clip_durations[2]=5.0)
assert durations_found[1] <= 0.5 + 0.001
assert dur > 0
def test_offset_plus_td_never_exceeds_input(self):
"""压力测试:多种时长组合,offset + td 永远不超过 first_input_duration。"""
test_cases = [
([0.1, 5.0], 0.5),
([0.5, 0.5], 0.5),
([1.0, 1.0, 1.0], 0.5),
([0.2, 0.3, 0.4], 0.5),
([10.0, 0.1], 0.5),
([3.0, 3.0, 3.0, 3.0], 0.5),
([5.0, 0.3, 5.0], 0.5), # P1: 中间片段短于 td
([5.0, 0.1, 0.1, 5.0], 0.5), # P1: 多个中间片段都短于 td
]
for durations, td in test_cases:
labels = [f"v{i}" for i in range(len(durations))]
transitions = ["cut"] + ["fade"] * (len(durations) - 1)
f, dur = build_xfade_filter_chain(
clip_durations=durations,
clip_video_labels=labels,
transitions=transitions,
transition_duration=td,
)
assert dur >= 0, f" durations={durations} td={td} → dur={dur}"
# 解析 filter 验证 offset + td 的合理性
import re
# 按 xfade 步骤索引追踪第二个输入
xfade_idx = 0
for part in f.split(";"):
if "xfade=" not in part:
continue
# 格式: [prev][next]xfade=transition=X:duration=D:offset=O[out]
m = re.search(
r"xfade=transition=(\w+):duration=([\d.]+):offset=([\d.]+)",
part,
)
assert m, f"无法解析 xfade 参数: {part}"
offset_val = float(m.group(3))
dur_val = float(m.group(2))
assert offset_val >= 0
assert dur_val >= 0.001 # 至少 1ms
# P1 修复验证: td 不能超过第二个输入片段时长
second_input_idx = xfade_idx + 1
assert dur_val <= durations[second_input_idx] + 0.001, (
f"td={dur_val} > clip_durations[{second_input_idx}]={durations[second_input_idx]}"
)
xfade_idx += 1
# ── P0-3: effective_duration trim 逻辑验证 ────────────────────────────────────
class TestEffectiveDurationTrim:
"""验证 _build_filter_complex 中 effective_duration trim 逻辑。"""
def _make_service(self, clips, asset_paths=None):
from pathlib import Path
from video_processing.unified_render_service import UnifiedRenderService
plan = MagicMock()
plan.id = "test_plan"
work_dir = Path("/tmp/test_render")
if asset_paths is None:
asset_paths = {}
for c in clips:
asset_paths[c.asset_id] = Path(f"/tmp/{c.asset_id}")
return UnifiedRenderService(
plan=plan,
clips=clips,
asset_path_map=asset_paths,
work_dir=work_dir,
)
def _make_clip(self, clip_id, duration=0.0, actual_duration=5.0, clip_type="main", order=0):
from pathlib import Path
from video_processing.unified_render_service import ResolvedClip
return ResolvedClip(
clip_id=clip_id,
asset_id=f"asset_{clip_id}.mp4",
local_path=Path(f"/tmp/asset_{clip_id}.mp4"),
clip_type=clip_type,
order=order,
duration=duration,
actual_duration=actual_duration,
transition_effect="fade",
config={},
)
def test_trim_applied_when_duration_less_than_actual(self):
"""clip.duration < actual_duration → trim=duration=clip.duration。"""
clip = self._make_clip("c1", duration=3.0, actual_duration=10.0)
svc = self._make_service([clip])
layers = svc._group_clips_into_layers([clip])
fc, _ = svc._build_filter_complex(layers)
assert "trim=duration=3.0" in fc
def test_trim_uses_actual_when_no_duration_set(self):
"""clip.duration=0 → 使用 actual_duration 做 trim。"""
clip = self._make_clip("c1", duration=0.0, actual_duration=7.5)
svc = self._make_service([clip])
layers = svc._group_clips_into_layers([clip])
fc, _ = svc._build_filter_complex(layers)
assert "trim=duration=7.5" in fc
def test_trim_uses_min_of_duration_and_actual(self):
"""clip.duration > actual_duration → trim 到 actual_duration。"""
clip = self._make_clip("c1", duration=10.0, actual_duration=2.0)
svc = self._make_service([clip])
layers = svc._group_clips_into_layers([clip])
fc, _ = svc._build_filter_complex(layers)
assert "trim=duration=2.0" in fc
def test_no_trim_when_both_zero(self):
"""duration=0 且 actual_duration=0 → 不做 trim。"""
clip = self._make_clip("c1", duration=0.0, actual_duration=0.0)
svc = self._make_service([clip])
layers = svc._group_clips_into_layers([clip])
fc, _ = svc._build_filter_complex(layers)
assert "trim=" not in fc
def test_xfade_uses_effective_durations(self):
"""多片段 xfade 使用 trim 后的有效时长。"""
clip1 = self._make_clip("c1", duration=3.0, actual_duration=10.0, order=0)
clip2 = self._make_clip("c2", duration=4.0, actual_duration=10.0, order=1)
svc = self._make_service([clip1, clip2])
layers = svc._group_clips_into_layers([clip1, clip2])
fc, _ = svc._build_filter_complex(layers)
assert "xfade=" in fc
# 两个 clip 的 trim 应该分别用 3.0 和 4.0
assert "trim=duration=3.0" in fc
assert "trim=duration=4.0" in fc
# ── P0-2: get_download_url 预签名 URL 逻辑验证 ────────────────────────────────
class TestGetDownloadUrl:
"""验证 OSSStorageService.get_download_url 逻辑。"""
def test_returns_signed_url_when_bucket_configured(self):
"""bucket 已配置 → 返回签名 URL。"""
from app.core.storage import OSSStorageService
with patch.object(OSSStorageService, "__init__", lambda self: None):
svc = OSSStorageService()
svc.bucket = MagicMock()
svc.bucket.sign_url.return_value = "https://signed-url.oss.com/file.mp4?signature=xxx"
svc.public_url = "https://bucket.oss.com"
result = svc.get_download_url("uploads/video.mp4")
svc.bucket.sign_url.assert_called_once_with("GET", "uploads/video.mp4", 3600)
assert "signed-url" in result
def test_returns_raw_url_when_bucket_none(self):
"""bucket 未配置 → 返回原始公网 URL,并记录 warning。"""
from app.core.storage import OSSStorageService
with patch.object(OSSStorageService, "__init__", lambda self: None):
svc = OSSStorageService()
svc.bucket = None
svc.public_url = "https://bucket.oss.com"
svc.local_url_prefix = "/generated-files"
with patch.object(svc, "_normalize_storage_key", return_value="uploads/video.mp4"):
result = svc.get_download_url("uploads/video.mp4")
assert result == "https://bucket.oss.com/uploads/video.mp4"
def test_local_generated_url_returned_as_is(self):
"""本地生成文件 URL → 原样返回,不走 OSS。"""
from app.core.storage import OSSStorageService
with patch.object(OSSStorageService, "__init__", lambda self: None):
svc = OSSStorageService()
svc.bucket = None
svc.local_url_prefix = "/generated-files"
result = svc.get_download_url("/generated-files/abc123.mp4")
assert result == "/generated-files/abc123.mp4"
def test_normalize_strips_full_url(self):
"""完整 URL → 提取路径部分作为 storage_key。"""
from app.core.storage import OSSStorageService
with patch.object(OSSStorageService, "__init__", lambda self: None):
svc = OSSStorageService()
key = svc._normalize_storage_key("https://bucket.oss-cn-hangzhou.aliyuncs.com/uploads/video.mp4")
assert key == "uploads/video.mp4"
def test_normalize_preserves_plain_key(self):
"""纯路径 → 保持不变。"""
from app.core.storage import OSSStorageService
with patch.object(OSSStorageService, "__init__", lambda self: None):
svc = OSSStorageService()
key = svc._normalize_storage_key("uploads/video.mp4")
assert key == "uploads/video.mp4"
def test_sign_url_failure_falls_back(self):
"""sign_url 异常 → 回退到原始 URL,不崩溃。"""
from app.core.storage import OSSStorageService
with patch.object(OSSStorageService, "__init__", lambda self: None):
svc = OSSStorageService()
svc.bucket = MagicMock()
svc.bucket.sign_url.side_effect = Exception("OSS error")
svc.public_url = "https://bucket.oss.com"
with patch.object(svc, "_normalize_storage_key", return_value="uploads/video.mp4"):
result = svc.get_download_url("uploads/video.mp4")
assert result == "https://bucket.oss.com/uploads/video.mp4"
def test_diagnostic_logging_on_bucket_none(self, caplog):
"""bucket 未配置时记录 warning 日志。"""
from app.core.storage import OSSStorageService
with patch.object(OSSStorageService, "__init__", lambda self: None):
svc = OSSStorageService()
svc.bucket = None
svc.public_url = "https://bucket.oss.com"
svc.local_url_prefix = "/generated-files"
with patch.object(svc, "_normalize_storage_key", return_value="uploads/video.mp4"):
with caplog.at_level(logging.WARNING):
svc.get_download_url("https://bucket.oss.com/uploads/video.mp4")
assert any("OSS bucket not configured" in r.message for r in caplog.records)
def test_diagnostic_logging_on_sign_success(self, caplog):
"""签名成功时记录 info 日志。"""
from app.core.storage import OSSStorageService
with patch.object(OSSStorageService, "__init__", lambda self: None):
svc = OSSStorageService()
svc.bucket = MagicMock()
svc.bucket.sign_url.return_value = "https://signed.oss.com/file.mp4?sig=xxx"
svc.public_url = "https://bucket.oss.com"
with caplog.at_level(logging.INFO):
svc.get_download_url("uploads/video.mp4")
assert any("signed URL generated" in r.message for r in caplog.records)
+598
View File
@@ -0,0 +1,598 @@
"""
PlanGeneratorService 单元测试
覆盖(6 组测试):
- ONE_TAKE 模式:素材顺序分配给 main clips
- PIP 模式:第1个素材→main,其余→overlay
- VOICE_OVER 模式:素材→main clips (B-roll)
- VOICE_PIP 模式:第1个→background, 第2个→corner_voice, 其余→b_roll
- 无 clip_configs 时自动生成默认结构
- 空素材列表时 clips 创建但无素材分配
"""
from __future__ import annotations
import os
import sys
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, List, Optional
from unittest.mock import MagicMock, patch
os.environ.setdefault("JWT_SECRET_KEY", "unit-test-secret-key-for-testing")
os.environ.setdefault("DATABASE_URL", "sqlite:///test.db")
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[2] / "apps" / "api"))
from packages.domain.edit_plan import EditPlan, EditPlanStatus
from packages.domain.edit_plan_clip import EditPlanClip, EditPlanClipStatus
from packages.domain.edit_template import EditTemplate, EditTemplateStatus
from packages.domain.editing_mode import EditingMode
from packages.domain.template_clip_config import ClipType, TemplateClipConfig, TransitionEffect
# ---------------------------------------------------------------------------
# Stub Repositories
# ---------------------------------------------------------------------------
class StubEditPlanRepository:
"""内存中的 EditPlan 仓储 stub"""
def __init__(self) -> None:
self._plans: dict[str, EditPlan] = {}
self._counter = 0
def _next_id(self) -> str:
self._counter += 1
return f"plan-{self._counter:03d}"
def get(self, plan_id: str) -> Optional[EditPlan]:
return self._plans.get(plan_id)
def create(self, plan: EditPlan) -> EditPlan:
if not plan.id:
plan = EditPlan(
id=self._next_id(),
template_id=plan.template_id,
name=plan.name,
status=plan.status,
total_duration=plan.total_duration,
source_edit_plan_id=plan.source_edit_plan_id,
project_id=plan.project_id,
created_by_user_id=plan.created_by_user_id,
config=plan.config,
created_at=plan.created_at,
updated_at=plan.updated_at,
)
self._plans[plan.id] = plan
return plan
def update(self, plan: EditPlan) -> EditPlan:
self._plans[plan.id] = plan
return plan
def list_all(self, **kwargs) -> List[EditPlan]:
return list(self._plans.values())
def count(self, **kwargs) -> int:
return len(self._plans)
def delete(self, plan_id: str) -> bool:
return self._plans.pop(plan_id, None) is not None
class StubEditPlanClipRepository:
"""内存中的 EditPlanClip 仓储 stub"""
def __init__(self) -> None:
self._clips: dict[str, EditPlanClip] = {}
self._counter = 0
def _next_id(self) -> str:
self._counter += 1
return f"clip-{self._counter:03d}"
def get(self, clip_id: str) -> Optional[EditPlanClip]:
return self._clips.get(clip_id)
def create(self, clip: EditPlanClip) -> EditPlanClip:
if not clip.id:
clip = EditPlanClip(
id=self._next_id(),
plan_id=clip.plan_id,
template_clip_config_id=clip.template_clip_config_id,
clip_type=clip.clip_type,
order=clip.order,
asset_id=clip.asset_id,
text_content=clip.text_content,
start_time=clip.start_time,
duration=clip.duration,
transition_effect=clip.transition_effect,
status=clip.status,
config=clip.config,
created_at=clip.created_at,
updated_at=clip.updated_at,
)
self._clips[clip.id] = clip
return clip
def update(self, clip: EditPlanClip) -> EditPlanClip:
self._clips[clip.id] = clip
return clip
def list_by_plan(
self,
plan_id: str,
*,
status: Optional[EditPlanClipStatus] = None,
skip: int = 0,
limit: int = 100,
) -> List[EditPlanClip]:
items = [c for c in self._clips.values() if c.plan_id == plan_id]
items.sort(key=lambda c: c.order)
if status:
items = [c for c in items if c.status == status]
return items[skip : skip + limit]
def delete(self, clip_id: str) -> bool:
return self._clips.pop(clip_id, None) is not None
# ---------------------------------------------------------------------------
# Helper: 构建 PlanGeneratorServicepatch 仓储)
# ---------------------------------------------------------------------------
def _make_generator():
"""创建使用 stub 仓储的 PlanGeneratorService"""
from apps.api.app.services.plan_generator_service import PlanGeneratorService
plan_repo = StubEditPlanRepository()
clip_repo = StubEditPlanClipRepository()
with (
patch(
"apps.api.app.services.plan_generator_service.SQLAlchemyEditPlanRepository",
return_value=plan_repo,
),
patch(
"apps.api.app.services.plan_generator_service.SQLAlchemyEditPlanClipRepository",
return_value=clip_repo,
),
):
db = MagicMock()
svc = PlanGeneratorService(db)
# 替换为 stub
svc._plan_repo = plan_repo
svc._clip_repo = clip_repo
return svc, plan_repo, clip_repo
def _make_template(
editing_mode: str = "one_take",
config: Optional[dict] = None,
) -> EditTemplate:
"""创建测试用 EditTemplate"""
return EditTemplate(
id="tpl-001",
name="测试模板",
description="",
template_type="default",
editing_mode=editing_mode,
config=config or {},
preview_url="",
sort_weight=0,
status=EditTemplateStatus.ACTIVE,
created_at=datetime.now(timezone.utc),
updated_at=datetime.now(timezone.utc),
)
def _make_clip_configs(
template_id: str = "tpl-001",
specs: Optional[List[dict]] = None,
) -> List[TemplateClipConfig]:
"""创建测试用 TemplateClipConfig 列表
specs 示例: [{"clip_type": ClipType.INTRO, "order": 0}, ...]
"""
if specs is None:
specs = [
{"clip_type": ClipType.INTRO, "order": 0, "min_duration": 2.0, "max_duration": 4.0},
{"clip_type": ClipType.MAIN, "order": 1, "min_duration": 3.0, "max_duration": 7.0},
{"clip_type": ClipType.OUTRO, "order": 2, "min_duration": 2.0, "max_duration": 4.0},
]
configs = []
for i, spec in enumerate(specs):
cfg = TemplateClipConfig(
id=f"cfg-{i:03d}",
template_id=template_id,
clip_type=spec.get("clip_type", ClipType.MAIN),
order=spec.get("order", i),
min_duration=spec.get("min_duration", 0.0),
max_duration=spec.get("max_duration", 0.0),
text_template=spec.get("text_template", ""),
material_requirements=spec.get("material_requirements"),
transition_effect=spec.get("transition_effect", TransitionEffect.CUT),
config=spec.get("config"),
created_at=datetime.now(timezone.utc),
updated_at=datetime.now(timezone.utc),
)
configs.append(cfg)
return configs
# ---------------------------------------------------------------------------
# 测试:ONE_TAKE 模式
# ---------------------------------------------------------------------------
class TestGenerateOneTakePlan:
"""ONE_TAKE 模式:素材顺序分配给 main clips"""
def test_generate_one_take_plan(self):
"""3个clip_configs + 3个asset_ids → 按顺序分配"""
svc, plan_repo, clip_repo = _make_generator()
template = _make_template(editing_mode=EditingMode.ONE_TAKE.value)
clip_configs = _make_clip_configs()
asset_ids = ["asset-1", "asset-2", "asset-3"]
result = svc.generate_from_template(
template=template,
clip_configs=clip_configs,
asset_ids=asset_ids,
project_id="proj-001",
created_by_user_id="user-001",
)
plan = result["plan"]
clips = result["clips"]
assert plan.template_id == "tpl-001"
assert plan.config["editing_mode"] == "one_take"
assert plan.status == EditPlanStatus.EDITING
assert len(clips) == 3
# 按 order 排序后检查素材分配
sorted_clips = sorted(clips, key=lambda c: c.order)
# intro clip (order=0) 不是 main 类型,不分配素材
# main clip (order=1) → asset-2ONE_TAKE 只分配给 main clips
# outro clip (order=2) 不是 main 类型
main_clips = [c for c in sorted_clips if c.clip_type == ClipType.MAIN.value]
assert len(main_clips) == 1
assert main_clips[0].asset_id == "asset-1"
def test_one_take_plan_name_from_template(self):
"""name 为空时自动取模板名"""
svc, _, _ = _make_generator()
template = _make_template(editing_mode=EditingMode.ONE_TAKE.value)
result = svc.generate_from_template(
template=template,
clip_configs=[],
asset_ids=["asset-1"],
)
assert "测试模板" in result["plan"].name
def test_one_take_custom_name(self):
"""指定 name 时使用自定义名称"""
svc, _, _ = _make_generator()
template = _make_template(editing_mode=EditingMode.ONE_TAKE.value)
result = svc.generate_from_template(
template=template,
clip_configs=[],
asset_ids=["asset-1"],
name="我的剪辑",
)
assert result["plan"].name == "我的剪辑"
# ---------------------------------------------------------------------------
# 测试:PIP 模式
# ---------------------------------------------------------------------------
class TestGeneratePipPlan:
"""PIP 模式:第1个素材→main,其余→overlay"""
def test_generate_pip_plan(self):
"""4个素材 → 第1个→main,其余→overlay"""
svc, _, _ = _make_generator()
template = _make_template(editing_mode=EditingMode.PIP.value)
# 无 clip_configs,自动生成默认结构
asset_ids = ["bg-asset", "overlay-1", "overlay-2", "overlay-3"]
result = svc.generate_from_template(
template=template,
clip_configs=[],
asset_ids=asset_ids,
)
plan = result["plan"]
clips = result["clips"]
assert plan.config["editing_mode"] == "pip"
# 自动生成: 1 main + 3 overlay
assert len(clips) == 4
main_clips = [c for c in clips if c.clip_type == ClipType.MAIN.value]
overlay_clips = [c for c in clips if c.clip_type == "overlay"]
assert len(main_clips) == 1
assert len(overlay_clips) == 3
# 第1个素材 → main
assert main_clips[0].asset_id == "bg-asset"
# 其余 → overlay
assert overlay_clips[0].asset_id == "overlay-1"
assert overlay_clips[1].asset_id == "overlay-2"
assert overlay_clips[2].asset_id == "overlay-3"
# ---------------------------------------------------------------------------
# 测试:VOICE_OVER 模式
# ---------------------------------------------------------------------------
class TestGenerateVoiceOverPlan:
"""VOICE_OVER 模式:素材→main clips (B-roll)"""
def test_generate_voice_over_plan(self):
"""3个素材 → 3个 main clips,每个标记为 b_roll"""
svc, _, _ = _make_generator()
template = _make_template(editing_mode=EditingMode.VOICE_OVER.value)
asset_ids = ["video-1", "video-2", "video-3"]
result = svc.generate_from_template(
template=template,
clip_configs=[],
asset_ids=asset_ids,
)
plan = result["plan"]
clips = result["clips"]
assert plan.config["editing_mode"] == "voice_over"
assert len(clips) == 3
# 所有 clips 都是 main 类型
for clip in clips:
assert clip.clip_type == ClipType.MAIN.value
# 素材按顺序分配
assert clips[0].asset_id == "video-1"
assert clips[1].asset_id == "video-2"
assert clips[2].asset_id == "video-3"
# 每个 clip 的 config 标记为 b_roll
for clip in clips:
assert clip.config.get("role") == "b_roll"
# ---------------------------------------------------------------------------
# 测试:VOICE_PIP 模式
# ---------------------------------------------------------------------------
class TestGenerateVoicePipPlan:
"""VOICE_PIP 模式:第1个→background, 第2个→corner_voice, 其余→b_roll"""
def test_generate_voice_pip_plan(self):
"""4个素材 → background + corner_voice + 2 b_roll"""
svc, _, _ = _make_generator()
template = _make_template(editing_mode=EditingMode.VOICE_PIP.value)
asset_ids = ["bg-video", "corner-video", "broll-1", "broll-2"]
result = svc.generate_from_template(
template=template,
clip_configs=[],
asset_ids=asset_ids,
)
plan = result["plan"]
clips = result["clips"]
assert plan.config["editing_mode"] == "voice_pip"
assert len(clips) == 4
bg_clips = [c for c in clips if c.clip_type == "background"]
corner_clips = [c for c in clips if c.clip_type == "corner_voice"]
broll_clips = [c for c in clips if c.clip_type == "b_roll"]
assert len(bg_clips) == 1
assert len(corner_clips) == 1
assert len(broll_clips) == 2
# 素材分配
assert bg_clips[0].asset_id == "bg-video"
assert corner_clips[0].asset_id == "corner-video"
assert broll_clips[0].asset_id == "broll-1"
assert broll_clips[1].asset_id == "broll-2"
# ---------------------------------------------------------------------------
# 测试:无 clip_configs 时自动生成默认结构
# ---------------------------------------------------------------------------
class TestGenerateWithoutClipConfigs:
"""无 clip_configs 时根据 editing_mode 生成默认 clip 结构"""
def test_one_take_default_clips(self):
"""ONE_TAKE + 3个素材 → 3个 main clips"""
svc, _, _ = _make_generator()
template = _make_template(editing_mode=EditingMode.ONE_TAKE.value)
result = svc.generate_from_template(
template=template,
clip_configs=[],
asset_ids=["a1", "a2", "a3"],
)
clips = result["clips"]
assert len(clips) == 3
for clip in clips:
assert clip.clip_type == ClipType.MAIN.value
def test_pip_default_clips(self):
"""PIP + 3个素材 → 1 main + 2 overlay"""
svc, _, _ = _make_generator()
template = _make_template(editing_mode=EditingMode.PIP.value)
result = svc.generate_from_template(
template=template,
clip_configs=[],
asset_ids=["a1", "a2", "a3"],
)
clips = result["clips"]
assert len(clips) == 3
main_clips = [c for c in clips if c.clip_type == ClipType.MAIN.value]
overlay_clips = [c for c in clips if c.clip_type == "overlay"]
assert len(main_clips) == 1
assert len(overlay_clips) == 2
def test_voice_pip_default_clips(self):
"""VOICE_PIP + 4个素材 → 1 background + 1 corner_voice + 2 b_roll"""
svc, _, _ = _make_generator()
template = _make_template(editing_mode=EditingMode.VOICE_PIP.value)
result = svc.generate_from_template(
template=template,
clip_configs=[],
asset_ids=["a1", "a2", "a3", "a4"],
)
clips = result["clips"]
assert len(clips) == 4
# ---------------------------------------------------------------------------
# 测试:空素材列表
# ---------------------------------------------------------------------------
class TestGenerateEmptyAssets:
"""空素材列表时 clips 创建但无素材分配"""
def test_empty_assets(self):
"""空 asset_ids → clips 创建但 asset_id 为空"""
svc, _, _ = _make_generator()
template = _make_template(editing_mode=EditingMode.ONE_TAKE.value)
clip_configs = _make_clip_configs()
result = svc.generate_from_template(
template=template,
clip_configs=clip_configs,
asset_ids=[],
)
plan = result["plan"]
clips = result["clips"]
assert plan.total_duration > 0 # clips 有默认时长
assert len(clips) == 3
for clip in clips:
assert clip.asset_id == ""
def test_empty_assets_pip(self):
"""PIP 模式空素材 → 1个 main clip(至少1个)"""
svc, _, _ = _make_generator()
template = _make_template(editing_mode=EditingMode.PIP.value)
result = svc.generate_from_template(
template=template,
clip_configs=[],
asset_ids=[],
)
clips = result["clips"]
# 至少1个 main clipn = max(asset_count, 1) = 1
assert len(clips) == 1
assert clips[0].clip_type == ClipType.MAIN.value
# ---------------------------------------------------------------------------
# 测试:plan config 继承模板配置
# ---------------------------------------------------------------------------
class TestPlanConfigInheritance:
"""plan config 继承模板的 cover/title/subtitle/bgm"""
def test_inherit_template_config(self):
"""模板有 cover/title/bgm 配置 → plan 继承"""
svc, _, _ = _make_generator()
template_config = {
"editing_mode": "one_take",
"cover": {"type": "ai_frame"},
"title": {"text": "测试标题", "font": "思源黑体"},
"bgm": {"url": "https://example.com/bgm.mp3"},
}
template = _make_template(
editing_mode=EditingMode.ONE_TAKE.value,
config=template_config,
)
result = svc.generate_from_template(
template=template,
clip_configs=[],
asset_ids=["a1"],
)
plan_config = result["plan"].config
assert plan_config["editing_mode"] == "one_take"
assert plan_config["cover"]["type"] == "ai_frame"
assert plan_config["title"]["text"] == "测试标题"
assert plan_config["bgm"]["url"] == "https://example.com/bgm.mp3"
# ---------------------------------------------------------------------------
# 测试:total_duration 计算
# ---------------------------------------------------------------------------
class TestTotalDuration:
"""total_duration 正确计算"""
def test_duration_from_clip_configs(self):
"""有 clip_configs 时,duration 取 min/max 中间值"""
svc, _, _ = _make_generator()
template = _make_template(editing_mode=EditingMode.ONE_TAKE.value)
clip_configs = _make_clip_configs(
specs=[
{"clip_type": ClipType.INTRO, "order": 0, "min_duration": 2.0, "max_duration": 4.0},
{"clip_type": ClipType.MAIN, "order": 1, "min_duration": 4.0, "max_duration": 6.0},
{"clip_type": ClipType.OUTRO, "order": 2, "min_duration": 2.0, "max_duration": 4.0},
]
)
result = svc.generate_from_template(
template=template,
clip_configs=clip_configs,
asset_ids=["a1"],
)
# intro: (2+4)/2=3, main: (4+6)/2=5, outro: (2+4)/2=3 → total=11
assert result["plan"].total_duration == 11.0
def test_duration_from_default_clips(self):
"""无 clip_configs 时,每个 clip 默认 5 秒"""
svc, _, _ = _make_generator()
template = _make_template(editing_mode=EditingMode.ONE_TAKE.value)
result = svc.generate_from_template(
template=template,
clip_configs=[],
asset_ids=["a1", "a2", "a3"],
)
# 3 个 clips × 5 秒 = 15 秒
assert result["plan"].total_duration == 15.0
+190
View File
@@ -0,0 +1,190 @@
"""
EditTemplate editing_mode 字段单元测试
覆盖:
- EditTemplate.create() 带 editing_mode
- 默认值 "one_take"
- 无效 editing_mode 抛 ValueError
- EditingMode 枚举值完整性
- config_schemas 中 editing_mode 和 transition_enabled 字段
"""
from __future__ import annotations
import os
import sys
from datetime import datetime, timezone
from pathlib import Path
os.environ.setdefault("JWT_SECRET_KEY", "unit-test-secret-key-for-testing")
os.environ.setdefault("DATABASE_URL", "sqlite:///test.db")
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[2] / "apps" / "api"))
from packages.domain.config_schemas import (
DEFAULT_EDIT_PLAN_CONFIG,
DEFAULT_EDIT_TEMPLATE_CONFIG,
normalize_plan_config,
normalize_template_config,
)
from packages.domain.edit_template import EditTemplate, EditTemplateStatus
from packages.domain.editing_mode import EditingMode
# ---------------------------------------------------------------------------
# 测试:EditTemplate.create() 带 editing_mode
# ---------------------------------------------------------------------------
class TestEditTemplateEditingMode:
"""EditTemplate editing_mode 字段测试"""
def test_create_with_editing_mode(self):
"""create() 指定 editing_mode"""
tpl = EditTemplate.create(
name="测试模板",
editing_mode=EditingMode.PIP.value,
)
assert tpl.editing_mode == "pip"
def test_create_default_editing_mode(self):
"""create() 不指定 editing_mode → 默认 one_take"""
tpl = EditTemplate.create(name="默认模式模板")
assert tpl.editing_mode == "one_take"
def test_create_all_editing_modes(self):
"""所有 EditingMode 枚举值均可创建"""
for mode in EditingMode:
tpl = EditTemplate.create(
name=f"模板-{mode.value}",
editing_mode=mode.value,
)
assert tpl.editing_mode == mode.value
def test_create_invalid_editing_mode(self):
"""无效 editing_mode 抛 ValueError"""
with pytest.raises(ValueError, match="无效的 editing_mode"):
EditTemplate.create(
name="无效模板",
editing_mode="invalid_mode",
)
def test_editing_mode_case_sensitive(self):
"""editing_mode 大小写敏感"""
with pytest.raises(ValueError):
EditTemplate.create(
name="大写模式",
editing_mode="ONE_TAKE", # 应小写
)
# ---------------------------------------------------------------------------
# 测试:EditingMode 枚举完整性
# ---------------------------------------------------------------------------
class TestEditingModeEnum:
"""EditingMode 枚举值测试"""
def test_enum_values(self):
"""枚举包含4种模式"""
assert EditingMode.ONE_TAKE.value == "one_take"
assert EditingMode.PIP.value == "pip"
assert EditingMode.VOICE_OVER.value == "voice_over"
assert EditingMode.VOICE_PIP.value == "voice_pip"
def test_enum_count(self):
"""枚举共4个成员"""
assert len(EditingMode) == 4
def test_str_enum(self):
"""EditingMode 是 StrEnum"""
assert isinstance(EditingMode.ONE_TAKE, str)
assert EditingMode.ONE_TAKE == "one_take"
# ---------------------------------------------------------------------------
# 测试:config_schemas 中的 editing_mode
# ---------------------------------------------------------------------------
class TestConfigSchemasEditingMode:
"""config_schemas editing_mode 和 transition_enabled 字段测试"""
def test_default_plan_config_has_editing_mode(self):
"""DEFAULT_EDIT_PLAN_CONFIG 包含 editing_mode"""
assert "editing_mode" in DEFAULT_EDIT_PLAN_CONFIG
assert DEFAULT_EDIT_PLAN_CONFIG["editing_mode"] == "one_take"
def test_default_template_config_has_editing_mode(self):
"""DEFAULT_EDIT_TEMPLATE_CONFIG 包含 editing_mode"""
assert "editing_mode" in DEFAULT_EDIT_TEMPLATE_CONFIG
assert DEFAULT_EDIT_TEMPLATE_CONFIG["editing_mode"] == "one_take"
def test_default_template_config_has_transition_enabled(self):
"""DEFAULT_EDIT_TEMPLATE_CONFIG 包含 transition_enabled"""
assert "transition_enabled" in DEFAULT_EDIT_TEMPLATE_CONFIG
assert DEFAULT_EDIT_TEMPLATE_CONFIG["transition_enabled"] is True
def test_normalize_plan_config_editing_mode(self):
"""normalize_plan_config 处理 editing_mode"""
config = normalize_plan_config({"editing_mode": "pip"})
assert config["editing_mode"] == "pip"
def test_normalize_plan_config_default_editing_mode(self):
"""normalize_plan_config 空输入 → 默认 editing_mode"""
config = normalize_plan_config({})
assert config["editing_mode"] == "one_take"
def test_normalize_template_config_editing_mode(self):
"""normalize_template_config 处理 editing_mode"""
config = normalize_template_config({"editing_mode": "voice_pip"})
assert config["editing_mode"] == "voice_pip"
def test_normalize_template_config_transition_enabled(self):
"""normalize_template_config 处理 transition_enabled"""
config = normalize_template_config({"transition_enabled": False})
assert config["transition_enabled"] is False
def test_normalize_template_config_defaults(self):
"""normalize_template_config 空输入 → 默认值"""
config = normalize_template_config({})
assert config["editing_mode"] == "one_take"
assert config["transition_enabled"] is True
# ---------------------------------------------------------------------------
# 测试:EditTemplate 实体直接构造
# ---------------------------------------------------------------------------
class TestEditTemplateEntityDirect:
"""直接构造 EditTemplate 实体测试 editing_mode"""
def test_direct_construction(self):
"""直接构造带 editing_mode 的实体"""
now = datetime.now(timezone.utc)
tpl = EditTemplate(
id="tpl-test",
name="直接构造",
description="",
template_type="default",
editing_mode="voice_over",
config={},
preview_url="",
sort_weight=0,
status=EditTemplateStatus.ACTIVE,
created_at=now,
updated_at=now,
)
assert tpl.editing_mode == "voice_over"
def test_all_enum_values_accepted(self):
"""所有 EditingMode 枚举值均可通过 create() 验证"""
for mode in EditingMode:
tpl = EditTemplate.create(
name=f"模板-{mode.value}",
editing_mode=mode.value,
)
assert tpl.editing_mode == mode.value
+404
View File
@@ -0,0 +1,404 @@
"""UnifiedRenderService 单元测试.
测试图层分组算法、filter_complex 构建、以及渲染流程。
"""
from __future__ import annotations
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
from unittest.mock import MagicMock, patch
import pytest
from video_processing.unified_render_service import (
RenderLayer,
RenderResult,
ResolvedClip,
UnifiedRenderService,
_resolve_layer_role,
)
# ── Fixtures ──────────────────────────────────────────────────────────────────
@dataclass
class FakeClip:
"""模拟 EditPlanClip。"""
id: str
plan_id: str = "plan_001"
clip_type: str = "main"
order: int = 0
asset_id: str = ""
text_content: str = ""
start_time: float = 0.0
duration: float = 0.0
transition_effect: str = "cut"
status: str = "ready"
config: dict[str, Any] = field(default_factory=dict)
@dataclass
class FakePlan:
"""模拟 EditPlan。"""
id: str = "plan_001"
name: str = "测试计划"
def _make_clip(
clip_id: str,
clip_type: str = "main",
order: int = 0,
asset_id: str = "",
duration: float = 0.0,
transition_effect: str = "cut",
config: dict[str, Any] | None = None,
) -> FakeClip:
return FakeClip(
id=clip_id,
clip_type=clip_type,
order=order,
asset_id=asset_id or f"asset_{clip_id}.mp4",
duration=duration,
transition_effect=transition_effect,
config=config or {},
)
def _make_service(
clips: list[FakeClip] | None = None,
asset_paths: dict[str, Path] | None = None,
work_dir: Path | None = None,
) -> UnifiedRenderService:
"""创建测试用的 UnifiedRenderService 实例。
如果未提供 asset_paths,自动从 clips 生成默认映射
asset_id → /tmp/asset_{clip_id}.mp4)。
"""
plan = FakePlan()
clips = clips or []
work_dir = work_dir or Path("/tmp/test_render")
if asset_paths is None:
asset_paths = {}
for c in clips:
if c.asset_id:
asset_paths[c.asset_id] = Path(f"/tmp/{c.asset_id}")
return UnifiedRenderService(
plan=plan,
clips=clips,
asset_path_map=asset_paths,
work_dir=work_dir,
)
def _patch_path_exists():
"""Patch Path.exists() 让测试路径返回 True。"""
return patch("pathlib.Path.exists", return_value=True)
# ── 测试 _resolve_layer_role ─────────────────────────────────────────────────
class TestResolveLayerRole:
"""测试 clip_type → layer role 映射。"""
def test_main_default(self):
assert _resolve_layer_role("main", {}) == "main"
def test_main_with_b_roll_role(self):
assert _resolve_layer_role("main", {"role": "b_roll"}) == "broll"
def test_overlay(self):
assert _resolve_layer_role("overlay", {}) == "overlay"
def test_background(self):
assert _resolve_layer_role("background", {}) == "background"
def test_corner_voice(self):
assert _resolve_layer_role("corner_voice", {}) == "corner_voice"
def test_b_roll(self):
assert _resolve_layer_role("b_roll", {}) == "broll"
def test_intro(self):
assert _resolve_layer_role("intro", {}) == "main"
def test_outro(self):
assert _resolve_layer_role("outro", {}) == "main"
# ── 测试图层分组 ──────────────────────────────────────────────────────────────
class TestGroupClipsIntoLayers:
"""测试 _group_clips_into_layers 方法。"""
def test_group_clips_one_take(self):
"""4 个 main clips → 1 个 main_layer。"""
clips = [
_make_clip("c1", "main", order=0),
_make_clip("c2", "main", order=1),
_make_clip("c3", "main", order=2),
_make_clip("c4", "main", order=3),
]
svc = _make_service(clips)
with _patch_path_exists(), patch("video_processing.unified_render_service.probe_duration", return_value=5.0):
resolved = svc._resolve_clips()
layers = svc._group_clips_into_layers(resolved)
assert len(layers) == 1
assert layers[0].role == "main"
assert len(layers[0].clips) == 4
assert layers[0].z_index == 0
def test_group_clips_pip(self):
"""1 main + 2 overlay → main_layer + overlay_layer。"""
clips = [
_make_clip("c1", "main", order=0),
_make_clip("c2", "overlay", order=1),
_make_clip("c3", "overlay", order=2),
]
svc = _make_service(clips)
with _patch_path_exists(), patch("video_processing.unified_render_service.probe_duration", return_value=5.0):
resolved = svc._resolve_clips()
layers = svc._group_clips_into_layers(resolved)
roles = {lyr.role for lyr in layers}
assert "main" in roles
assert "overlay" in roles
main_layer = next(lyr for lyr in layers if lyr.role == "main")
overlay_layer = next(lyr for lyr in layers if lyr.role == "overlay")
assert len(main_layer.clips) == 1
assert len(overlay_layer.clips) == 2
assert overlay_layer.z_index > main_layer.z_index
def test_group_clips_voice_over(self):
"""3 个 main(b_roll) clips → 1 个 broll_layer。"""
clips = [
_make_clip("c1", "main", order=0, config={"role": "b_roll"}),
_make_clip("c2", "main", order=1, config={"role": "b_roll"}),
_make_clip("c3", "main", order=2, config={"role": "b_roll"}),
]
svc = _make_service(clips)
with _patch_path_exists(), patch("video_processing.unified_render_service.probe_duration", return_value=5.0):
resolved = svc._resolve_clips()
layers = svc._group_clips_into_layers(resolved)
assert len(layers) == 1
assert layers[0].role == "broll"
assert len(layers[0].clips) == 3
def test_group_clips_voice_pip(self):
"""1 background + 1 corner_voice + 2 b_roll → 3 layers。"""
clips = [
_make_clip("c1", "background", order=0),
_make_clip("c2", "corner_voice", order=1),
_make_clip("c3", "b_roll", order=2),
_make_clip("c4", "b_roll", order=3),
]
svc = _make_service(clips)
with _patch_path_exists(), patch("video_processing.unified_render_service.probe_duration", return_value=5.0):
resolved = svc._resolve_clips()
layers = svc._group_clips_into_layers(resolved)
roles = {lyr.role for lyr in layers}
assert roles == {"background", "corner_voice", "broll"}
assert len(layers) == 3
# z_index 排序
assert layers[0].z_index <= layers[1].z_index <= layers[2].z_index
def test_group_clips_intro_outro(self):
"""intro + 2 main + outro → 1 main_layer4 clips,按 order 排序)。"""
clips = [
_make_clip("intro", "intro", order=0),
_make_clip("c1", "main", order=1),
_make_clip("c2", "main", order=2),
_make_clip("outro", "outro", order=3),
]
svc = _make_service(clips)
with _patch_path_exists(), patch("video_processing.unified_render_service.probe_duration", return_value=5.0):
resolved = svc._resolve_clips()
layers = svc._group_clips_into_layers(resolved)
assert len(layers) == 1
assert layers[0].role == "main"
assert len(layers[0].clips) == 4
# 按 order 排序
orders = [c.order for c in layers[0].clips]
assert orders == [0, 1, 2, 3]
# ── 测试 _resolve_clips ──────────────────────────────────────────────────────
class TestResolveClips:
"""测试 _resolve_clips 方法。"""
def test_skip_missing_asset(self):
"""跳过 asset_id 在 asset_path_map 中找不到的 clip。"""
clips = [
_make_clip("c1", "main", order=0, asset_id="asset_1.mp4"),
_make_clip("c2", "main", order=1, asset_id="missing.mp4"),
]
# 只有 asset_1.mp4 存在
asset_paths = {"asset_1.mp4": Path("/tmp/asset_1.mp4")}
svc = _make_service(clips, asset_paths)
with _patch_path_exists(), patch("video_processing.unified_render_service.probe_duration", return_value=5.0):
resolved = svc._resolve_clips()
assert len(resolved) == 1
assert resolved[0].clip_id == "c1"
def test_skip_empty_asset_id(self):
"""跳过 asset_id 为空的 clip。"""
clips = [
_make_clip("c1", "main", order=0, asset_id=""),
_make_clip("c2", "main", order=1, asset_id="asset_2.mp4"),
]
asset_paths = {"asset_2.mp4": Path("/tmp/asset_2.mp4")}
svc = _make_service(clips, asset_paths)
with _patch_path_exists(), patch("video_processing.unified_render_service.probe_duration", return_value=5.0):
resolved = svc._resolve_clips()
assert len(resolved) == 1
assert resolved[0].clip_id == "c2"
def test_sort_by_order(self):
"""解析后的 clips 按 order 排序。"""
clips = [
_make_clip("c3", "main", order=3, asset_id="a3.mp4"),
_make_clip("c1", "main", order=1, asset_id="a1.mp4"),
_make_clip("c2", "main", order=2, asset_id="a2.mp4"),
]
asset_paths = {
"a1.mp4": Path("/tmp/a1.mp4"),
"a2.mp4": Path("/tmp/a2.mp4"),
"a3.mp4": Path("/tmp/a3.mp4"),
}
svc = _make_service(clips, asset_paths)
with _patch_path_exists(), patch("video_processing.unified_render_service.probe_duration", return_value=5.0):
resolved = svc._resolve_clips()
orders = [c.order for c in resolved]
assert orders == [1, 2, 3]
# ── 测试 _build_filter_complex ───────────────────────────────────────────────
class TestBuildFilterComplex:
"""测试 _build_filter_complex 方法。"""
def test_single_layer_single_clip(self):
"""只有 1 个 main clip → 简单 scale + setpts。"""
clips = [_make_clip("c1", "main", order=0)]
asset_paths = {"asset_c1.mp4": Path("/tmp/asset_c1.mp4")}
svc = _make_service(clips, asset_paths)
with _patch_path_exists(), patch("video_processing.unified_render_service.probe_duration", return_value=5.0):
resolved = svc._resolve_clips()
layers = svc._group_clips_into_layers(resolved)
fc, input_args = svc._build_filter_complex(layers)
assert "-i" in input_args
assert "/tmp/asset_c1.mp4" in input_args
assert "scale=" in fc
assert "[final_video]" in fc
def test_single_layer_multi_clips(self):
"""多个 main clips → xfade 串联。"""
clips = [
_make_clip("c1", "main", order=0, duration=3.0),
_make_clip("c2", "main", order=1, duration=3.0),
]
asset_paths = {
"asset_c1.mp4": Path("/tmp/asset_c1.mp4"),
"asset_c2.mp4": Path("/tmp/asset_c2.mp4"),
}
svc = _make_service(clips, asset_paths)
with _patch_path_exists(), patch("video_processing.unified_render_service.probe_duration", return_value=5.0):
resolved = svc._resolve_clips()
layers = svc._group_clips_into_layers(resolved)
fc, input_args = svc._build_filter_complex(layers)
assert input_args.count("-i") == 2
assert "xfade=" in fc
assert "[final_video]" in fc
def test_with_overlay(self):
"""main + overlay → overlay 滤镜。"""
clips = [
_make_clip("c1", "main", order=0),
_make_clip("c2", "overlay", order=1),
]
asset_paths = {
"asset_c1.mp4": Path("/tmp/asset_c1.mp4"),
"asset_c2.mp4": Path("/tmp/asset_c2.mp4"),
}
svc = _make_service(clips, asset_paths)
with _patch_path_exists(), patch("video_processing.unified_render_service.probe_duration", return_value=5.0):
resolved = svc._resolve_clips()
layers = svc._group_clips_into_layers(resolved)
fc, input_args = svc._build_filter_complex(layers)
assert "overlay=" in fc
assert "[final_video]" in fc
def test_empty_layers_raises(self):
"""空图层列表抛出 ValueError。"""
svc = _make_service()
with pytest.raises(ValueError, match="没有可渲染的图层"):
svc._build_filter_complex([])
# ── 测试 render 方法 ─────────────────────────────────────────────────────────
class TestRender:
"""测试 render 方法。"""
def test_render_empty_clips_raises(self):
"""没有 clips 时抛出 ValueError。"""
svc = _make_service(clips=[], asset_paths={})
with pytest.raises(ValueError, match="没有可渲染的片段"):
svc.render()
def test_render_with_missing_assets_raises(self):
"""所有 clips 素材缺失时抛出 ValueError。"""
clips = [_make_clip("c1", "main", order=0, asset_id="missing.mp4")]
svc = _make_service(clips, asset_paths={})
with pytest.raises(ValueError, match="没有可渲染的片段"):
svc.render()
def test_render_success(self):
"""正常渲染流程。"""
clips = [_make_clip("c1", "main", order=0)]
asset_paths = {"asset_c1.mp4": Path("/tmp/asset_c1.mp4")}
svc = _make_service(clips, asset_paths)
with (
_patch_path_exists(),
patch("video_processing.unified_render_service.probe_duration", return_value=5.0),
patch.object(svc, "_execute_ffmpeg") as mock_exec,
patch.object(svc, "_probe_output", return_value=(5.0, 1024, 1280, 720)),
):
result = svc.render()
assert isinstance(result, RenderResult)
assert result.duration == 5.0
assert result.file_size == 1024
assert result.width == 1280
assert result.height == 720
mock_exec.assert_called_once()
+204
View File
@@ -0,0 +1,204 @@
"""UUID 字段长度单元测试 — 验证标准 UUID(带横杠,36字符)可正常插入.
覆盖:
- 所有从 varchar(32) 扩到 varchar(36) 的表
- 插入标准 UUID 格式(带横杠)不报 StringDataRightTruncation
- 字段长度从 32 扩到 36 后 ORM 模型定义正确
"""
from __future__ import annotations
import uuid
import pytest
from packages.adapters.sqlalchemy_impl.models import (
Base,
ClassificationJobModel,
EditPlanClipModel,
EditPlanModel,
EditTemplateModel,
GeneratedVideoModel,
GenerationTaskModel,
IngestJobModel,
JobModel,
ProjectModel,
TemplateClipConfigModel,
)
def _uuid() -> str:
"""生成标准 UUID 字符串(带横杠,36 字符)."""
return str(uuid.uuid4())
class TestUUIDFieldLengthModels:
"""验证 ORM 模型中 UUID 字段定义为 String(36)."""
def _get_column_type_length(self, model_cls, column_name: str) -> int:
"""获取模型列的 String 长度."""
col = model_cls.__table__.columns[column_name]
return col.type.length
# ── ProjectModel ──────────────────────────────────────────────────────────
def test_project_id_is_36(self):
assert self._get_column_type_length(ProjectModel, "id") == 36
def test_project_owner_user_id_is_36(self):
assert self._get_column_type_length(ProjectModel, "owner_user_id") == 36
# ── EditTemplateModel ─────────────────────────────────────────────────────
def test_edit_template_id_is_36(self):
assert self._get_column_type_length(EditTemplateModel, "id") == 36
# ── EditPlanModel ─────────────────────────────────────────────────────────
@pytest.mark.parametrize(
"col",
["id", "template_id", "source_edit_plan_id", "project_id", "created_by_user_id"],
)
def test_edit_plan_uuid_fields_are_36(self, col):
assert self._get_column_type_length(EditPlanModel, col) == 36
# ── TemplateClipConfigModel ───────────────────────────────────────────────
@pytest.mark.parametrize("col", ["id", "template_id"])
def test_template_clip_config_uuid_fields_are_36(self, col):
assert self._get_column_type_length(TemplateClipConfigModel, col) == 36
# ── EditPlanClipModel ─────────────────────────────────────────────────────
@pytest.mark.parametrize(
"col",
["id", "plan_id", "template_clip_config_id", "asset_id"],
)
def test_edit_plan_clip_uuid_fields_are_36(self, col):
assert self._get_column_type_length(EditPlanClipModel, col) == 36
# ── IngestJobModelP0 bug 所在表)─────────────────────────────────────────
@pytest.mark.parametrize(
"col",
["id", "project_id", "library_id", "result_asset_id"],
)
def test_ingest_job_uuid_fields_are_36(self, col):
assert self._get_column_type_length(IngestJobModel, col) == 36
def test_ingest_job_library_id_accepts_standard_uuid(self):
"""P0 回归:library_id 必须能容纳标准 UUID(36 字符带横杠)."""
standard_uuid = "550e8400-e29b-41d4-a716-446655440000"
assert len(standard_uuid) == 36
col = IngestJobModel.__table__.columns["library_id"]
assert col.type.length >= 36
# ── ClassificationJobModel ────────────────────────────────────────────────
@pytest.mark.parametrize("col", ["id", "project_id", "asset_id"])
def test_classification_job_uuid_fields_are_36(self, col):
assert self._get_column_type_length(ClassificationJobModel, col) == 36
# ── GenerationTaskModel ───────────────────────────────────────────────────
@pytest.mark.parametrize(
"col",
[
"id",
"project_id",
"strategy_id",
"asset_library_id",
"voice_library_id",
"created_by_user_id",
"source_edit_plan_id",
"batch_id",
],
)
def test_generation_task_uuid_fields_are_36(self, col):
assert self._get_column_type_length(GenerationTaskModel, col) == 36
# ── GeneratedVideoModel ───────────────────────────────────────────────────
@pytest.mark.parametrize(
"col",
["id", "project_id", "generation_task_id", "duplicate_of"],
)
def test_generated_video_uuid_fields_are_36(self, col):
assert self._get_column_type_length(GeneratedVideoModel, col) == 36
# ── JobModel ──────────────────────────────────────────────────────────────
@pytest.mark.parametrize(
"col",
["id", "project_id", "source_id", "created_by_user_id"],
)
def test_job_uuid_fields_are_36(self, col):
assert self._get_column_type_length(JobModel, col) == 36
class TestUUIDInsertWithHyphens:
"""验证标准 UUID(带横杠)可构造 ORM 对象,字段长度足够."""
def test_ingest_job_model_construct_with_standard_uuid(self):
"""P0 回归测试:用标准 UUID 构造 IngestJobModel 不报错."""
std_uuid = _uuid()
assert len(std_uuid) == 36
job = IngestJobModel(
id=_uuid(),
project_id=_uuid(),
library_id=std_uuid, # ← 关键:标准 UUID 带横杠
storage_key="test/file.mp4",
result_asset_id=_uuid(),
)
assert job.library_id == std_uuid
assert len(job.library_id) == 36
def test_project_model_construct_with_standard_uuid(self):
project = ProjectModel(
id=_uuid(),
owner_user_id=_uuid(),
name="test",
)
assert len(project.id) == 36
assert len(project.owner_user_id) == 36
def test_generation_task_model_construct_with_standard_uuid(self):
task = GenerationTaskModel(
id=_uuid(),
project_id=_uuid(),
asset_library_id=_uuid(),
created_by_user_id=_uuid(),
source_edit_plan_id=_uuid(),
batch_id=_uuid(),
)
assert len(task.id) == 36
assert len(task.asset_library_id) == 36
def test_edit_plan_model_construct_with_standard_uuid(self):
plan = EditPlanModel(
id=_uuid(),
template_id=_uuid(),
source_edit_plan_id=_uuid(),
project_id=_uuid(),
created_by_user_id=_uuid(),
)
assert len(plan.id) == 36
assert len(plan.template_id) == 36
def test_generated_video_model_construct_with_standard_uuid(self):
video = GeneratedVideoModel(
id=_uuid(),
project_id=_uuid(),
generation_task_id=_uuid(),
duplicate_of=_uuid(),
name="test.mp4",
file_url="https://example.com/test.mp4",
file_size=1000,
duration=10.0,
width=1280,
height=720,
fps=25.0,
)
assert len(video.id) == 36
assert len(video.duplicate_of) == 36