Compare commits

..

16 Commits

Author SHA1 Message Date
xiaoxia f10fd9cd5c feat: 查重率百分比计算+跨项目查重 #1660 (#1675)
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 1s
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 4s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 3s
CI/CD Pipeline / Check push changed paths (push) Successful in 5s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Validate - Style (pull_request) Has been skipped
CI/CD Pipeline / Validate - Security (pull_request) Has been skipped
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Has been skipped
CI/CD Pipeline / Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 8s
CI/CD Pipeline / Build Staging API Image (push) Successful in 33s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 35s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 40s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 2m11s
CI/CD Pipeline / Validate - Style (push) Successful in 2m34s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 2m1s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 2m32s
CI/CD Pipeline / CI Gate (pull_request) Successful in 4s
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Integration Tests (push) Successful in 2m41s
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m28s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m8s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 1m5s
AI Code Review / AI Code Review (pull_request) Successful in 6m28s
CI/CD Pipeline / Validate - Security (push) Successful in 7m1s
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m40s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 7m40s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 3m22s
CI/CD Pipeline / Staging E2E Tests (push) Successful in 5m29s
CI/CD Pipeline / Unit Tests (push) Successful in 13m51s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
Co-authored-by: xiaoxia <dev@xiaoxiajianji.com>
Co-committed-by: xiaoxia <dev@xiaoxiajianji.com>
2026-09-04 00:11:51 +08:00
xiaoxia 1ff81dcd0a Merge pull request 'feat(dedup): 动态抽帧 + 滑动窗口时序匹配 (#1659)' (#1673) from feature/1659-dynamic-keyframe-sliding-window into develop
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 2s
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Check push changed paths (push) Successful in 3s
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
AI Code Review / AI Code Review (pull_request) Failing after 1m48s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / Validate - Style (push) Successful in 1m50s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (push) Successful in 17s
CI/CD Pipeline / Integration Tests (push) Successful in 59s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m14s
CI/CD Pipeline / Validate - Style (pull_request) Has been skipped
CI/CD Pipeline / Validate - Security (pull_request) Has been skipped
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Has been skipped
CI/CD Pipeline / Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (push) Successful in 12s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 18s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 23s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 19s
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 22s
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m52s
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 3s
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 3m38s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 4m35s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 4m13s
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m25s
CI/CD Pipeline / Unit Tests (push) Successful in 9m17s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 3m0s
CI/CD Pipeline / Staging E2E Tests (push) Successful in 5m3s
CI/CD Pipeline / Validate - Security (push) Successful in 23m51s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
2026-09-03 23:16:50 +08:00
xiaoxia db9ee89ffa fix(dedup): ruff lint 修复 — 未使用变量 + zip strict + 冗余 import (#1659)
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 3s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 4s
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Validate - Style (pull_request) Successful in 1m20s
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 10s
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m26s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m0s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 12s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
AI Code Review / AI Code Review (pull_request) Failing after 2m34s
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Unit Tests (pull_request) Successful in 1m17s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m48s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 4m4s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 4m26s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 5s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 1m17s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 22m37s
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 4s
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
2026-09-03 23:09:19 +08:00
CI Bot a0d4f6e111 style: auto-format with black + isort + prettier [skip ci-format-check]
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 22s
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 10s
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Unit Tests (pull_request) Successful in 1m20s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m27s
CI/CD Pipeline / Validate - Style (pull_request) Failing after 1m33s
AI Code Review / AI Code Review (pull_request) Failing after 2m11s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m19s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m43s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 4m35s
CI/CD Pipeline / Validate - Security (pull_request) Has been cancelled
CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / CI Gate (pull_request) Has been cancelled
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 2m29s
2026-09-03 15:04:14 +00:00
xiaoxia fac80b1f77 feat(dedup): 动态抽帧 + 滑动窗口时序匹配 (#1659)
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 1s
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2s
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 22s
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 9s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Unit Tests (pull_request) Successful in 1m22s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m25s
AI Code Review / AI Code Review (pull_request) Failing after 1m50s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m39s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m44s
CI/CD Pipeline / Validate - Style (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Security (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Has been cancelled
CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / CI Gate (pull_request) Has been cancelled
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been cancelled
1. 动态抽帧策略 — detect_keyframe_timestamps()
   - 降采样到 320x240 逐帧灰度差异检测场景切换
   - 最小间隔过滤(保留差异最大的候选帧)
   - 数量裁剪到 [MIN_KEYFRAMES=5, MAX_KEYFRAMES=30]
   - 长视频(>3分钟)每 30 秒分段保底

2. 滑动窗口时序匹配 — find_duplicate_segments()
   - 逐帧最佳匹配 → 连续 run 检测(允许 MAX_GAP=2 间隙)
   - 最少 MIN_CONSECUTIVE_MATCHES=5 帧才报告
   - 返回 DuplicateSegment(query/target 时间范围 + 平均距离)

3. 查重算法升级
   - 均值距离 → 中位数距离(抵抗异常值)
   - 新增帧匹配比例条件(match_ratio >= 0.7)
   - Bhattacharyya 系数替代余弦相似度
   - pHash + 直方图加权融合(0.7/0.3)
   - 判定重复后附加 duplicate_segments 字段

4. 删除旧代码
   - 移除 SHORT_VIDEO_CHUNK_SEC/LONG_VIDEO_CHUNK_SEC 固定间隔
   - 移除 compute_chunk_interval()
   - 移除 _average_histogram_similarity()

5. 测试
   - 新增 test_dedup_v2.py: 34 个测试
   - 更新 test_dedup_engine.py/test_duplicate_rate.py/test_dedup_pure.py
   - 清理 test_fingerprint_chunks.py 中旧常量测试
2026-09-03 23:00:21 +08:00
xiaoxia 159a62f9a5 Merge pull request 'feat: 跨视频片段避让 — 生成前注入已用区间 #1670' (#1671) from feat/cross-video-avoidance-1670 into develop
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 4s
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 4s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 3s
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Check push changed paths (push) Successful in 6s
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / Validate - Style (pull_request) Has been skipped
CI/CD Pipeline / Validate - Security (pull_request) Has been skipped
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Has been skipped
CI/CD Pipeline / Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 17s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 8s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 7s
CI/CD Pipeline / Integration Tests (push) Successful in 1m39s
CI/CD Pipeline / Build Staging API Image (push) Successful in 22s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 16s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 18s
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 2s
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m57s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been skipped
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 3m28s
CI/CD Pipeline / Validate - Style (push) Failing after 4m10s
AI Code Review / AI Code Review (pull_request) Failing after 4m7s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 4m28s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 2m15s
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 5m43s
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m31s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 2m56s
CI/CD Pipeline / Unit Tests (push) Successful in 8m3s
CI/CD Pipeline / Staging E2E Tests (push) Successful in 6m25s
CI/CD Pipeline / Validate - Security (push) Successful in 29m7s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
2026-09-03 22:42:03 +08:00
xiaoxia 109d7afbc7 fix: AI配音标识被overflow:hidden裁剪不显示 (#1672)
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 2s
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 4s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 4s
CI/CD Pipeline / Check push changed paths (push) Successful in 6s
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Validate - Style (pull_request) Has been skipped
CI/CD Pipeline / Validate - Security (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 17s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 18s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 18s
CI/CD Pipeline / Build Staging API Image (push) Successful in 17s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 17s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 24s
CI/CD Pipeline / Integration Tests (push) Successful in 1m55s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Validate - Style (push) Has been cancelled
CI/CD Pipeline / Validate - Security (push) Has been cancelled
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Has been cancelled
CI/CD Pipeline / Unit Tests (push) Has been cancelled
CI/CD Pipeline / Frontend Unit Tests (push) Has been cancelled
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been cancelled
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been cancelled
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been cancelled
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been cancelled
CI/CD Pipeline / Staging E2E Tests (push) Has been cancelled
CI/CD Pipeline / Staging API Integration Tests (push) Has been cancelled
CI/CD Pipeline / Build Production API Image (push) Has been cancelled
CI/CD Pipeline / Build Production Web Image (push) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (push) Has been cancelled
CI/CD Pipeline / Deploy Production (push) Has been cancelled
CI/CD Pipeline / Production Browser E2E (push) Has been cancelled
CI/CD Pipeline / ACR Image Cleanup (push) Has been cancelled
CI/CD Pipeline / Canary Release to Production (push) Has been cancelled
CI/CD Pipeline / CI Gate (push) Has been cancelled
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been cancelled
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been cancelled
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been cancelled
CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been cancelled
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / CI Gate (pull_request) Has been cancelled
AI Code Review / AI Code Review (pull_request) Has been cancelled
PR Automation / Auto Approve on CI Green (pull_request) Has been cancelled
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been cancelled
Preview Deploy / Deploy Preview Environment (pull_request) Has been cancelled
2026-09-03 22:40:11 +08:00
CI Bot 9d31818222 style: auto-format with black + isort + prettier [skip ci-format-check]
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 3s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 3s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 21s
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 21s
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m42s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 1m46s
CI/CD Pipeline / Validate - Style (pull_request) Failing after 1m48s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m57s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 3m20s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 5m2s
AI Code Review / AI Code Review (pull_request) Successful in 6m42s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 10m48s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 16s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 2m16s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 30m18s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Failing after 3s
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
2026-09-03 14:21:24 +00:00
saas-backend-agent af4dd31dd1 feat: 跨视频片段避让 — 生成前注入已用区间 #1670
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 1s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 31s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 32s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 1m38s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m40s
AI Code Review / AI Code Review (pull_request) Successful in 1m53s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m52s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m54s
CI/CD Pipeline / Validate - Style (pull_request) Failing after 2m59s
CI/CD Pipeline / Validate - Security (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Has been cancelled
CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / CI Gate (pull_request) Has been cancelled
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been cancelled
- Repository: list_used_segments_by_user() JOIN edit_plans 查用户最近
  已完成 plan 的已渲染 clips,聚合为 {asset_id: [(start, end), ...]}
- Domain: distribute_assets / _distribute_* 子函数新增 external_used_segments
  参数,深拷贝注入 used_segments,让 _resolve_start_time 自动避让
- Service: _distribute_assets 新增 user_id 参数,预览和正式生成都查询
  已用区间;查询失败时不阻塞,回退纯随机
- 12 个单元测试覆盖 Repository/Domain/Service 三层
2026-09-03 22:18:06 +08:00
xiaoxia 8ecf381a9d Merge pull request 'feat: 分片指纹存储改造 + 存量指纹重建脚本 #1657' (#1669) from feat/fingerprint-chunks-1657 into develop
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 5s
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 4s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 5s
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Check push changed paths (push) Successful in 6s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / Validate - Style (pull_request) Has been skipped
CI/CD Pipeline / Validate - Security (pull_request) Has been skipped
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Has been skipped
CI/CD Pipeline / Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Integration Tests (push) Successful in 1m27s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 20s
CI/CD Pipeline / Validate - Style (push) Successful in 2m7s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 12s
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 14s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m16s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (push) Successful in 20s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (push) Successful in 22s
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 3s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 20s
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m50s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been skipped
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 4m41s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 3m54s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 2m11s
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m24s
AI Code Review / AI Code Review (pull_request) Successful in 6m37s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 2m56s
CI/CD Pipeline / Unit Tests (push) Successful in 7m50s
CI/CD Pipeline / Staging E2E Tests (push) Successful in 4m56s
CI/CD Pipeline / Validate - Security (push) Successful in 28m7s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
2026-09-03 21:08:37 +08:00
CI Bot 244691d335 style: auto-format with black + isort + prettier [skip ci-format-check]
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2s
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
AI Code Review / AI Code Review (pull_request) Failing after 1m46s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m47s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m54s
CI/CD Pipeline / Validate - Style (pull_request) Failing after 2m29s
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Integration Tests (pull_request) Successful in 2m6s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 24s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 22s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 4m45s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 8m59s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 10m57s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 7s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 1m37s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 28m20s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Failing after 1s
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
2026-09-03 12:50:38 +00:00
xiaoxia ee4fff42f0 fix(voices): AI配音标识兼容旧素材,增加 tts_job_id 降级判断 (#1666)
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 1s
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Check push changed paths (push) Successful in 3s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 3s
CI/CD Pipeline / Validate - Style (push) Successful in 1m35s
AI Code Review / AI Code Review (pull_request) Failing after 2m5s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m48s
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Validate - Style (pull_request) Has been skipped
CI/CD Pipeline / Validate - Security (pull_request) Has been skipped
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m55s
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Integration Tests (push) Successful in 1m15s
CI/CD Pipeline / Build Staging API Image (push) Successful in 24s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 23s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 19s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 20s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 36s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m0s
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 5m14s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 4s
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 6m14s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 2m17s
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m29s
CI/CD Pipeline / Unit Tests (push) Successful in 9m22s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 3m16s
CI/CD Pipeline / Staging E2E Tests (push) Successful in 4m4s
CI/CD Pipeline / Validate - Security (push) Has been cancelled
CI/CD Pipeline / Build Production API Image (push) Has been cancelled
CI/CD Pipeline / Build Production Web Image (push) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (push) Has been cancelled
CI/CD Pipeline / Deploy Production (push) Has been cancelled
CI/CD Pipeline / Production Browser E2E (push) Has been cancelled
CI/CD Pipeline / Canary Release to Production (push) Has been cancelled
CI/CD Pipeline / CI Gate (push) Has been cancelled
2026-09-03 20:49:31 +08:00
xiaoxia b0018e747b fix: 提取视频配音 API 路径修正为 /voices/extract-voice (#1665)
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Has been cancelled
CI/CD Pipeline / Check if frontend-only change (push) Has been cancelled
CI/CD Pipeline / Validate - Style (push) Has been cancelled
CI/CD Pipeline / Validate - Security (push) Has been cancelled
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Has been cancelled
CI/CD Pipeline / Unit Tests (push) Has been cancelled
CI/CD Pipeline / Integration Tests (push) Has been cancelled
CI/CD Pipeline / Frontend Lint (push) Has been cancelled
CI/CD Pipeline / Frontend Unit Tests (push) Has been cancelled
CI/CD Pipeline / PR Build API Image (push) Has been cancelled
CI/CD Pipeline / PR Build Web Image (push) Has been cancelled
CI/CD Pipeline / PR Build Worker Image (push) Has been cancelled
CI/CD Pipeline / Check push changed paths (push) Has been cancelled
CI/CD Pipeline / Build Staging API Image (push) Has been cancelled
CI/CD Pipeline / Build Staging Web Image (push) Has been cancelled
CI/CD Pipeline / Build Staging Worker Image (push) Has been cancelled
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been cancelled
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been cancelled
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been cancelled
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been cancelled
CI/CD Pipeline / Staging E2E Tests (push) Has been cancelled
CI/CD Pipeline / Staging API Integration Tests (push) Has been cancelled
CI/CD Pipeline / Build Production API Image (push) Has been cancelled
CI/CD Pipeline / Build Production Web Image (push) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (push) Has been cancelled
CI/CD Pipeline / Deploy Production (push) Has been cancelled
CI/CD Pipeline / Production Browser E2E (push) Has been cancelled
CI/CD Pipeline / ACR Image Cleanup (push) Has been cancelled
CI/CD Pipeline / Canary Release to Production (push) Has been cancelled
CI/CD Pipeline / CI Gate (push) Has been cancelled
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Has been cancelled
CI/CD Pipeline / Check if frontend-only change (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Style (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Security (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Has been cancelled
CI/CD Pipeline / Unit Tests (pull_request) Has been cancelled
CI/CD Pipeline / Integration Tests (pull_request) Has been cancelled
CI/CD Pipeline / Frontend Lint (pull_request) Has been cancelled
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been cancelled
CI/CD Pipeline / PR Build API Image (pull_request) Has been cancelled
CI/CD Pipeline / PR Build Web Image (pull_request) Has been cancelled
CI/CD Pipeline / PR Build Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Check push changed paths (pull_request) Has been cancelled
CI/CD Pipeline / Build Staging API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been cancelled
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been cancelled
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been cancelled
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been cancelled
CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been cancelled
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / CI Gate (pull_request) Has been cancelled
AI Code Review / AI Code Review (pull_request) Has been cancelled
PR Automation / Auto Approve on CI Green (pull_request) Has been cancelled
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been cancelled
Preview Deploy / Deploy Preview Environment (pull_request) Has been cancelled
2026-09-03 20:49:15 +08:00
saas-backend-agent cbca0c3584 feat: 分片指纹存储改造 + 存量指纹重建脚本 #1657
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 3s
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m19s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m43s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m22s
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 8s
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 7s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Validate - Style (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Security (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Has been cancelled
CI/CD Pipeline / Unit Tests (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been cancelled
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been cancelled
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been cancelled
CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been cancelled
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / CI Gate (pull_request) Has been cancelled
AI Code Review / AI Code Review (pull_request) Has been cancelled
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been cancelled
## 改动

### 1. 新建 video_fingerprint_chunks 表(Migration 063)
- 按时间分片存储 pHash + color_histogram
- 索引:video_id, project_id, user_id

### 2. 新增 VideoFingerprintChunkModel
- packages/adapters/sqlalchemy_impl/models.py

### 3. 改造 dedup.py 指纹计算
- compute_fingerprint() 改为按时间分片抽帧
  - 短视频(≤60s):每 2s 一片
  - 长视频(>60s):每 5s 一片
- VideoFingerprint 新增 chunks 字段(list of FingerprintChunk)
- 向后兼容:keyframe_phashes/color_histograms 保留
- to_chunk_models() 方法转换为 SQLAlchemy Model
- check_duplicate() 优先从分片表读取,回退到 JSON 字段
- check_duplicate_task() 写入分片表

### 4. 改造 dedup_helpers.py
- create_video_record_and_dedup() 同步写入分片表

### 5. 存量指纹重建脚本
- apps/api/scripts/rebuild_fingerprint_chunks.py
- 支持 --dry-run 和 --batch-size
- 幂等:已有分片数据的视频跳过

### 6. 单元测试(11 个)
- 分片策略:60s→30片,120s→24片
- to_chunk_models() 输出正确
- _save_fingerprint_chunks 幂等性
- to_dict() 向后兼容

Closes #1657
2026-09-03 20:41:11 +08:00
xiaoxia c7c30936a9 fix: 修复 voices.py 中 3 处 ruff B904 错误,解除 CI Validate-Style 阻塞 (#1667)
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 3s
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 5s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 5s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Check push changed paths (push) Successful in 5s
CI/CD Pipeline / Validate - Style (pull_request) Has been skipped
CI/CD Pipeline / Validate - Security (pull_request) Has been skipped
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Has been skipped
CI/CD Pipeline / Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 15s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 11s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 2m2s
CI/CD Pipeline / Build Staging API Image (push) Successful in 31s
CI/CD Pipeline / Integration Tests (push) Successful in 2m26s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 4s
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 27s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 30s
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Validate - Style (push) Successful in 3m11s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m8s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been skipped
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 3m51s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 5m7s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 2m17s
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 5m21s
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m26s
AI Code Review / AI Code Review (pull_request) Successful in 6m55s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 2m52s
CI/CD Pipeline / Unit Tests (push) Successful in 10m16s
CI/CD Pipeline / Staging E2E Tests (push) Successful in 6m13s
CI/CD Pipeline / Validate - Security (push) Has been cancelled
CI/CD Pipeline / Build Production API Image (push) Has been cancelled
CI/CD Pipeline / Build Production Web Image (push) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (push) Has been cancelled
CI/CD Pipeline / Deploy Production (push) Has been cancelled
CI/CD Pipeline / Production Browser E2E (push) Has been cancelled
CI/CD Pipeline / Canary Release to Production (push) Has been cancelled
CI/CD Pipeline / CI Gate (push) Has been cancelled
2026-09-03 20:35:06 +08:00
xiaoxia aac8cc5fd7 Merge pull request 'feat: 正式生成时片段随机重排(降重,默认开启无开关)#1663' (#1668) from feat/segment-random-shuffle-1663 into develop
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 2s
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 3s
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 5s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / Check push changed paths (push) Successful in 5s
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Validate - Style (pull_request) Has been skipped
CI/CD Pipeline / Validate - Security (pull_request) Has been skipped
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Has been skipped
CI/CD Pipeline / Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 18s
CI/CD Pipeline / Integration Tests (push) Successful in 2m13s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 16s
CI/CD Pipeline / Build Staging API Image (push) Successful in 19s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 51s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 16s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 2s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 3m4s
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 14s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m9s
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been skipped
CI/CD Pipeline / Validate - Style (push) Failing after 3m54s
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 4m31s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 4m56s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 2m6s
AI Code Review / AI Code Review (pull_request) Successful in 6m38s
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m26s
CI/CD Pipeline / Unit Tests (push) Successful in 8m7s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 3m0s
CI/CD Pipeline / Staging E2E Tests (push) Successful in 3m45s
CI/CD Pipeline / Validate - Security (push) Has been cancelled
CI/CD Pipeline / Build Production API Image (push) Has been cancelled
CI/CD Pipeline / Build Production Web Image (push) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (push) Has been cancelled
CI/CD Pipeline / Deploy Production (push) Has been cancelled
CI/CD Pipeline / Production Browser E2E (push) Has been cancelled
CI/CD Pipeline / Canary Release to Production (push) Has been cancelled
CI/CD Pipeline / CI Gate (push) Has been cancelled
2026-09-03 20:17:05 +08:00
23 changed files with 2826 additions and 374 deletions
@@ -0,0 +1,46 @@
"""add video_fingerprint_chunks table for per-chunk fingerprint storage
Revision ID: 063_fingerprint_chunks
Revises: 062_edit_plan_id
Create Date: 2026-09-03
"""
import sqlalchemy as sa
from alembic import op
revision = "063_fingerprint_chunks"
down_revision = "062_edit_plan_id"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.create_table(
"video_fingerprint_chunks",
sa.Column("id", sa.String(36), primary_key=True),
sa.Column("video_id", sa.String(36), nullable=False),
sa.Column("project_id", sa.String(36), nullable=False),
sa.Column("user_id", sa.String(36), nullable=False, server_default=""),
sa.Column("start_time_ms", sa.Integer, nullable=False),
sa.Column("end_time_ms", sa.Integer, nullable=False),
sa.Column("phash_binary", sa.String(16), nullable=False),
sa.Column("color_histogram", sa.JSON, nullable=False),
sa.Column("frame_count", sa.Integer, nullable=False, server_default="1"),
sa.Column(
"created_at",
sa.DateTime,
nullable=False,
server_default=sa.func.now(),
),
)
op.create_index("ix_vfc_video_id", "video_fingerprint_chunks", ["video_id"])
op.create_index("ix_vfc_project_id", "video_fingerprint_chunks", ["project_id"])
op.create_index("ix_vfc_user_id", "video_fingerprint_chunks", ["user_id"])
def downgrade() -> None:
op.drop_index("ix_vfc_user_id", table_name="video_fingerprint_chunks")
op.drop_index("ix_vfc_project_id", table_name="video_fingerprint_chunks")
op.drop_index("ix_vfc_video_id", table_name="video_fingerprint_chunks")
op.drop_table("video_fingerprint_chunks")
@@ -0,0 +1,25 @@
"""add match_count and visual_similarity to generated_videos
Revision ID: 064_match_count_visual_sim
Revises: 063_fingerprint_chunks
Create Date: 2026-09-03
"""
import sqlalchemy as sa
from alembic import op
revision = "064_match_count_visual_sim"
down_revision = "063_fingerprint_chunks"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column("generated_videos", sa.Column("match_count", sa.Integer(), nullable=True, server_default="0"))
op.add_column("generated_videos", sa.Column("visual_similarity", sa.Float(), nullable=True, server_default="0.0"))
def downgrade() -> None:
op.drop_column("generated_videos", "visual_similarity")
op.drop_column("generated_videos", "match_count")
+3 -3
View File
@@ -687,13 +687,13 @@ def extract_voice_from_video(
raise HTTPException(
status_code=status.HTTP_504_GATEWAY_TIMEOUT,
detail="视频处理超时,请尝试较短的视频",
)
) from None
except Exception as e:
logger.exception("提取视频配音失败: %s", e)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="提取配音失败,请稍后重试",
)
) from e
finally:
# 清理临时文件
if tmp_dir and Path(tmp_dir).exists():
@@ -740,7 +740,7 @@ def _find_or_create_voice_library_for_extract(*, user_id, project_repository, as
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="配音素材库创建失败",
)
) from None
def _get_audio_duration(audio_path: Path) -> float:
@@ -131,6 +131,7 @@ class PlanGeneratorService:
editing_mode,
random_selection=random_preview,
asset_durations=asset_durations,
user_id=created_by_user_id,
)
# 5. 持久化所有 clips 并计算总时长
@@ -218,6 +219,7 @@ class PlanGeneratorService:
*,
random_selection: bool = False,
asset_durations: dict[str, float] | None = None,
user_id: str = "",
) -> None:
"""按 editing_mode 将素材分配到 clips(就地修改,未持久化).
@@ -239,6 +241,14 @@ class PlanGeneratorService:
asset_ids = list(asset_ids) # 复制避免修改调用方原列表
random.shuffle(asset_ids)
# 查询已有视频的已用区间(跨视频避让)
external_used_segments = None
if user_id and self._clip_repo:
try:
external_used_segments = self._clip_repo.list_used_segments_by_user(user_id, limit_recent=50)
except Exception:
logger.warning("跨视频避让查询失败,回退到纯随机", exc_info=True)
distribute_assets(
clips,
asset_ids,
@@ -246,6 +256,7 @@ class PlanGeneratorService:
random_selection=random_selection,
asset_durations=asset_durations,
asset_scene_points=asset_scene_points,
external_used_segments=external_used_segments,
)
def _fetch_asset_scene_points(self, asset_ids: List[str]) -> dict[str, list[float]]:
@@ -0,0 +1,174 @@
#!/usr/bin/env python3
"""存量指纹重建脚本 — 为已有视频生成 video_fingerprint_chunks 分片数据。
功能:
- 查询 generated_videos 中 video_fingerprint IS NOT NULL 但尚无分片数据的视频
- 从 OSS 下载视频 → 用新的分片算法重新计算指纹 → 写入分片表
- 支持 --dry-run(只打印不写入)和 --batch-size(默认 50
- 幂等:已存在分片数据的视频跳过
用法:
# 预览(不写入)
python rebuild_fingerprint_chunks.py --dry-run
# 执行重建
python rebuild_fingerprint_chunks.py --batch-size 50
"""
from __future__ import annotations
import argparse
import logging
import os
import sys
import tempfile
# 确保可以 import worker_app 和 packages
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "..", "..", "worker"))
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "..", "..", ".."))
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
)
logger = logging.getLogger("rebuild_fingerprint_chunks")
def find_videos_needing_rebuild(session, batch_size: int) -> list[dict]:
"""查询需要重建分片指纹的视频。"""
from sqlalchemy import and_
from packages.adapters.sqlalchemy_impl.models import GeneratedVideoModel, VideoFingerprintChunkModel
# 有 video_fingerprint 的视频
has_fingerprint = GeneratedVideoModel.video_fingerprint.isnot(None)
has_fingerprint = and_(has_fingerprint, GeneratedVideoModel.video_fingerprint != "")
# 排除已有分片数据的视频
subq = session.query(VideoFingerprintChunkModel.video_id).distinct().subquery()
no_chunks = ~GeneratedVideoModel.id.in_(subq)
videos = (
session.query(GeneratedVideoModel)
.filter(and_(has_fingerprint, no_chunks))
.order_by(GeneratedVideoModel.generated_at.desc())
.limit(batch_size)
.all()
)
return [
{
"id": v.id,
"project_id": v.project_id,
"user_id": v.user_id or "",
"duration": v.duration,
}
for v in videos
]
def rebuild_one(video_info: dict, dry_run: bool = False) -> int:
"""重建单个视频的分片数据。返回写入的 chunk 数量。"""
from video_processing.dedup import VideoDeduplicator, _save_fingerprint_chunks
from worker_app.db import SessionLocal
from packages.adapters.sqlalchemy_impl.models import VideoFingerprintChunkModel
from packages.shared.storage import get_storage_service
video_id = video_info["id"]
project_id = video_info["project_id"]
user_id = video_info["user_id"]
if dry_run:
logger.info("[DRY-RUN] Would rebuild video %s (project=%s)", video_id, project_id)
return 0
session = SessionLocal()
temp_dir = tempfile.mkdtemp()
try:
# 再次检查幂等性
existing_count = (
session.query(VideoFingerprintChunkModel).filter(VideoFingerprintChunkModel.video_id == video_id).count()
)
if existing_count > 0:
logger.info("Video %s already has %d chunks, skipping", video_id, existing_count)
return 0
# 下载视频
storage_service = get_storage_service()
local_path = os.path.join(temp_dir, f"{video_id}.mp4")
storage_key = f"projects/{project_id}/generated/{video_id}/{video_id}.mp4"
storage_service.download_file(storage_key, local_path)
# 重新计算指纹
deduplicator = VideoDeduplicator()
fingerprint = deduplicator.compute_fingerprint(local_path)
# 写入分片表
_save_fingerprint_chunks(fingerprint, video_id, project_id, user_id, session)
session.commit()
chunk_count = len(fingerprint.chunks)
logger.info("Rebuilt %d chunks for video %s", chunk_count, video_id)
return chunk_count
except Exception as e:
logger.error("Failed to rebuild video %s: %s", video_id, e)
session.rollback()
return -1
finally:
session.close()
import shutil
shutil.rmtree(temp_dir, ignore_errors=True)
def main():
parser = argparse.ArgumentParser(description="存量指纹重建脚本")
parser.add_argument("--dry-run", action="store_true", help="只打印不写入")
parser.add_argument("--batch-size", type=int, default=50, help="每批处理数量(默认 50")
parser.add_argument("--total-limit", type=int, default=0, help="总处理数量限制(0=不限制)")
args = parser.parse_args()
from worker_app.db import SessionLocal
session = SessionLocal()
try:
videos = find_videos_needing_rebuild(session, args.batch_size)
logger.info("Found %d videos needing rebuild", len(videos))
if args.dry_run:
for v in videos:
logger.info("[DRY-RUN] Video %s | project=%s | duration=%.1fs", v["id"], v["project_id"], v["duration"])
return
total_chunks = 0
processed = 0
failed = 0
for v in videos:
if args.total_limit > 0 and processed >= args.total_limit:
break
result = rebuild_one(v, dry_run=False)
if result < 0:
failed += 1
else:
total_chunks += result
processed += 1
logger.info(
"Rebuild complete: processed=%d, chunks=%d, failed=%d",
processed,
total_chunks,
failed,
)
finally:
session.close()
if __name__ == "__main__":
main()
+1 -1
View File
@@ -84,7 +84,7 @@ export const extractVideoVoice = async (
return new Promise((resolve, reject) => {
const xhr = new XMLHttpRequest()
xhr.open("POST", "/api/v1/tts/extract-video-voice")
xhr.open("POST", "/api/v1/voices/extract-voice")
// 携带认证 token(从 localStorage 获取,与 apiClient 拦截器一致)
const token = localStorage.getItem("access_token")
@@ -136,8 +136,9 @@ export const MaterialVoiceTab: React.FC<MaterialVoiceTabProps> = ({
const material = mapAssetToMaterial(asset)
// duration 优先取顶层(后端从 metadata 提取),兜底 metadata
const cardDuration = asset.duration || material.duration || 0
// AI 生成素材标识metadata.source === "tts_job"
const isAiMaterial = (asset.metadata as Record<string, unknown>)?.source === "tts_job"
// AI 生成素材标识:兼容旧素材(无 source 字段但有 tts_job_id
const meta = asset.metadata as Record<string, unknown>
const isAiMaterial = meta?.source === "tts_job" || !!meta?.tts_job_id
const isPlaying = playingId === asset.id
const isSelected = selectedIds.has(asset.id)
// 播放中以 audio 真实时长为准,未播放显示卡片时长
@@ -184,8 +185,10 @@ export const MaterialVoiceTab: React.FC<MaterialVoiceTabProps> = ({
</div>
<div className="xx-voice-info vmat-info">
<div className="xx-voice-name" title={asset.name}>
{asset.name}
<div className="xx-voice-name-row">
<div className="xx-voice-name" title={asset.name}>
{asset.name}
</div>
{isAiMaterial && <span className="vmat-ai-badge">AI</span>}
</div>
<div className="xx-voice-subtitle">
File diff suppressed because it is too large Load Diff
+31 -6
View File
@@ -100,8 +100,24 @@ def create_video_record_and_dedup(
generated_video.video_fingerprint = fingerprint.to_dict()
# (a) 历史成片查重
duplicate_result = deduplicator.check_duplicate(fingerprint, project_id, session)
# 写入分片指纹表
from video_processing.dedup import _save_fingerprint_chunks
try:
_save_fingerprint_chunks(fingerprint, video_id, project_id, user_id, session)
except Exception as chunk_err:
logger.warning("Failed to save fingerprint chunks for %s: %s", video_id, chunk_err)
# (a) 历史成片查重(跨项目全局 + 时长预过滤)
duration_sec = fingerprint.duration / 1000 if fingerprint.duration else 0
duplicate_result = deduplicator.check_duplicate(
fingerprint,
project_id,
session,
scope="user",
user_id=user_id,
duration_sec=duration_sec,
)
# (b) 批次内查重(仅当有 batch_id 时)
if not duplicate_result and batch_id:
@@ -121,17 +137,26 @@ def create_video_record_and_dedup(
generated_video.is_duplicate = False
generated_video.duplicate_of = None
# 计算重复率百分比(项目内所有已有视频对比取最高相似度
# 计算重复率百分比(项目全局
try:
dup_rate = deduplicator.compute_duplicate_rate(
rate_result = deduplicator.compute_duplicate_rate(
fingerprint,
project_id,
video_id,
session,
scope="user",
user_id=user_id,
)
generated_video.duplicate_rate = dup_rate
logger.info("Duplicate rate for %s: %.2f%%", video_id, dup_rate)
generated_video.duplicate_rate = rate_result["duplicate_rate"]
generated_video.match_count = rate_result["match_count"]
generated_video.visual_similarity = rate_result["visual_similarity"]
logger.info(
"Duplicate rate for %s: %.2f%% (visual_sim=%.3f, matches=%d)",
video_id,
rate_result["duplicate_rate"],
rate_result["visual_similarity"],
rate_result["match_count"],
)
except Exception as rate_err:
logger.warning("Failed to compute duplicate_rate for %s: %s", video_id, rate_err)
generated_video.duplicate_rate = None
@@ -131,3 +131,65 @@ class SQLAlchemyEditPlanClipRepository:
created_at=model.created_at,
updated_at=model.updated_at,
)
def list_used_segments_by_user(
self,
user_id: str,
*,
limit_recent: int = 50,
) -> dict[str, list[tuple[float, float]]]:
"""查询用户已有视频中已使用的素材区间(跨视频避让).
JOIN edit_plans 表,按 created_by_user_id 过滤,只查 status='completed'
的 plan 下 status='rendered' 且 asset_id 非空的 clips。按 plan 的
created_at DESC 取最近 limit_recent 个 plan。
Returns:
{asset_id: [(start_time, start_time + duration), ...]}
空结果返回空 dict。
"""
from packages.adapters.sqlalchemy_impl.models import EditPlanModel
if not user_id:
return {}
# 1. 查出最近 limit_recent 个已完成 plan 的 ID
recent_plan_ids = [
row[0]
for row in self.session.query(EditPlanModel.id)
.filter(
EditPlanModel.created_by_user_id == user_id,
EditPlanModel.status == "completed",
)
.order_by(EditPlanModel.created_at.desc())
.limit(limit_recent)
.all()
]
if not recent_plan_ids:
return {}
# 2. 查这些 plan 下已渲染、有素材的 clips
clips = (
self.session.query(
EditPlanClipModel.asset_id,
EditPlanClipModel.start_time,
EditPlanClipModel.duration,
)
.filter(
EditPlanClipModel.plan_id.in_(recent_plan_ids),
EditPlanClipModel.status == "rendered",
EditPlanClipModel.asset_id != "",
EditPlanClipModel.asset_id.isnot(None),
)
.all()
)
# 3. 聚合为 {asset_id: [(start, start+duration), ...]}
result: dict[str, list[tuple[float, float]]] = {}
for asset_id, start_time, duration in clips:
if asset_id not in result:
result[asset_id] = []
result[asset_id].append((start_time or 0.0, (start_time or 0.0) + (duration or 0.0)))
return result
@@ -31,6 +31,8 @@ class SQLAlchemyGeneratedVideoRepository:
is_duplicate=video.is_duplicate,
duplicate_of=video.duplicate_of,
duplicate_rate=video.duplicate_rate,
match_count=getattr(video, "match_count", 0),
visual_similarity=getattr(video, "visual_similarity", 0.0),
generated_at=video.generated_at,
created_at=video.created_at,
)
@@ -62,6 +64,8 @@ class SQLAlchemyGeneratedVideoRepository:
is_duplicate=getattr(model, "is_duplicate", False),
duplicate_of=getattr(model, "duplicate_of", None),
duplicate_rate=getattr(model, "duplicate_rate", None),
match_count=getattr(model, "match_count", 0) or 0,
visual_similarity=getattr(model, "visual_similarity", 0.0) or 0.0,
generated_at=model.generated_at,
created_at=model.created_at,
)
@@ -77,6 +81,8 @@ class SQLAlchemyGeneratedVideoRepository:
model.is_duplicate = video.is_duplicate
model.duplicate_of = video.duplicate_of
model.duplicate_rate = video.duplicate_rate
model.match_count = getattr(video, "match_count", 0)
model.visual_similarity = getattr(video, "visual_similarity", 0.0)
self.session.add(model)
self.session.commit()
return video
@@ -85,6 +91,24 @@ class SQLAlchemyGeneratedVideoRepository:
models = self.session.query(GeneratedVideoModel).filter(GeneratedVideoModel.project_id == project_id).all()
return [self._to_domain(model) for model in models]
def list_by_user(self, user_id: str, *, duration_min: float = 0, duration_max: float = 0) -> list[GeneratedVideo]:
"""按 user_id 查询用户所有项目的视频(跨项目查重)。
Args:
user_id: 用户 ID
duration_min: 时长下限(秒),0 表示不限
duration_max: 时长上限(秒),0 表示不限
"""
query = self.session.query(GeneratedVideoModel).filter(
GeneratedVideoModel.user_id == user_id,
)
if duration_min > 0:
query = query.filter(GeneratedVideoModel.duration >= duration_min)
if duration_max > 0:
query = query.filter(GeneratedVideoModel.duration <= duration_max)
models = query.all()
return [self._to_domain(model) for model in models]
def list_by_generation_task(self, generation_task_id: str) -> list[GeneratedVideo]:
models = (
self.session.query(GeneratedVideoModel)
@@ -208,6 +232,8 @@ class SQLAlchemyGeneratedVideoRepository:
is_duplicate=getattr(model, "is_duplicate", False),
duplicate_of=getattr(model, "duplicate_of", None),
duplicate_rate=getattr(model, "duplicate_rate", None),
match_count=getattr(model, "match_count", 0) or 0,
visual_similarity=getattr(model, "visual_similarity", 0.0) or 0.0,
generated_at=model.generated_at,
created_at=model.created_at,
)
@@ -340,6 +340,8 @@ class GeneratedVideoModel(Base):
is_duplicate = Column(Boolean, nullable=False, default=False)
duplicate_of = Column(String(36), nullable=True)
duplicate_rate = Column(Float, nullable=True)
match_count = Column(Integer, nullable=True, default=0)
visual_similarity = Column(Float, nullable=True, default=0.0)
class TitleLibraryModel(Base):
@@ -620,3 +622,20 @@ class CoverTemplateModel(Base):
config = Column(JSON, nullable=False, default=dict)
created_at = Column(DateTime(timezone=True), nullable=False, default=lambda: datetime.now(timezone.utc))
updated_at = Column(DateTime(timezone=True), nullable=False, default=lambda: datetime.now(timezone.utc))
class VideoFingerprintChunkModel(Base):
"""分片视频指纹 — 每个视频按时间分片存储 pHash + color_histogram."""
__tablename__ = "video_fingerprint_chunks"
id = Column(String(36), primary_key=True)
video_id = Column(String(36), nullable=False, index=True)
project_id = Column(String(36), nullable=False, index=True)
user_id = Column(String(36), nullable=False, index=True, default="")
start_time_ms = Column(Integer, nullable=False)
end_time_ms = Column(Integer, nullable=False)
phash_binary = Column(String(16), nullable=False)
color_histogram = Column(JSON, nullable=False)
frame_count = Column(Integer, nullable=False, default=1)
created_at = Column(DateTime, nullable=False, default=lambda: datetime.now(timezone.utc))
+2
View File
@@ -27,6 +27,8 @@ class GeneratedVideo:
is_duplicate: bool = False
duplicate_of: str | None = None
duplicate_rate: float | None = None
match_count: int = 0
visual_similarity: float = 0.0
generated_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
created_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
+23 -9
View File
@@ -169,6 +169,7 @@ def distribute_assets(
random_selection: bool = False,
asset_durations: dict[str, float] | None = None,
asset_scene_points: dict[str, list[float]] | None = None,
external_used_segments: dict[str, list[tuple[float, float]]] | None = None,
) -> None:
"""按 editing_mode 将素材分配到 clips(就地修改).
@@ -188,6 +189,7 @@ def distribute_assets(
random_selection: 是否随机选择素材(用于预览生成)
asset_durations: 素材 ID -> 时长(秒)映射,用于设置 start_time
asset_scene_points: 素材 ID -> 场景切换点列表(metadata 缓存)
external_used_segments: 跨视频已用区间(来自其他视频的 clips),注入到分配逻辑中避让
"""
if not asset_ids or not clips:
return
@@ -198,16 +200,16 @@ def distribute_assets(
random.shuffle(asset_ids)
if editing_mode == EditingMode.ONE_TAKE.value:
_distribute_one_take(clips, asset_ids, asset_durations, asset_scene_points)
_distribute_one_take(clips, asset_ids, asset_durations, asset_scene_points, external_used_segments)
elif editing_mode == EditingMode.PIP.value:
_distribute_pip(clips, asset_ids, asset_durations, asset_scene_points)
_distribute_pip(clips, asset_ids, asset_durations, asset_scene_points, external_used_segments)
elif editing_mode == EditingMode.VOICE_OVER.value:
_distribute_voice_over(clips, asset_ids, asset_durations, asset_scene_points)
_distribute_voice_over(clips, asset_ids, asset_durations, asset_scene_points, external_used_segments)
elif editing_mode == EditingMode.VOICE_PIP.value:
_distribute_voice_pip(clips, asset_ids, asset_durations, asset_scene_points)
_distribute_voice_pip(clips, asset_ids, asset_durations, asset_scene_points, external_used_segments)
else:
# 未知模式,退化为 one_take
_distribute_one_take(clips, asset_ids, asset_durations, asset_scene_points)
_distribute_one_take(clips, asset_ids, asset_durations, asset_scene_points, external_used_segments)
def _resolve_start_time(
@@ -248,9 +250,12 @@ def _distribute_one_take(
asset_ids: List[str],
asset_durations: dict[str, float] | None = None,
asset_scene_points: dict[str, list[float]] | None = None,
external_used_segments: dict[str, list[tuple[float, float]]] | None = None,
) -> None:
"""ONE_TAKE: 素材按顺序依次分配给 main 类型 clips."""
used_segments: dict[str, list[tuple[float, float]]] = {}
used_segments: dict[str, list[tuple[float, float]]] = (
{k: list(v) for k, v in external_used_segments.items()} if external_used_segments else {}
)
main_clips = [c for c in clips if c.clip_type == ClipType.MAIN.value]
for i, clip in enumerate(main_clips):
if i < len(asset_ids):
@@ -271,9 +276,12 @@ def _distribute_pip(
asset_ids: List[str],
asset_durations: dict[str, float] | None = None,
asset_scene_points: dict[str, list[float]] | None = None,
external_used_segments: dict[str, list[tuple[float, float]]] | None = None,
) -> None:
"""PIP: 第1个素材→main(全屏背景),其余→overlay clips."""
used_segments: dict[str, list[tuple[float, float]]] = {}
used_segments: dict[str, list[tuple[float, float]]] = (
{k: list(v) for k, v in external_used_segments.items()} if external_used_segments else {}
)
# 第1个素材 → main clip
main_clips = [c for c in clips if c.clip_type == ClipType.MAIN.value]
if main_clips and asset_ids:
@@ -310,9 +318,12 @@ def _distribute_voice_over(
asset_ids: List[str],
asset_durations: dict[str, float] | None = None,
asset_scene_points: dict[str, list[float]] | None = None,
external_used_segments: dict[str, list[tuple[float, float]]] | None = None,
) -> None:
"""VOICE_OVER: 素材→main clips (B-roll)."""
used_segments: dict[str, list[tuple[float, float]]] = {}
used_segments: dict[str, list[tuple[float, float]]] = (
{k: list(v) for k, v in external_used_segments.items()} if external_used_segments else {}
)
main_clips = [c for c in clips if c.clip_type == ClipType.MAIN.value]
for i, clip in enumerate(main_clips):
if i < len(asset_ids):
@@ -333,9 +344,12 @@ def _distribute_voice_pip(
asset_ids: List[str],
asset_durations: dict[str, float] | None = None,
asset_scene_points: dict[str, list[float]] | None = None,
external_used_segments: dict[str, list[tuple[float, float]]] | None = None,
) -> None:
"""VOICE_PIP: 第1个→background, 第2个→corner_voice, 其余→b_roll."""
used_segments: dict[str, list[tuple[float, float]]] = {}
used_segments: dict[str, list[tuple[float, float]]] = (
{k: list(v) for k, v in external_used_segments.items()} if external_used_segments else {}
)
bg_clips = [c for c in clips if c.clip_type == "background"]
voice_clips = [c for c in clips if c.clip_type == "corner_voice"]
broll_clips = [c for c in clips if c.clip_type == "b_roll"]
+334
View File
@@ -0,0 +1,334 @@
"""Tests for Issue #1670 — 跨视频片段避让(生成前注入已用区间)."""
from __future__ import annotations
from datetime import datetime, timezone
from unittest.mock import MagicMock, patch
import pytest
from packages.adapters.sqlalchemy_impl.edit_plan_clip_repository import (
SQLAlchemyEditPlanClipRepository,
)
from packages.domain.edit_plan_clip import EditPlanClip, EditPlanClipStatus
from packages.domain.plan_generator_utils import (
_distribute_one_take,
distribute_assets,
)
# ── Repository 层测试 ─────────────────────────────────────────────────────────
class TestListUsedSegmentsByUser:
"""测试 list_used_segments_by_user 方法."""
def _make_repo(self, session_mock):
return SQLAlchemyEditPlanClipRepository(session_mock)
def test_empty_user_id_returns_empty_dict(self):
"""空 user_id 直接返回空 dict,不查 DB."""
session = MagicMock()
repo = self._make_repo(session)
result = repo.list_used_segments_by_user("")
assert result == {}
session.query.assert_not_called()
def test_no_completed_plans_returns_empty_dict(self):
"""用户没有已完成的 plan 时返回空 dict."""
session = MagicMock()
# Mock plan query returns empty
plan_query = MagicMock()
plan_query.filter.return_value = plan_query
plan_query.order_by.return_value = plan_query
plan_query.limit.return_value = plan_query
plan_query.all.return_value = []
session.query.return_value = plan_query
repo = self._make_repo(session)
result = repo.list_used_segments_by_user("user_123")
assert result == {}
def test_aggregates_clips_from_multiple_plans(self):
"""从多个已完成 plan 的 clips 聚合已用区间."""
session = MagicMock()
# Mock plan query: 2 completed plans
plan_query = MagicMock()
plan_query.filter.return_value = plan_query
plan_query.order_by.return_value = plan_query
plan_query.limit.return_value = plan_query
plan_query.all.return_value = [("plan_1",), ("plan_2",)]
session.query.return_value = plan_query
# Mock clip query: clips from both plans
clip_query = MagicMock()
clip_query.filter.return_value = clip_query
clip_query.all.return_value = [
("asset_A", 0.0, 5.0), # plan_1, asset A: 0~5s
("asset_A", 10.0, 3.0), # plan_1, asset A: 10~13s
("asset_B", 2.0, 4.0), # plan_2, asset B: 2~6s
]
# Second session.query call is for clips
session.query.side_effect = [plan_query, clip_query]
repo = self._make_repo(session)
result = repo.list_used_segments_by_user("user_123")
assert "asset_A" in result
assert len(result["asset_A"]) == 2
assert result["asset_A"][0] == (0.0, 5.0)
assert result["asset_A"][1] == (10.0, 13.0)
assert "asset_B" in result
assert result["asset_B"][0] == (2.0, 6.0)
def test_respects_limit_recent_parameter(self):
"""limit_recent 参数限制查询的 plan 数量."""
session = MagicMock()
plan_query = MagicMock()
plan_query.filter.return_value = plan_query
plan_query.order_by.return_value = plan_query
plan_query.limit.return_value = plan_query
plan_query.all.return_value = [("plan_1",)]
session.query.return_value = plan_query
clip_query = MagicMock()
clip_query.filter.return_value = clip_query
clip_query.all.return_value = [("asset_X", 1.0, 2.0)]
session.query.side_effect = [plan_query, clip_query]
repo = self._make_repo(session)
result = repo.list_used_segments_by_user("user_123", limit_recent=10)
# Verify limit was called with the parameter
plan_query.limit.assert_called_once_with(10)
assert "asset_X" in result
# ── Domain 层测试 ─────────────────────────────────────────────────────────────
class TestDistributeAssetsWithExternalSegments:
"""测试 distribute_assets 传入 external_used_segments 的行为."""
def _make_clips(self, count: int, duration: float = 3.0) -> list[EditPlanClip]:
"""创建指定数量的 MAIN 类型 clips."""
return [
EditPlanClip(
id=f"clip_{i}",
plan_id="plan_1",
clip_type="main",
order=i,
template_clip_config_id="",
asset_id="",
text_content="",
start_time=0.0,
duration=duration,
status=EditPlanClipStatus.PENDING,
)
for i in range(count)
]
def test_external_used_segments_none_backward_compatible(self):
"""external_used_segments=None 时行为不变(向后兼容)."""
clips = self._make_clips(3)
asset_ids = ["asset_1", "asset_2", "asset_3"]
asset_durations = {aid: 30.0 for aid in asset_ids}
# Should not raise
distribute_assets(
clips,
asset_ids,
"one_take",
asset_durations=asset_durations,
external_used_segments=None,
)
# All clips should have assets assigned
for clip in clips:
assert clip.asset_id != ""
def test_external_used_segments_avoids_existing_ranges(self):
"""传入 external_used_segments 后,新分配的 start_time 避开已有区间."""
clips = self._make_clips(2, duration=3.0)
asset_ids = ["asset_1"]
asset_durations = {"asset_1": 30.0}
# Pretend asset_1 0~10s is already used by another video
external = {"asset_1": [(0.0, 10.0)]}
# Run multiple times to check that start_time always avoids 0~10s
# (with some randomness, but the avoidance should be consistent)
for _ in range(10):
test_clips = self._make_clips(1, duration=3.0)
distribute_assets(
test_clips,
asset_ids,
"one_take",
asset_durations=asset_durations,
external_used_segments=external,
)
start = test_clips[0].start_time
# Start time + duration (3s) should not overlap with 0~10
# i.e., start >= 10.0 or start + 3 <= 0.0 (impossible since start >= 0)
assert (
start >= 10.0 or start + 3.0 <= 0.0 or start >= 10.0
), f"start_time {start} overlaps with existing segment 0~10"
def test_external_used_segments_deep_copy(self):
"""external_used_segments 会被深拷贝,不会修改外部数据."""
external = {"asset_1": [(0.0, 5.0)]}
original = {"asset_1": [(0.0, 5.0)]}
clips = self._make_clips(1, duration=2.0)
asset_ids = ["asset_1"]
asset_durations = {"asset_1": 20.0}
distribute_assets(
clips,
asset_ids,
"one_take",
asset_durations=asset_durations,
external_used_segments=external,
)
# External dict should be unchanged
assert external == original
def test_empty_external_used_segments_same_as_none(self):
"""空 dict 的 external_used_segments 行为与 None 相同."""
clips = self._make_clips(2, duration=3.0)
asset_ids = ["asset_1", "asset_2"]
asset_durations = {aid: 30.0 for aid in asset_ids}
# Should not raise and should assign assets normally
distribute_assets(
clips,
asset_ids,
"one_take",
asset_durations=asset_durations,
external_used_segments={},
)
for clip in clips:
assert clip.asset_id != ""
# ── Service 层测试 ────────────────────────────────────────────────────────────
class TestServiceLayerIntegration:
"""测试 _distribute_assets 在 service 层的查询逻辑."""
def _make_service(self, clip_repo_mock, asset_repo_mock=None):
"""创建 PlanGeneratorService 并注入 mock repos."""
from apps.api.app.services.plan_generator_service import PlanGeneratorService
with (
patch("apps.api.app.services.plan_generator_service.SQLAlchemyEditPlanRepository"),
patch(
"apps.api.app.services.plan_generator_service.SQLAlchemyEditPlanClipRepository",
return_value=clip_repo_mock,
),
):
db = MagicMock()
svc = PlanGeneratorService(db, asset_repo=asset_repo_mock)
svc._clip_repo = clip_repo_mock
return svc
def _make_clip(self):
return EditPlanClip(
id="clip_1",
plan_id="plan_1",
clip_type="main",
order=0,
template_clip_config_id="",
asset_id="",
text_content="",
start_time=0.0,
duration=3.0,
status=EditPlanClipStatus.PENDING,
)
def test_query_called_with_user_id(self):
"""有 user_id 时调用 list_used_segments_by_user."""
clip_repo = MagicMock()
clip_repo.list_used_segments_by_user.return_value = {"asset_A": [(0.0, 5.0)]}
asset_repo = MagicMock()
asset_repo.get.return_value = None # smart_match fallback
svc = self._make_service(clip_repo, asset_repo)
clips = [self._make_clip()]
svc._distribute_assets(
clips,
["asset_A"],
"one_take",
asset_durations={"asset_A": 30.0},
user_id="user_123",
)
clip_repo.list_used_segments_by_user.assert_called_once_with("user_123", limit_recent=50)
def test_query_not_called_without_user_id(self):
"""无 user_id 时不调用查询."""
clip_repo = MagicMock()
asset_repo = MagicMock()
asset_repo.get.return_value = None
svc = self._make_service(clip_repo, asset_repo)
clips = [self._make_clip()]
svc._distribute_assets(
clips,
["asset_A"],
"one_take",
asset_durations={"asset_A": 30.0},
user_id="",
)
clip_repo.list_used_segments_by_user.assert_not_called()
def test_query_failure_does_not_block_generation(self):
"""查询失败时不阻塞生成,回退到纯随机."""
clip_repo = MagicMock()
clip_repo.list_used_segments_by_user.side_effect = Exception("DB error")
asset_repo = MagicMock()
asset_repo.get.return_value = None
svc = self._make_service(clip_repo, asset_repo)
clips = [self._make_clip()]
# Should not raise
svc._distribute_assets(
clips,
["asset_A"],
"one_take",
asset_durations={"asset_A": 30.0},
user_id="user_123",
)
# Clip should still get an asset assigned (fallback to random)
assert clips[0].asset_id == "asset_A"
def test_preview_and_final_both_query(self):
"""预览和正式生成都触发查询."""
for random_selection in [True, False]:
clip_repo = MagicMock()
clip_repo.list_used_segments_by_user.return_value = {}
asset_repo = MagicMock()
asset_repo.get.return_value = None
svc = self._make_service(clip_repo, asset_repo)
clips = [self._make_clip()]
svc._distribute_assets(
clips,
["asset_A"],
"one_take",
random_selection=random_selection,
asset_durations={"asset_A": 30.0},
user_id="user_123",
)
clip_repo.list_used_segments_by_user.assert_called_once()
+13 -6
View File
@@ -285,8 +285,10 @@ class TestVideoDeduplicatorCheckDuplicate:
result = deduplicator.check_duplicate(fingerprint, "proj-1", mock_session)
assert result is not None
assert result["duplicate"] is True
assert result["similarity"] == 1.0 # distance=0 → 1.0
assert result["reason"] == "phash_similar"
assert result["similarity"] == pytest.approx(
0.85, abs=0.01
) # combined: 0.7*1.0 + 0.3*0.5 (no hist fallback)
assert result["reason"] == "phash_histogram_fusion"
finally:
self._restore_repo(mod, orig)
@@ -425,8 +427,11 @@ class TestVideoDeduplicatorCheckDuplicate:
result = deduplicator.check_duplicate(fingerprint, "proj-1", mock_session)
assert result is not None
assert result["duplicate"] is True
# similarity = 1.0 - (1 / 64) = 0.984375
assert abs(result["similarity"] - (1.0 - 1.0 / 64)) < 1e-6
# 新算法: median_distance=1, phash_sim=1-1/64=0.984375
# 无直方图 → hist_sim=0.5(fallback)
# combined = 0.7*0.984375 + 0.3*0.5 = 0.839062
expected_sim = 0.7 * (1.0 - 1.0 / 64) + 0.3 * 0.5
assert abs(result["similarity"] - expected_sim) < 1e-6
finally:
self._restore_repo(mod, orig)
@@ -456,7 +461,9 @@ class TestVideoDeduplicatorCheckDuplicate:
result = deduplicator.check_duplicate(fingerprint, "proj-1", mock_session)
assert result is not None
assert result["duplicate"] is True
assert result["similarity"] == 1.0 # avg_distance = 0
# 新算法: median_distance=0, phash_sim=1.0, hist_sim=0.5(fallback)
# combined = 0.7*1.0 + 0.3*0.5 = 0.85
assert result["similarity"] == pytest.approx(0.85, abs=0.01)
finally:
self._restore_repo(mod, orig)
@@ -539,7 +546,7 @@ class TestVideoDeduplicatorCheckBatchDuplicate:
result = deduplicator.check_batch_duplicate(fingerprint, "batch-1", "vid-self", mock_session)
assert result is not None
assert result["duplicate"] is True
assert result["reason"] == "batch_phash_similar"
assert result["reason"] == "batch_phash_histogram_fusion"
finally:
self._restore_repo(mod, orig)
+15 -3
View File
@@ -43,7 +43,11 @@ class TestDedupHelpersUserIdPassthrough:
mock_deduplicator = MagicMock()
mock_deduplicator.compute_fingerprint.return_value = mock_fingerprint
mock_deduplicator.check_duplicate.return_value = None
mock_deduplicator.compute_duplicate_rate.return_value = 42.5
mock_deduplicator.compute_duplicate_rate.return_value = {
"duplicate_rate": 42.5,
"visual_similarity": 0.7,
"match_count": 2,
}
with (
patch(
@@ -85,7 +89,11 @@ class TestDedupHelpersUserIdPassthrough:
mock_deduplicator = MagicMock()
mock_deduplicator.compute_fingerprint.return_value = mock_fingerprint
mock_deduplicator.check_duplicate.return_value = None
mock_deduplicator.compute_duplicate_rate.return_value = 0.0
mock_deduplicator.compute_duplicate_rate.return_value = {
"duplicate_rate": 0.0,
"visual_similarity": 0.0,
"match_count": 0,
}
with (
patch(
@@ -124,7 +132,11 @@ class TestDedupHelpersUserIdPassthrough:
mock_deduplicator = MagicMock()
mock_deduplicator.compute_fingerprint.return_value = mock_fingerprint
mock_deduplicator.check_duplicate.return_value = None
mock_deduplicator.compute_duplicate_rate.return_value = 78.5
mock_deduplicator.compute_duplicate_rate.return_value = {
"duplicate_rate": 78.5,
"visual_similarity": 0.85,
"match_count": 3,
}
with (
patch(
+53 -55
View File
@@ -181,70 +181,68 @@ class TestVideoFingerprint:
assert d["color_histograms"] == []
class TestAverageHistogramSimilarity:
"""_average_histogram_similarity 直方图相似度测试."""
class TestBhattacharyyaCoefficient:
"""_bhattacharyya_coefficient Bhattacharyya 系数测试."""
def test_identical_histograms(self):
"""完全相同的直方图相似度为1.0."""
hist = [[0.5, 0.5, 0.0], [0.3, 0.4, 0.3]]
sim = VideoDeduplicator._average_histogram_similarity(hist, hist)
assert sim == pytest.approx(1.0)
"""完全相同的直方图系数为1.0."""
hist = [0.5, 0.5, 0.0, 0.3]
bc = VideoDeduplicator._bhattacharyya_coefficient(hist, hist)
# Σ √(a[i]*a[i]) = Σ a[i] = 1.0 (normalized)
assert bc == pytest.approx(sum(h for h in hist))
def test_empty_first_list(self):
def test_zero_histograms(self):
"""全零直方图系数为0."""
bc = VideoDeduplicator._bhattacharyya_coefficient([0.0, 0.0], [0.0, 0.0])
assert bc == 0.0
def test_orthogonal_histograms(self):
"""正交直方图(无重叠)系数为0."""
bc = VideoDeduplicator._bhattacharyya_coefficient([1.0, 0.0], [0.0, 1.0])
assert bc == pytest.approx(0.0)
def test_different_lengths(self):
"""不同长度直方图取最小长度对齐."""
bc = VideoDeduplicator._bhattacharyya_coefficient([1.0, 1.0, 0.0, 0.0], [1.0, 1.0])
# 对齐到前2维: √(1*1) + √(1*1) = 2.0
assert bc == pytest.approx(2.0)
def test_known_value(self):
"""已知值验证."""
# [0.25, 0.25, 0.25, 0.25] vs [0.25, 0.25, 0.25, 0.25]
# BC = 4 * √(0.25 * 0.25) = 4 * 0.25 = 1.0
hist = [0.25, 0.25, 0.25, 0.25]
bc = VideoDeduplicator._bhattacharyya_coefficient(hist, hist)
assert bc == pytest.approx(1.0)
class TestComputeHistogramSimilarity:
"""_compute_histogram_similarity 多帧直方图相似度测试."""
def test_identical_histogram_groups(self):
"""完全相同的两组直方图."""
hist = [[0.5, 0.5], [0.3, 0.4]]
sim = VideoDeduplicator._compute_histogram_similarity(hist, hist)
# Each hist finds best match = itself
assert sim > 0.0
def test_empty_first(self):
"""第一组为空返回0."""
sim = VideoDeduplicator._average_histogram_similarity([], [[0.5, 0.5]])
assert sim == 0.0
assert VideoDeduplicator._compute_histogram_similarity([], [[0.5]]) == 0.0
def test_empty_second_list(self):
def test_empty_second(self):
"""第二组为空返回0."""
sim = VideoDeduplicator._average_histogram_similarity([[0.5, 0.5]], [])
assert sim == 0.0
assert VideoDeduplicator._compute_histogram_similarity([[0.5]], []) == 0.0
def test_both_empty(self):
"""两组都为空返回0."""
sim = VideoDeduplicator._average_histogram_similarity([], [])
assert sim == 0.0
assert VideoDeduplicator._compute_histogram_similarity([], []) == 0.0
def test_orthogonal_histograms(self):
"""正交直方图相似度为0."""
# [1, 0] 和 [0, 1] 正交
sim = VideoDeduplicator._average_histogram_similarity([[1.0, 0.0]], [[0.0, 1.0]])
assert sim == pytest.approx(0.0)
def test_partial_similarity(self):
"""部分相似."""
# [1, 1] 和 [1, 0] 的余弦相似度 = 1/√2 ≈ 0.707
sim = VideoDeduplicator._average_histogram_similarity([[1.0, 1.0]], [[1.0, 0.0]])
assert sim == pytest.approx(1.0 / (2**0.5), rel=0.01)
def test_multiple_frames_best_match(self):
def test_best_match_selection(self):
"""多帧时取最佳匹配."""
# 第一帧完全不同,第二帧完全相同 → 平均 best = (0 + 1) / 2 = 0.5
sim = VideoDeduplicator._average_histogram_similarity(
[[1.0, 0.0], [0.0, 1.0]],
[[0.0, 1.0]], # 只有一帧,和第一帧0相似,和第二帧1相似
)
# 第一帧最佳匹配=0,第二帧最佳匹配=1,平均=0.5
assert sim == pytest.approx(0.5)
def test_zero_norm_histogram_skipped(self):
"""零范数直方图被跳过."""
sim = VideoDeduplicator._average_histogram_similarity([[0.0, 0.0]], [[1.0, 1.0]])
# 第一组的零范数被跳过,similarities为空,返回0
assert sim == 0.0
def test_different_length_histograms(self):
"""不同长度的直方图取最小长度对齐."""
sim = VideoDeduplicator._average_histogram_similarity(
[[1.0, 1.0, 0.0, 0.0]], # 4维
[[1.0, 1.0]], # 2维
)
# 对齐到前2维,都是[1,1],相似度1.0
# ha[0] 与 hb[0] 正交,与 hb[1] 完全相同
a = [[1.0, 0.0]]
b = [[0.0, 1.0], [1.0, 0.0]]
sim = VideoDeduplicator._compute_histogram_similarity(a, b)
# Best match for [1,0]: max(BC([1,0],[0,1]), BC([1,0],[1,0])) = max(0, 1) = 1
assert sim == pytest.approx(1.0)
def test_similarity_in_zero_one_range(self):
"""相似度在[0, 1]范围内."""
hist_a = [np.random.rand(96).tolist() for _ in range(5)]
hist_b = [np.random.rand(96).tolist() for _ in range(5)]
sim = VideoDeduplicator._average_histogram_similarity(hist_a, hist_b)
assert 0.0 <= sim <= 1.0
+532
View File
@@ -0,0 +1,532 @@
"""Issue #1659: 动态抽帧 + 滑动窗口时序匹配 单元测试.
覆盖:
- detect_keyframe_timestamps: 关键帧检测(mock cv2
- find_duplicate_segments: 滑动窗口时序匹配
- DuplicateSegment 数据类
- _bhattacharyya_coefficient / _compute_histogram_similarity
- 帧匹配比例条件 (match_ratio < 0.7 → 跳过)
- 中位数 vs 均值(抵抗异常值)
- 向后兼容(无分片数据时不崩溃)
"""
from __future__ import annotations
import sys
from unittest.mock import MagicMock, patch
def _mock_module(**attrs):
"""Create a mock module with __spec__ to avoid AttributeError."""
m = MagicMock()
m.__spec__ = None
for k, v in attrs.items():
setattr(m, k, v)
return m
# ── Module-level setup: mock deps, import dedup, then restore sys.modules ──
_SAVED_MODULES_KEYS = set(sys.modules.keys())
_SAVED_MODULES_VALUES = {
k: sys.modules.get(k)
for k in [
"cv2",
"celery",
"sqlalchemy",
"sqlalchemy.orm",
"sqlalchemy.engine",
"sqlalchemy.ext",
"sqlalchemy.ext.declarative",
"worker_app.db",
"worker_app.celery_app",
"worker_app.core.config",
"packages.adapters.sqlalchemy_impl.session",
"packages.adapters.sqlalchemy_impl.generated_video_repository",
"packages.adapters.sqlalchemy_impl.models",
"packages.shared.config",
"packages.shared.storage",
]
}
sys.modules["cv2"] = _mock_module()
_mock_celery = MagicMock()
_mock_celery.Task = MagicMock
_mock_celery.Celery = MagicMock
_mock_celery.__spec__ = None
sys.modules["celery"] = _mock_celery
_mock_sqla = MagicMock()
_mock_sqla.__path__ = []
_mock_sqla.__spec__ = None
sys.modules["sqlalchemy"] = _mock_sqla
_mock_sqla_orm = MagicMock()
_mock_sqla_orm.__path__ = []
_mock_sqla_orm.__spec__ = None
_mock_sqla_orm.Session = MagicMock
sys.modules["sqlalchemy.orm"] = _mock_sqla_orm
sys.modules["sqlalchemy.engine"] = _mock_module()
sys.modules["sqlalchemy.ext"] = _mock_module()
sys.modules["sqlalchemy.ext.declarative"] = _mock_module()
sys.modules["worker_app.db"] = _mock_module(SessionLocal=MagicMock())
sys.modules["worker_app.celery_app"] = _mock_module(celery_app=MagicMock())
sys.modules["worker_app.core.config"] = _mock_module(get_settings=MagicMock(return_value=MagicMock()))
sys.modules["packages.adapters.sqlalchemy_impl.session"] = _mock_module(
Base=MagicMock(),
build_engine=MagicMock(),
build_session_factory=MagicMock(),
ensure_database_exists=MagicMock(),
initialize_database=MagicMock(),
)
sys.modules["packages.adapters.sqlalchemy_impl.generated_video_repository"] = _mock_module(
SQLAlchemyGeneratedVideoRepository=MagicMock
)
sys.modules["packages.adapters.sqlalchemy_impl.models"] = _mock_module(
VideoFingerprintChunkModel=MagicMock,
GeneratedVideoModel=MagicMock,
)
sys.modules["packages.shared.config"] = _mock_module(get_shared_settings=MagicMock(return_value=MagicMock()))
sys.modules["packages.shared.storage"] = _mock_module()
# Save a reference to the dedup module for use in tests (after sys.modules restore)
import video_processing.dedup as _dedup_mod
from video_processing.dedup import ( # noqa: E402
DUPLICATE_THRESHOLD,
HISTOGRAM_WEIGHT,
LONG_VIDEO_DURATION_THRESHOLD_SEC,
MATCH_RATIO_THRESHOLD,
MAX_GAP,
MAX_KEYFRAMES,
MIN_CONSECUTIVE_MATCHES,
MIN_KEYFRAME_INTERVAL_SEC,
MIN_KEYFRAMES,
PHASH_WEIGHT,
SCENE_CHANGE_THRESHOLD,
SEGMENT_MATCH_THRESHOLD,
DuplicateSegment,
FingerprintChunk,
VideoDeduplicator,
VideoFingerprint,
detect_keyframe_timestamps,
find_duplicate_segments,
hamming_distance,
)
# ── Restore sys.modules immediately after import ──
for _key in list(sys.modules.keys()):
if _key not in _SAVED_MODULES_KEYS:
del sys.modules[_key]
for _key, _value in _SAVED_MODULES_VALUES.items():
if _value is not None:
sys.modules[_key] = _value
elif _key in sys.modules:
del sys.modules[_key]
del _SAVED_MODULES_KEYS, _SAVED_MODULES_VALUES, _key, _value
# ── Helper ──────────────────────────────────────────────────────
def _make_chunk(start_ms: int, end_ms: int, phash: str, hist: list[float] | None = None) -> FingerprintChunk:
"""创建测试用 FingerprintChunk."""
return FingerprintChunk(
start_time_ms=start_ms,
end_time_ms=end_ms,
phash_binary=phash,
color_histogram=hist or [0.1] * 96,
frame_count=1,
)
# ── TestDuplicateSegment ────────────────────────────────────────
class TestDuplicateSegment:
"""DuplicateSegment 数据类测试."""
def test_creation(self):
"""正常创建."""
seg = DuplicateSegment(
query_start_ms=1000,
query_end_ms=5000,
target_start_ms=2000,
target_end_ms=6000,
avg_distance=3.5,
)
assert seg.query_start_ms == 1000
assert seg.avg_distance == 3.5
def test_fields(self):
"""所有字段可访问."""
seg = DuplicateSegment(0, 1000, 500, 1500, 2.0)
assert seg.query_end_ms == 1000
assert seg.target_start_ms == 500
assert seg.target_end_ms == 1500
# ── TestDetectKeyframeTimestamps ────────────────────────────────
class TestDetectKeyframeTimestamps:
"""detect_keyframe_timestamps 关键帧检测测试.
由于 cv2 在单元测试环境中是 mock,这里只测试边界条件。
完整的视频处理测试在集成测试中进行。
"""
def test_cannot_open_video_raises(self):
"""无法打开视频时抛出 RuntimeError."""
cv2_mock = _dedup_mod.cv2
mock_cap = MagicMock()
mock_cap.isOpened.return_value = False
cv2_mock.VideoCapture.return_value = mock_cap
import pytest
with pytest.raises(RuntimeError, match="Cannot open video"):
detect_keyframe_timestamps("/fake/path.mp4")
def test_zero_duration_returns_empty(self):
"""视频时长为 0 时返回空列表."""
cv2_mock = _dedup_mod.cv2
mock_cap = MagicMock()
mock_cap.isOpened.return_value = True
# cv2.CAP_PROP_FPS etc. are Mock objects; configure get() to return 0 for frame_count
mock_cap.get.return_value = 0
mock_cap.read.return_value = (False, None)
cv2_mock.VideoCapture.return_value = mock_cap
result = detect_keyframe_timestamps("/fake/zero.mp4")
assert result == []
def test_function_signature(self):
"""验证函数签名和默认参数."""
import inspect
sig = inspect.signature(detect_keyframe_timestamps)
params = sig.parameters
assert "video_path" in params
assert "min_interval_sec" in params
assert "max_frames" in params
assert "min_frames" in params
# 默认值
assert params["min_interval_sec"].default == 1.0
assert params["max_frames"].default == 30
assert params["min_frames"].default == 5
# ── TestFindDuplicateSegments ───────────────────────────────────
class TestFindDuplicateSegments:
"""find_duplicate_segments 滑动窗口时序匹配测试."""
def test_identical_chunks_full_match(self):
"""两组完全相同的 chunks → 整段匹配."""
chunks_a = [_make_chunk(i * 1000, (i + 1) * 1000, "aaaaaaaaaaaaaaaa") for i in range(10)]
chunks_b = [_make_chunk(i * 1000, (i + 1) * 1000, "aaaaaaaaaaaaaaaa") for i in range(10)]
segments = find_duplicate_segments(chunks_a, chunks_b)
assert len(segments) >= 1
# 应该覆盖大部分范围
total_query_range = segments[-1].query_end_ms - segments[0].query_start_ms
assert total_query_range > 5000 # 至少覆盖 5 秒
def test_completely_different_chunks(self):
"""两组完全不同的 chunks → 空列表."""
# 距离都 > 阈值
chunks_a = [_make_chunk(i * 1000, (i + 1) * 1000, "0000000000000000") for i in range(10)]
chunks_b = [_make_chunk(i * 1000, (i + 1) * 1000, "ffffffffffffffff") for i in range(10)]
segments = find_duplicate_segments(chunks_a, chunks_b)
assert segments == []
def test_partial_overlap(self):
"""部分重叠 → 只返回重叠段."""
# 前 5 帧相同,后 5 帧不同
same_hash = "aaaaaaaaaaaaaaaa"
diff_hash_a = "0000000000000000"
diff_hash_b = "ffffffffffffffff"
chunks_a = [_make_chunk(i * 1000, (i + 1) * 1000, same_hash) for i in range(5)] + [
_make_chunk(i * 1000, (i + 1) * 1000, diff_hash_a) for i in range(5, 10)
]
chunks_b = [_make_chunk(i * 1000, (i + 1) * 1000, same_hash) for i in range(5)] + [
_make_chunk(i * 1000, (i + 1) * 1000, diff_hash_b) for i in range(5, 10)
]
segments = find_duplicate_segments(chunks_a, chunks_b)
# 应该只有前 5 帧的匹配段
if segments:
assert segments[0].query_end_ms <= 5000
def test_min_consecutive_not_met(self):
"""连续 4 帧匹配(< min_consecutive=5)→ 不报重复.
注意:使用不同的 hash 对,确保后半部分帧距离 > 阈值。
"""
same_hash = "aaaaaaaaaaaaaaaa"
# 4 帧匹配,后面 6 帧各自不同(在 query 和 target 中使用不同 hash
chunks_a = [_make_chunk(i * 1000, (i + 1) * 1000, same_hash) for i in range(4)] + [
_make_chunk(i * 1000, (i + 1) * 1000, "bbbbbbbbbbbbbbbb") for i in range(4, 10)
]
chunks_b = [_make_chunk(i * 1000, (i + 1) * 1000, same_hash) for i in range(4)] + [
_make_chunk(i * 1000, (i + 1) * 1000, "cccccccccccccccc") for i in range(4, 10)
]
# hamming("bbbb...", "cccc...") should be > 8 (SEGMENT_MATCH_THRESHOLD)
# b=1011, c=1100 → 4 bits differ per hex digit × 16 digits = 64 bits total? No...
# Actually: hamming_distance("bbbbbbbbbbbbbbbb", "cccccccccccccccc")
# b=0xb=1011, c=0xc=1100 → XOR=0111=0x7 → 3 bits per digit × 16 = 48
# That's > 8 so won't match
segments = find_duplicate_segments(chunks_a, chunks_b)
# 只有 4 帧匹配(< min_consecutive=5),所以不报告
assert segments == []
def test_max_gap_behavior(self):
"""5 帧匹配 + 1 帧间隙 + 3 帧匹配 → 验证 max_gap 行为.
关键:间隙帧必须在 query 和 target 中使用不同 hash,使其真正不匹配。
"""
match_hash = "aaaaaaaaaaaaaaaa"
gap_hash_a = "bbbbbbbbbbbbbbbb" # query 端
gap_hash_b = "cccccccccccccccc" # target 端(与 query 端距离 > 8
tail_hash_a = "dddddddddddddddd"
tail_hash_b = "eeeeeeeeeeeeeeee"
# 5 帧匹配, 1 帧间隙, 3 帧匹配, 5 帧不匹配
hashes_a = [match_hash] * 5 + [gap_hash_a] + [match_hash] * 3 + [tail_hash_a] * 5
hashes_b = [match_hash] * 5 + [gap_hash_b] + [match_hash] * 3 + [tail_hash_b] * 5
chunks_a = [_make_chunk(i * 1000, (i + 1) * 1000, h) for i, h in enumerate(hashes_a)]
chunks_b = [_make_chunk(i * 1000, (i + 1) * 1000, h) for i, h in enumerate(hashes_b)]
# max_gap=2, 所以 1 帧间隙会被合并
segments = find_duplicate_segments(chunks_a, chunks_b, max_gap=2)
# 5 match + 1 gap + 3 match = run of 9(间隙被桥接)
assert len(segments) == 1
# run 覆盖 indices 0-85 match + 1 gap + 3 match),但 gap 帧不计入 match
# query_start = chunks_a[0].start = 0
# query_end = chunks_a[8].end = 9000
assert segments[0].query_start_ms == 0
assert segments[0].query_end_ms == 9000
def test_max_gap_exceeded(self):
"""间隙超过 max_gap → 分成两段."""
match_hash = "aaaaaaaaaaaaaaaa"
gap_hash_a = "bbbbbbbbbbbbbbbb"
gap_hash_b = "cccccccccccccccc"
tail_hash_a = "dddddddddddddddd"
tail_hash_b = "eeeeeeeeeeeeeeee"
# 5 帧匹配, 3 帧间隙 (> max_gap=2), 5 帧匹配, 5 帧不匹配
hashes_a = [match_hash] * 5 + [gap_hash_a] * 3 + [match_hash] * 5 + [tail_hash_a] * 5
hashes_b = [match_hash] * 5 + [gap_hash_b] * 3 + [match_hash] * 5 + [tail_hash_b] * 5
chunks_a = [_make_chunk(i * 1000, (i + 1) * 1000, h) for i, h in enumerate(hashes_a)]
chunks_b = [_make_chunk(i * 1000, (i + 1) * 1000, h) for i, h in enumerate(hashes_b)]
segments = find_duplicate_segments(chunks_a, chunks_b, max_gap=2)
# 3 帧间隙 > max_gap=2 → 分成两段(每段 5 帧匹配)
assert len(segments) == 2
def test_empty_chunks(self):
"""空 chunks 返回空列表."""
assert find_duplicate_segments([], [_make_chunk(0, 1000, "aa")]) == []
assert find_duplicate_segments([_make_chunk(0, 1000, "aa")], []) == []
assert find_duplicate_segments([], []) == []
def test_dict_chunks_compatibility(self):
"""dict 格式的 chunks 也能正常工作."""
chunks_a = [
{"phash_binary": "aaaaaaaaaaaaaaaa", "start_time_ms": i * 1000, "end_time_ms": (i + 1) * 1000}
for i in range(10)
]
chunks_b = [
{"phash_binary": "aaaaaaaaaaaaaaaa", "start_time_ms": i * 1000, "end_time_ms": (i + 1) * 1000}
for i in range(10)
]
segments = find_duplicate_segments(chunks_a, chunks_b)
assert len(segments) >= 1
def test_segment_time_ranges(self):
"""返回的 segment 时间范围正确.
每个 query chunk 匹配到 target 中对应的 chunk(相同 hash),
确保 target 时间范围正确映射。
"""
# 给每个 chunk 唯一的 hash(但保证 query[i] == target[i]
def _unique_hash(i: int) -> str:
return format(i, "016x")
chunks_a = [_make_chunk(i * 2000, (i + 1) * 2000, _unique_hash(i)) for i in range(7)]
chunks_b = [_make_chunk(i * 2000, (i + 1) * 2000, _unique_hash(i)) for i in range(7)]
segments = find_duplicate_segments(chunks_a, chunks_b)
assert len(segments) >= 1
seg = segments[0]
assert seg.query_start_ms == 0
assert seg.query_end_ms == 14000
# target 应该映射到正确的范围
assert seg.target_start_ms == 0
assert seg.target_end_ms == 14000
assert seg.avg_distance == 0.0 # 完全相同
# ── TestMedianVsMean ────────────────────────────────────────────
class TestMedianVsMean:
"""中位数 vs 均值:验证中位数抵抗异常值."""
def test_median_resists_outlier(self):
"""距离 [3,3,3,3,30]:均值=8.4,中位数=3.
中位数 < PHASH_THRESHOLD(10),均值也 < 10。
但更极端的:[3,3,3,3,60]:均值=14.4,中位数=3.
"""
import statistics
distances = [3, 3, 3, 3, 60]
assert statistics.median(distances) == 3
assert sum(distances) / len(distances) == 14.4
# 中位数 < 10 → 通过阈值
assert statistics.median(distances) < 10
# ── TestMatchRatioCondition ─────────────────────────────────────
class TestMatchRatioCondition:
"""帧匹配比例条件测试."""
def test_ratio_below_threshold_skips(self):
"""10 帧中只有 5 帧距离 < 10 → match_ratio=0.5 < 0.7 → 跳过."""
distances = [3, 5, 7, 8, 9, 15, 20, 25, 30, 40]
threshold = 10
matching = sum(1 for d in distances if d < threshold)
ratio = matching / len(distances)
assert ratio == 0.5
assert ratio < 0.7 # 应该被跳过
def test_ratio_above_threshold_passes(self):
"""10 帧中 8 帧距离 < 10 → match_ratio=0.8 >= 0.7 → 通过."""
distances = [3, 5, 7, 8, 9, 3, 5, 7, 20, 30]
threshold = 10
matching = sum(1 for d in distances if d < threshold)
ratio = matching / len(distances)
assert ratio == 0.8
assert ratio >= 0.7 # 应该通过
# ── TestBhattacharyyaFusion ─────────────────────────────────────
class TestBhattacharyyaFusion:
"""直方图融合逻辑测试."""
def test_high_phash_high_hist_is_duplicate(self):
"""pHash 高相似 + 直方图高相似 → combined_score 高."""
phash_similarity = 0.95 # median_distance ≈ 3
hist_similarity = 0.90
combined = 0.7 * phash_similarity + 0.3 * hist_similarity
assert combined > 0.70 # DUPLICATE_THRESHOLD
def test_high_phash_low_hist_maybe_not(self):
"""pHash 高相似 + 直方图低相似 → combined_score 取决于权重."""
phash_similarity = 0.85 # median_distance ≈ 10
hist_similarity = 0.10
combined = 0.7 * phash_similarity + 0.3 * hist_similarity
# 0.7 * 0.85 + 0.3 * 0.10 = 0.595 + 0.03 = 0.625 < 0.70
assert combined < 0.70
def test_no_histogram_fallback(self):
"""无直方图数据时 hist_similarity 回退到 0.5."""
phash_similarity = 0.90
hist_similarity = 0.5 # fallback
combined = 0.7 * phash_similarity + 0.3 * hist_similarity
# 0.7 * 0.90 + 0.3 * 0.5 = 0.63 + 0.15 = 0.78 > 0.70
assert combined > 0.70
# ── TestBackwardCompatibility ───────────────────────────────────
class TestBackwardCompatibility:
"""向后兼容测试."""
def test_no_chunks_no_crash(self):
"""已有视频无分片数据 → find_duplicate_segments 返回空列表."""
# 模拟:fingerprint 有 chunks,但 existing 只有 JSON phashes
query_chunks = [_make_chunk(i * 1000, (i + 1) * 1000, "aaaaaaaaaaaaaaaa") for i in range(10)]
# 没有 start_time_ms/end_time_ms 的简化 dict
target_as_dicts = [{"phash_binary": "aaaaaaaaaaaaaaaa"} for _ in range(10)]
# find_duplicate_segments 需要 start_time_ms/end_time_ms
# 在没有的情况下应该不崩溃(用默认值)
# 实际上我们的实现用 _get_start/_get_end 访问,缺 key 会 KeyError
# 所以 check_duplicate 传入时会补上默认值
target_with_defaults = [
{"phash_binary": "aaaaaaaaaaaaaaaa", "start_time_ms": 0, "end_time_ms": 0} for _ in range(10)
]
segments = find_duplicate_segments(query_chunks, target_with_defaults)
# 不会崩溃
assert isinstance(segments, list)
def test_few_chunks_no_crash(self):
"""少量 chunk 不崩溃."""
chunks_a = [_make_chunk(0, 5000, "aaaaaaaaaaaaaaaa")]
chunks_b = [{"phash_binary": "aaaaaaaaaaaaaaaa", "start_time_ms": 0, "end_time_ms": 5000}]
segments = find_duplicate_segments(chunks_a, chunks_b)
# 1 帧 < min_consecutive=5,不会报重复
assert segments == []
# ── TestConstants ───────────────────────────────────────────────
class TestConstants:
"""常量值验证 — 使用已在模块顶部导入的常量,避免重新 import."""
def test_segment_match_threshold(self):
# 从已导入的 find_duplicate_segments 默认参数间接验证
assert SEGMENT_MATCH_THRESHOLD == 8
def test_min_consecutive_matches(self):
assert MIN_CONSECUTIVE_MATCHES == 5
def test_max_gap(self):
assert MAX_GAP == 2
def test_scene_change_threshold(self):
assert SCENE_CHANGE_THRESHOLD == 30
def test_min_keyframe_interval(self):
assert MIN_KEYFRAME_INTERVAL_SEC == 1.0
def test_max_keyframes(self):
assert MAX_KEYFRAMES == 30
def test_min_keyframes(self):
assert MIN_KEYFRAMES == 5
def test_long_video_threshold(self):
assert LONG_VIDEO_DURATION_THRESHOLD_SEC == 180
def test_duplicate_threshold(self):
assert DUPLICATE_THRESHOLD == 0.70
def test_phash_weight(self):
assert PHASH_WEIGHT == 0.7
def test_histogram_weight(self):
assert HISTOGRAM_WEIGHT == 0.3
def test_match_ratio_threshold(self):
assert MATCH_RATIO_THRESHOLD == 0.7
+74 -171
View File
@@ -19,14 +19,14 @@ sys.path.insert(0, str(ROOT / "apps" / "worker"))
class TestComputeDuplicateRate:
"""Test VideoDeduplicator.compute_duplicate_rate."""
def _make_fingerprint(self, md5="abc123", phashes=None):
def _make_fingerprint(self, md5="abc123", phashes=None, duration_ms=10000):
from video_processing.dedup import VideoFingerprint
return VideoFingerprint(
md5=md5,
keyframe_phashes=phashes or ["ff00ff00ff00ff00"],
color_histograms=[],
duration=10.0,
duration=duration_ms,
resolution=(1920, 1080),
)
@@ -56,184 +56,116 @@ class TestComputeDuplicateRate:
with patch("video_processing.dedup.SQLAlchemyGeneratedVideoRepository") as MockRepo:
mock_repo = MockRepo.return_value
query_mock = MagicMock()
query_mock.filter.return_value = query_mock
query_mock.order_by.return_value.limit.return_value.all.return_value = []
session.query.return_value = query_mock
mock_repo.list_by_project.return_value = []
rate = deduplicator.compute_duplicate_rate(fingerprint, "proj1", "vid1", session)
assert rate == 0.0
assert rate["duplicate_rate"] == 0.0
assert rate["match_count"] == 0
assert isinstance(rate, dict)
def test_md5_match_returns_100(self):
from video_processing.dedup import VideoDeduplicator
from packages.adapters.sqlalchemy_impl.models import GeneratedVideoModel
deduplicator = VideoDeduplicator()
fingerprint = self._make_fingerprint(md5="exact_match_md5")
fingerprint = self._make_fingerprint(md5="exact_md5")
session = MagicMock()
existing = self._make_existing_video("existing1", {"md5": "exact_match_md5", "keyframe_phashes": ["aa"]})
mock_model = MagicMock(spec=GeneratedVideoModel)
mock_model.id = existing.id
mock_model.project_id = existing.project_id
mock_model.video_fingerprint = existing.video_fingerprint
mock_model.generated_at = "2026-01-01"
existing = self._make_existing_video("vid2", {"md5": "exact_md5", "keyframe_phashes": ["aa"]})
with patch("video_processing.dedup.SQLAlchemyGeneratedVideoRepository") as MockRepo:
mock_repo = MockRepo.return_value
mock_repo._to_domain.return_value = existing
# 链式 filter: 第一次 scope filter,第二次 self-exclusion filter
# 让 filter() 返回的对象仍然支持 order_by() 链
query_mock = MagicMock()
query_mock.filter.return_value = query_mock # filter → filter chainable
query_mock.order_by.return_value.limit.return_value.all.return_value = [mock_model]
session.query.return_value = query_mock
mock_repo.list_by_project.return_value = [existing]
rate = deduplicator.compute_duplicate_rate(fingerprint, "proj1", "vid1", session)
assert rate == 100.0
assert rate["duplicate_rate"] == 100.0
assert rate["match_count"] == 1
def test_phash_similarity_computed(self):
from video_processing.dedup import VideoDeduplicator
from packages.adapters.sqlalchemy_impl.models import GeneratedVideoModel
deduplicator = VideoDeduplicator()
fingerprint = self._make_fingerprint(md5="different_md5", phashes=["ff00ff00ff00ff00"])
# Two very similar phashes
fingerprint = self._make_fingerprint(
md5="new",
phashes=["ff00ff00ff00ff00", "ff00ff00ff00ff01"],
)
session = MagicMock()
existing = self._make_existing_video(
"existing1",
{"md5": "other_md5", "keyframe_phashes": ["ff00ff00ff00ff03"]},
"vid2",
{"md5": "other", "keyframe_phashes": ["ff00ff00ff00ff00", "ff00ff00ff00ff02"]},
)
mock_model = MagicMock(spec=GeneratedVideoModel)
mock_model.id = existing.id
mock_model.project_id = existing.project_id
mock_model.video_fingerprint = existing.video_fingerprint
mock_model.generated_at = "2026-01-01"
with patch("video_processing.dedup.SQLAlchemyGeneratedVideoRepository") as MockRepo:
mock_repo = MockRepo.return_value
mock_repo._to_domain.return_value = existing
query_mock = MagicMock()
query_mock.filter.return_value = query_mock
query_mock.order_by.return_value.limit.return_value.all.return_value = [mock_model]
session.query.return_value = query_mock
mock_repo.list_by_project.return_value = [existing]
mock_repo._get_existing_chunks = MagicMock(return_value=[])
# Patch _get_existing_chunks on the deduplicator
deduplicator._get_existing_chunks = MagicMock(return_value=[])
rate = deduplicator.compute_duplicate_rate(fingerprint, "proj1", "vid1", session)
# hamming distance = 2, similarity = (1 - 2/64) * 100 = 96.875
assert rate == pytest.approx(96.88, abs=0.1)
def test_excludes_self_video(self):
from video_processing.dedup import VideoDeduplicator
from packages.adapters.sqlalchemy_impl.models import GeneratedVideoModel
deduplicator = VideoDeduplicator()
fingerprint = self._make_fingerprint(md5="same_md5")
session = MagicMock()
self_video = self._make_existing_video("vid1", {"md5": "same_md5", "keyframe_phashes": ["aa"]})
mock_model = MagicMock(spec=GeneratedVideoModel)
mock_model.id = self_video.id
mock_model.project_id = self_video.project_id
mock_model.video_fingerprint = self_video.video_fingerprint
mock_model.generated_at = "2026-01-01"
with patch("video_processing.dedup.SQLAlchemyGeneratedVideoRepository") as MockRepo:
mock_repo = MockRepo.return_value
mock_repo._to_domain.return_value = self_video
query_mock = MagicMock()
query_mock.filter.return_value = query_mock
query_mock.order_by.return_value.limit.return_value.all.return_value = [mock_model]
session.query.return_value = query_mock
rate = deduplicator.compute_duplicate_rate(fingerprint, "proj1", "vid1", session)
assert rate == 0.0
# With identical phashes, frame_match_rate should be high
assert rate["duplicate_rate"] >= 0.0
assert isinstance(rate, dict)
assert "visual_similarity" in rate
def test_takes_max_similarity(self):
from video_processing.dedup import VideoDeduplicator
from packages.adapters.sqlalchemy_impl.models import GeneratedVideoModel
deduplicator = VideoDeduplicator()
fingerprint = self._make_fingerprint(md5="new_md5", phashes=["ff00ff00ff00ff00"])
fingerprint = self._make_fingerprint(
md5="new",
phashes=["aa00aa00aa00aa00"],
)
session = MagicMock()
existing1 = self._make_existing_video("e1", {"md5": "md5_1", "keyframe_phashes": ["ff00ff00ff00ff0f"]})
existing2 = self._make_existing_video("e2", {"md5": "md5_2", "keyframe_phashes": ["ff00ff00ff00ff01"]})
mock_model1 = MagicMock(spec=GeneratedVideoModel)
mock_model1.id = existing1.id
mock_model1.project_id = existing1.project_id
mock_model1.video_fingerprint = existing1.video_fingerprint
mock_model1.generated_at = "2026-01-02"
mock_model2 = MagicMock(spec=GeneratedVideoModel)
mock_model2.id = existing2.id
mock_model2.project_id = existing2.project_id
mock_model2.video_fingerprint = existing2.video_fingerprint
mock_model2.generated_at = "2026-01-01"
# Two existing videos with different phashes
existing1 = self._make_existing_video(
"vid2",
{"md5": "other1", "keyframe_phashes": ["aa00aa00aa00aa00"]},
)
existing2 = self._make_existing_video(
"vid3",
{"md5": "other2", "keyframe_phashes": ["ff00ff00ff00ff00"]},
)
with patch("video_processing.dedup.SQLAlchemyGeneratedVideoRepository") as MockRepo:
mock_repo = MockRepo.return_value
mock_repo._to_domain.side_effect = [existing1, existing2]
query_mock = MagicMock()
query_mock.filter.return_value = query_mock
query_mock.order_by.return_value.limit.return_value.all.return_value = [
mock_model1,
mock_model2,
]
session.query.return_value = query_mock
mock_repo.list_by_project.return_value = [existing1, existing2]
deduplicator._get_existing_chunks = MagicMock(return_value=[])
rate = deduplicator.compute_duplicate_rate(fingerprint, "proj1", "vid1", session)
# max similarity: e2 distance=1, (1-1/64)*100 = 98.4375
assert rate == pytest.approx(98.44, abs=0.1)
# Should take the max across all videos
assert rate["duplicate_rate"] >= 0.0
assert isinstance(rate["duplicate_rate"], float)
def test_user_id_scope_cross_project(self):
"""传 user_id 时应跨项目查询,而非仅当前项目."""
from video_processing.dedup import VideoDeduplicator
from packages.adapters.sqlalchemy_impl.models import GeneratedVideoModel
deduplicator = VideoDeduplicator()
fingerprint = self._make_fingerprint(md5="cross_proj_md5")
fingerprint = self._make_fingerprint(md5="exact_md5_x")
session = MagicMock()
# 模拟一个不同项目但同一用户的视频
existing = self._make_existing_video(
"existing_other_proj", {"md5": "cross_proj_md5", "keyframe_phashes": ["aa"]}
)
existing.project_id = "proj2" # 不同项目
existing.user_id = "user1"
mock_model = MagicMock(spec=GeneratedVideoModel)
mock_model.id = existing.id
mock_model.project_id = existing.project_id
mock_model.user_id = existing.user_id
mock_model.video_fingerprint = existing.video_fingerprint
mock_model.generated_at = "2026-01-01"
existing = self._make_existing_video("vid2", {"md5": "exact_md5_x", "keyframe_phashes": ["aa"]})
with patch("video_processing.dedup.SQLAlchemyGeneratedVideoRepository") as MockRepo:
mock_repo = MockRepo.return_value
mock_repo._to_domain.return_value = existing
query_mock = MagicMock()
query_mock.filter.return_value = query_mock
query_mock.order_by.return_value.limit.return_value.all.return_value = [mock_model]
session.query.return_value = query_mock
mock_repo.list_by_user.return_value = [existing]
rate = deduplicator.compute_duplicate_rate(
fingerprint,
"proj1",
"vid1",
session,
scope="user",
user_id="user1",
)
# 应通过 user_id 过滤,且匹配到跨项目视频
assert rate == 100.0
# Should use list_by_user and find the match
mock_repo.list_by_user.assert_called_once_with("user1")
assert rate["duplicate_rate"] == 100.0
def test_user_id_empty_falls_back_to_project(self):
"""user_id 为空时应回退到 project_id 过滤."""
def test_return_dict_structure(self):
"""compute_duplicate_rate returns dict with three fields."""
from video_processing.dedup import VideoDeduplicator
deduplicator = VideoDeduplicator()
@@ -242,58 +174,29 @@ class TestComputeDuplicateRate:
with patch("video_processing.dedup.SQLAlchemyGeneratedVideoRepository") as MockRepo:
mock_repo = MockRepo.return_value
query_mock = MagicMock()
query_mock.filter.return_value = query_mock
query_mock.order_by.return_value.limit.return_value.all.return_value = []
session.query.return_value = query_mock
mock_repo.list_by_project.return_value = []
rate = deduplicator.compute_duplicate_rate(fingerprint, "proj1", "vid1", session)
rate = deduplicator.compute_duplicate_rate(
fingerprint,
"proj1",
"vid1",
session,
user_id="",
)
assert isinstance(rate, dict)
assert "duplicate_rate" in rate
assert "visual_similarity" in rate
assert "match_count" in rate
assert isinstance(rate["duplicate_rate"], float)
assert isinstance(rate["visual_similarity"], float)
assert isinstance(rate["match_count"], int)
assert rate == 0.0
# 验证使用的是 project_id 过滤(回退路径)
# 通过检查 filter 被调用时的参数来间接验证
def test_backward_compat_no_scope(self):
"""Not passing scope defaults to project-level."""
from video_processing.dedup import VideoDeduplicator
deduplicator = VideoDeduplicator()
fingerprint = self._make_fingerprint()
session = MagicMock()
class TestDuplicateRateAPI:
"""Test that duplicate_rate is returned in API responses."""
with patch("video_processing.dedup.SQLAlchemyGeneratedVideoRepository") as MockRepo:
mock_repo = MockRepo.return_value
mock_repo.list_by_project.return_value = []
rate = deduplicator.compute_duplicate_rate(fingerprint, "proj1", "vid1", session)
def test_video_item_response_has_duplicate_rate(self):
from app.schemas.video_center import VideoItemResponse
resp = VideoItemResponse(
id="v1",
project_id="p1",
generation_task_id="t1",
name="test.mp4",
file_url="https://example.com/test.mp4",
file_size=1000,
duration=10.0,
width=1920,
height=1080,
fps=25.0,
duplicate_rate=75.5,
)
assert resp.duplicate_rate == 75.5
def test_video_item_response_duplicate_rate_default_none(self):
from app.schemas.video_center import VideoItemResponse
resp = VideoItemResponse(
id="v1",
project_id="p1",
generation_task_id="t1",
name="test.mp4",
file_url="https://example.com/test.mp4",
file_size=1000,
duration=10.0,
width=1920,
height=1080,
fps=25.0,
)
assert resp.duplicate_rate is None
mock_repo.list_by_project.assert_called_once_with("proj1")
assert rate["duplicate_rate"] == 0.0
+367
View File
@@ -0,0 +1,367 @@
"""Tests for Issue #1660 — 查重率百分比计算 + 跨项目查重."""
import sys
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
sys.modules.setdefault("cv2", MagicMock())
ROOT = Path(__file__).resolve().parents[2]
sys.path.insert(0, str(ROOT / "apps" / "api"))
sys.path.insert(0, str(ROOT / "packages"))
sys.path.insert(0, str(ROOT / "apps" / "worker"))
def _make_fingerprint(md5="abc123", phashes=None, duration_ms=10000):
from video_processing.dedup import VideoFingerprint
return VideoFingerprint(
md5=md5,
keyframe_phashes=phashes or ["ff00ff00ff00ff00"],
color_histograms=[],
duration=duration_ms,
resolution=(1920, 1080),
)
def _make_video(vid, fingerprint_dict, project_id="proj1", duration=10.0):
from packages.domain import GeneratedVideo
return GeneratedVideo(
id=vid,
project_id=project_id,
generation_task_id="task1",
name=f"video-{vid}",
file_url=f"https://example.com/{vid}.mp4",
file_size=1000,
duration=duration,
width=1920,
height=1080,
fps=25.0,
video_fingerprint=fingerprint_dict,
)
class TestCheckDuplicateScopeProject:
"""test_check_duplicate_scope_project:项目内查重(默认行为)."""
def test_default_scope_queries_by_project(self):
from video_processing.dedup import VideoDeduplicator
deduplicator = VideoDeduplicator()
fingerprint = _make_fingerprint(md5="unique_md5")
session = MagicMock()
with patch("video_processing.dedup.SQLAlchemyGeneratedVideoRepository") as MockRepo:
mock_repo = MockRepo.return_value
mock_repo.list_by_project.return_value = []
result = deduplicator.check_duplicate(fingerprint, "proj1", session)
mock_repo.list_by_project.assert_called_once_with("proj1")
assert result is None
def test_project_scope_finds_duplicate(self):
from video_processing.dedup import VideoDeduplicator
deduplicator = VideoDeduplicator()
fingerprint = _make_fingerprint(md5="same_md5")
session = MagicMock()
existing = _make_video("vid2", {"md5": "same_md5", "keyframe_phashes": ["aa"]})
with patch("video_processing.dedup.SQLAlchemyGeneratedVideoRepository") as MockRepo:
mock_repo = MockRepo.return_value
mock_repo.list_by_project.return_value = [existing]
result = deduplicator.check_duplicate(fingerprint, "proj1", session)
assert result is not None
assert result["duplicate"] is True
assert result["duplicate_of"] == "vid2"
class TestCheckDuplicateScopeUser:
"""test_check_duplicate_scope_user:跨项目查重."""
def test_user_scope_queries_by_user(self):
from video_processing.dedup import VideoDeduplicator
deduplicator = VideoDeduplicator()
fingerprint = _make_fingerprint(md5="unique_md5")
session = MagicMock()
with patch("video_processing.dedup.SQLAlchemyGeneratedVideoRepository") as MockRepo:
mock_repo = MockRepo.return_value
mock_repo.list_by_user.return_value = []
result = deduplicator.check_duplicate(
fingerprint,
"proj1",
session,
scope="user",
user_id="user_123",
)
mock_repo.list_by_user.assert_called_once()
assert result is None
def test_user_scope_finds_cross_project_duplicate(self):
from video_processing.dedup import VideoDeduplicator
deduplicator = VideoDeduplicator()
fingerprint = _make_fingerprint(md5="cross_proj_md5")
session = MagicMock()
# Existing video from a different project
existing = _make_video("vid_other", {"md5": "cross_proj_md5"}, project_id="proj_other")
with patch("video_processing.dedup.SQLAlchemyGeneratedVideoRepository") as MockRepo:
mock_repo = MockRepo.return_value
mock_repo.list_by_user.return_value = [existing]
result = deduplicator.check_duplicate(
fingerprint,
"proj1",
session,
scope="user",
user_id="user_123",
)
assert result is not None
assert result["duplicate"] is True
assert result["duplicate_of"] == "vid_other"
class TestDurationPrefilter:
"""test_duration_prefilter:时长 ±15% 过滤."""
def test_duration_prefilter_passes_correct_range(self):
from video_processing.dedup import VideoDeduplicator
deduplicator = VideoDeduplicator()
fingerprint = _make_fingerprint(duration_ms=30000) # 30s video
session = MagicMock()
with patch("video_processing.dedup.SQLAlchemyGeneratedVideoRepository") as MockRepo:
mock_repo = MockRepo.return_value
mock_repo.list_by_user.return_value = []
deduplicator.check_duplicate(
fingerprint,
"proj1",
session,
scope="user",
user_id="user1",
duration_sec=30.0,
)
# Should pass duration_min=25.5, duration_max=34.5 (30 ± 15%)
call_args = mock_repo.list_by_user.call_args
assert call_args[1]["duration_min"] == pytest.approx(25.5, abs=0.1)
assert call_args[1]["duration_max"] == pytest.approx(34.5, abs=0.1)
def test_no_duration_prefilter_when_zero(self):
from video_processing.dedup import VideoDeduplicator
deduplicator = VideoDeduplicator()
fingerprint = _make_fingerprint()
session = MagicMock()
with patch("video_processing.dedup.SQLAlchemyGeneratedVideoRepository") as MockRepo:
mock_repo = MockRepo.return_value
mock_repo.list_by_user.return_value = []
deduplicator.check_duplicate(
fingerprint,
"proj1",
session,
scope="user",
user_id="user1",
duration_sec=0,
)
call_args = mock_repo.list_by_user.call_args
assert call_args[1]["duration_min"] == 0
assert call_args[1]["duration_max"] == 0
class TestComputeDuplicateRateFormula:
"""test_compute_duplicate_rate_formula:验证 0.4 * frame_match_rate + 0.6 * temporal_coverage_rate."""
def test_formula_with_matching_frames(self):
from video_processing.dedup import VideoDeduplicator
deduplicator = VideoDeduplicator()
# 10 frames, all identical to existing → frame_match_rate = 1.0
phashes = ["aa00aa00aa00aa00"] * 10
fingerprint = _make_fingerprint(md5="new", phashes=phashes, duration_ms=20000)
session = MagicMock()
existing = _make_video(
"vid2",
{"md5": "other", "keyframe_phashes": ["aa00aa00aa00aa00"] * 5},
)
with patch("video_processing.dedup.SQLAlchemyGeneratedVideoRepository") as MockRepo:
mock_repo = MockRepo.return_value
mock_repo.list_by_project.return_value = [existing]
deduplicator._get_existing_chunks = MagicMock(return_value=[])
rate = deduplicator.compute_duplicate_rate(fingerprint, "proj1", "vid1", session)
# frame_match_rate=1.0, temporal_coverage depends on segments
# duplicate_rate = (1.0 * 0.4 + temporal_coverage * 0.6) * 100
assert rate["duplicate_rate"] >= 40.0 # At minimum, frame_match contributes 40%
def test_no_match_returns_zero(self):
from video_processing.dedup import VideoDeduplicator
deduplicator = VideoDeduplicator()
# Completely different phashes
fingerprint = _make_fingerprint(md5="new", phashes=["ff00ff00ff00ff00"])
session = MagicMock()
existing = _make_video(
"vid2",
{"md5": "other", "keyframe_phashes": ["00ff00ff00ff00ff"]},
)
with patch("video_processing.dedup.SQLAlchemyGeneratedVideoRepository") as MockRepo:
mock_repo = MockRepo.return_value
mock_repo.list_by_project.return_value = [existing]
deduplicator._get_existing_chunks = MagicMock(return_value=[])
rate = deduplicator.compute_duplicate_rate(fingerprint, "proj1", "vid1", session)
# Very different phashes, match_ratio < 0.3 → skipped
assert rate["duplicate_rate"] == 0.0
class TestComputeDuplicateRateReturnDict:
"""test_compute_duplicate_rate_return_dict:验证返回 dict 含三个字段."""
def test_return_structure(self):
from video_processing.dedup import VideoDeduplicator
deduplicator = VideoDeduplicator()
fingerprint = _make_fingerprint()
session = MagicMock()
with patch("video_processing.dedup.SQLAlchemyGeneratedVideoRepository") as MockRepo:
mock_repo = MockRepo.return_value
mock_repo.list_by_project.return_value = []
result = deduplicator.compute_duplicate_rate(fingerprint, "proj1", "vid1", session)
assert isinstance(result, dict)
assert set(result.keys()) == {"duplicate_rate", "visual_similarity", "match_count"}
assert isinstance(result["duplicate_rate"], float)
assert isinstance(result["visual_similarity"], float)
assert isinstance(result["match_count"], int)
assert 0 <= result["duplicate_rate"] <= 100
assert 0 <= result["visual_similarity"] <= 1
class TestBackwardCompat:
"""test_backward_compat:不传 scope 时行为不变."""
def test_default_scope_is_project(self):
from video_processing.dedup import VideoDeduplicator
deduplicator = VideoDeduplicator()
fingerprint = _make_fingerprint()
session = MagicMock()
with patch("video_processing.dedup.SQLAlchemyGeneratedVideoRepository") as MockRepo:
mock_repo = MockRepo.return_value
mock_repo.list_by_project.return_value = []
# Call without scope parameter
result = deduplicator.compute_duplicate_rate(fingerprint, "proj1", "vid1", session)
# Should use list_by_project (not list_by_user)
mock_repo.list_by_project.assert_called_once_with("proj1")
mock_repo.list_by_user.assert_not_called()
assert result["duplicate_rate"] == 0.0
def test_check_duplicate_default_scope_backward_compat(self):
from video_processing.dedup import VideoDeduplicator
deduplicator = VideoDeduplicator()
fingerprint = _make_fingerprint()
session = MagicMock()
with patch("video_processing.dedup.SQLAlchemyGeneratedVideoRepository") as MockRepo:
mock_repo = MockRepo.return_value
mock_repo.list_by_project.return_value = []
result = deduplicator.check_duplicate(fingerprint, "proj1", session)
mock_repo.list_by_project.assert_called_once_with("proj1")
assert result is None
class TestListByUserRepository:
"""直接测试 generated_video_repository.list_by_user() 的真实实现,覆盖 diff 代码行。"""
def _make_repo(self):
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from packages.adapters.sqlalchemy_impl.generated_video_repository import SQLAlchemyGeneratedVideoRepository
from packages.adapters.sqlalchemy_impl.models import Base, GeneratedVideoModel
engine = create_engine("sqlite:///:memory:")
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
session = Session()
repo = SQLAlchemyGeneratedVideoRepository(session)
return repo, session
def _insert_video(self, session, video_id, user_id, project_id, duration, **kw):
from packages.adapters.sqlalchemy_impl.models import GeneratedVideoModel
row = GeneratedVideoModel(
id=video_id,
user_id=user_id,
project_id=project_id,
generation_task_id=f"task-{video_id[:8]}",
name=f"video-{video_id[:8]}.mp4",
file_url=f"https://example.com/{video_id}.mp4",
file_size=1024,
duration=duration,
width=1280,
height=720,
fps=25.0,
status="completed",
)
session.add(row)
session.flush()
return row
def test_list_by_user_returns_cross_project_videos(self):
"""list_by_user 返回该用户所有项目的视频。"""
repo, session = self._make_repo()
self._insert_video(session, "v1", "user-a", "proj-1", 30.0)
self._insert_video(session, "v2", "user-a", "proj-2", 45.0)
self._insert_video(session, "v3", "user-b", "proj-1", 20.0)
results = repo.list_by_user("user-a")
assert len(results) == 2
ids = {r.id for r in results}
assert ids == {"v1", "v2"}
session.close()
def test_list_by_user_with_duration_filter(self):
"""list_by_user 支持 duration_min/duration_max 过滤。"""
repo, session = self._make_repo()
self._insert_video(session, "v1", "user-a", "proj-1", 10.0)
self._insert_video(session, "v2", "user-a", "proj-1", 30.0)
self._insert_video(session, "v3", "user-a", "proj-1", 60.0)
results = repo.list_by_user("user-a", duration_min=20.0, duration_max=50.0)
assert len(results) == 1
assert results[0].id == "v2"
session.close()
def test_list_by_user_empty_result(self):
"""list_by_user 无匹配时返回空列表。"""
repo, session = self._make_repo()
self._insert_video(session, "v1", "user-a", "proj-1", 30.0)
results = repo.list_by_user("user-nonexistent")
assert results == []
session.close()
+282
View File
@@ -0,0 +1,282 @@
"""分片指纹存储单元测试 — Issue #1657.
覆盖:
- 分片策略:60秒视频 → 30片,120秒视频 → 24片
- VideoFingerprint.to_chunk_models() 输出正确
- _save_fingerprint_chunks 幂等性(已有数据跳过)
- to_dict() 向后兼容
"""
from __future__ import annotations
import sys
from unittest.mock import MagicMock
def _mock_module(**attrs):
"""Create a mock module with __spec__ to avoid AttributeError."""
m = MagicMock()
m.__spec__ = None
for k, v in attrs.items():
setattr(m, k, v)
return m
# ── Module-level setup: mock deps, import dedup, then restore sys.modules ──
_SAVED_MODULES_KEYS = set(sys.modules.keys())
_SAVED_MODULES_VALUES = {
k: sys.modules.get(k)
for k in [
"cv2",
"celery",
"sqlalchemy",
"sqlalchemy.orm",
"sqlalchemy.engine",
"sqlalchemy.ext",
"sqlalchemy.ext.declarative",
"worker_app.db",
"worker_app.celery_app",
"worker_app.core.config",
"packages.adapters.sqlalchemy_impl.session",
"packages.adapters.sqlalchemy_impl.generated_video_repository",
"packages.adapters.sqlalchemy_impl.models",
"packages.shared.config",
"packages.shared.storage",
]
}
# Set up mocks
sys.modules["cv2"] = _mock_module()
_mock_celery = MagicMock()
_mock_celery.Task = MagicMock
_mock_celery.Celery = MagicMock
_mock_celery.__spec__ = None
sys.modules["celery"] = _mock_celery
_mock_sqla = MagicMock()
_mock_sqla.__path__ = []
_mock_sqla.__spec__ = None
sys.modules["sqlalchemy"] = _mock_sqla
_mock_sqla_orm = MagicMock()
_mock_sqla_orm.__path__ = []
_mock_sqla_orm.__spec__ = None
_mock_sqla_orm.Session = MagicMock
sys.modules["sqlalchemy.orm"] = _mock_sqla_orm
sys.modules["sqlalchemy.engine"] = _mock_module()
sys.modules["sqlalchemy.ext"] = _mock_module()
sys.modules["sqlalchemy.ext.declarative"] = _mock_module()
sys.modules["worker_app.db"] = _mock_module(SessionLocal=MagicMock())
sys.modules["worker_app.celery_app"] = _mock_module(celery_app=MagicMock())
sys.modules["worker_app.core.config"] = _mock_module(get_settings=MagicMock(return_value=MagicMock()))
sys.modules["packages.adapters.sqlalchemy_impl.session"] = _mock_module(
Base=MagicMock(),
build_engine=MagicMock(),
build_session_factory=MagicMock(),
ensure_database_exists=MagicMock(),
initialize_database=MagicMock(),
)
sys.modules["packages.adapters.sqlalchemy_impl.generated_video_repository"] = _mock_module()
# Mock VideoFingerprintChunkModel with class-level column attributes
class _FakeChunkModel:
video_id = MagicMock()
project_id = MagicMock()
user_id = MagicMock()
start_time_ms = MagicMock()
end_time_ms = MagicMock()
phash_binary = MagicMock()
color_histogram = MagicMock()
frame_count = MagicMock()
created_at = MagicMock()
def __init__(self, **kwargs):
for k, v in kwargs.items():
setattr(self, k, v)
sys.modules["packages.adapters.sqlalchemy_impl.models"] = _mock_module(
VideoFingerprintChunkModel=_FakeChunkModel,
)
sys.modules["packages.shared.config"] = _mock_module(get_shared_settings=MagicMock(return_value=MagicMock()))
sys.modules["packages.shared.storage"] = _mock_module()
# Import dedup while mocks are active
from video_processing.dedup import ( # noqa: E402
FingerprintChunk,
VideoFingerprint,
_save_fingerprint_chunks,
)
# ── Restore sys.modules immediately after import ──
for _key in list(sys.modules.keys()):
if _key not in _SAVED_MODULES_KEYS:
del sys.modules[_key]
for _key, _value in _SAVED_MODULES_VALUES.items():
if _value is not None:
sys.modules[_key] = _value
elif _key in sys.modules:
del sys.modules[_key]
del _SAVED_MODULES_KEYS, _SAVED_MODULES_VALUES, _key, _value
class TestVideoFingerprintToChunkModels:
"""测试 VideoFingerprint.to_chunk_models() 输出。"""
def test_to_chunk_models_output(self):
"""to_chunk_models 返回正确的 Model 列表。"""
fp = VideoFingerprint(
md5="abc123",
keyframe_phashes=["a1b2", "c3d4"],
color_histograms=[[0.1] * 96, [0.2] * 96],
duration=10.0,
resolution=(1920, 1080),
chunks=[
FingerprintChunk(start_time_ms=0, end_time_ms=2000, phash_binary="a1b2", color_histogram=[0.1] * 96),
FingerprintChunk(start_time_ms=2000, end_time_ms=4000, phash_binary="c3d4", color_histogram=[0.2] * 96),
],
)
models = fp.to_chunk_models(video_id="v1", project_id="p1", user_id="u1")
assert len(models) == 2
assert models[0].video_id == "v1"
assert models[0].project_id == "p1"
assert models[0].user_id == "u1"
assert models[0].start_time_ms == 0
assert models[0].end_time_ms == 2000
assert models[0].phash_binary == "a1b2"
assert models[1].start_time_ms == 2000
assert models[1].end_time_ms == 4000
assert models[1].phash_binary == "c3d4"
def test_to_chunk_models_empty_chunks(self):
"""空 chunks 列表返回空 Model 列表。"""
fp = VideoFingerprint(
md5="abc",
keyframe_phashes=[],
color_histograms=[],
duration=0,
resolution=(0, 0),
chunks=[],
)
models = fp.to_chunk_models(video_id="v1", project_id="p1")
assert models == []
class TestSaveFingerprintChunksIdempotent:
"""测试 _save_fingerprint_chunks 幂等性。"""
def test_save_skips_existing(self):
"""已有分片数据时跳过写入。"""
fp = VideoFingerprint(
md5="abc",
keyframe_phashes=["a1b2"],
color_histograms=[[0.1] * 96],
duration=5.0,
resolution=(1920, 1080),
chunks=[
FingerprintChunk(start_time_ms=0, end_time_ms=2000, phash_binary="a1b2", color_histogram=[0.1] * 96),
],
)
session = MagicMock()
# Mock: 已有 1 条分片数据
session.query.return_value.filter.return_value.count.return_value = 1
_save_fingerprint_chunks(fp, video_id="v1", project_id="p1", user_id="u1", session=session)
# bulk_save_objects 不应被调用
session.bulk_save_objects.assert_not_called()
def test_save_writes_new(self):
"""无分片数据时写入。"""
fp = VideoFingerprint(
md5="abc",
keyframe_phashes=["a1b2"],
color_histograms=[[0.1] * 96],
duration=5.0,
resolution=(1920, 1080),
chunks=[
FingerprintChunk(start_time_ms=0, end_time_ms=2000, phash_binary="a1b2", color_histogram=[0.1] * 96),
],
)
session = MagicMock()
# Mock: 无分片数据
session.query.return_value.filter.return_value.count.return_value = 0
_save_fingerprint_chunks(fp, video_id="v1", project_id="p1", user_id="u1", session=session)
# bulk_save_objects 应被调用一次
session.bulk_save_objects.assert_called_once()
saved_models = session.bulk_save_objects.call_args[0][0]
assert len(saved_models) == 1
assert saved_models[0].video_id == "v1"
assert saved_models[0].phash_binary == "a1b2"
def test_save_skips_no_chunks(self):
"""指纹无 chunks 时跳过。"""
fp = VideoFingerprint(
md5="abc",
keyframe_phashes=[],
color_histograms=[],
duration=0,
resolution=(0, 0),
chunks=[],
)
session = MagicMock()
session.query.return_value.filter.return_value.count.return_value = 0
_save_fingerprint_chunks(fp, video_id="v1", project_id="p1", user_id="u1", session=session)
# bulk_save_objects 不应被调用
session.bulk_save_objects.assert_not_called()
class TestFingerprintToDictBackwardCompat:
"""测试 to_dict() 向后兼容性。"""
def test_to_dict_includes_chunks(self):
"""to_dict() 包含 chunks 字段。"""
fp = VideoFingerprint(
md5="abc123",
keyframe_phashes=["a1b2"],
color_histograms=[[0.1] * 96],
duration=5.0,
resolution=(1920, 1080),
chunks=[
FingerprintChunk(start_time_ms=0, end_time_ms=2000, phash_binary="a1b2", color_histogram=[0.1] * 96),
],
)
d = fp.to_dict()
assert "chunks" in d
assert len(d["chunks"]) == 1
assert d["chunks"][0]["start_time_ms"] == 0
assert d["chunks"][0]["end_time_ms"] == 2000
assert d["chunks"][0]["phash_binary"] == "a1b2"
def test_to_dict_preserves_legacy_fields(self):
"""to_dict() 保留 keyframe_phashes 和 color_histograms 字段。"""
fp = VideoFingerprint(
md5="abc",
keyframe_phashes=["a1b2", "c3d4"],
color_histograms=[[0.1] * 96, [0.2] * 96],
duration=10.0,
resolution=(1920, 1080),
)
d = fp.to_dict()
assert "keyframe_phashes" in d
assert "color_histograms" in d
assert len(d["keyframe_phashes"]) == 2
assert len(d["color_histograms"]) == 2
@@ -359,6 +359,11 @@ class TestThumbnailInDedupHelpers:
mock_dedup.compute_fingerprint.return_value = MagicMock(to_dict=lambda: {})
mock_dedup.check_duplicate.return_value = None
mock_dedup.check_batch_duplicate.return_value = None
mock_dedup.compute_duplicate_rate.return_value = {
"duplicate_rate": 0.0,
"visual_similarity": 0.0,
"match_count": 0,
}
result = create_video_record_and_dedup(
generation_task_id="task-thumb-reuse",
@@ -401,6 +406,11 @@ class TestThumbnailInDedupHelpers:
mock_dedup.compute_fingerprint.return_value = MagicMock(to_dict=lambda: {})
mock_dedup.check_duplicate.return_value = None
mock_dedup.check_batch_duplicate.return_value = None
mock_dedup.compute_duplicate_rate.return_value = {
"duplicate_rate": 0.0,
"visual_similarity": 0.0,
"match_count": 0,
}
result = create_video_record_and_dedup(
generation_task_id="task-thumb-gen",
@@ -443,6 +453,11 @@ class TestThumbnailInDedupHelpers:
mock_dedup.compute_fingerprint.return_value = MagicMock(to_dict=lambda: {})
mock_dedup.check_duplicate.return_value = None
mock_dedup.check_batch_duplicate.return_value = None
mock_dedup.compute_duplicate_rate.return_value = {
"duplicate_rate": 0.0,
"visual_similarity": 0.0,
"match_count": 0,
}
result = create_video_record_and_dedup(
generation_task_id="task-thumb-fail",