xiaoxia/xiaoxia-saas

feat(dedup): 动态抽帧 + 滑动窗口时序匹配 (#1659) #1673

Merged

xiaoxia merged 3 commits from feature/1659-dynamic-keyframe-sliding-window into develop

2026-09-03 23:16:51 +08:00

Author	SHA1	Message	Date
xiaoxia	db9ee89ffa	fix(dedup): ruff lint 修复 — 未使用变量 + zip strict + 冗余 import (#1659 ) CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped Details CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 3s Details CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped Details CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 4s Details CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped Details CI/CD Pipeline / Validate - Style (pull_request) Successful in 1m20s Details CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped Details CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped Details CI/CD Pipeline / PR Build API Image (pull_request) Successful in 10s Details CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped Details Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m26s Details CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m0s Details CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped Details CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped Details CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 12s Details CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped Details AI Code Review / AI Code Review (pull_request) Failing after 2m34s Details CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped Details CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped Details CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped Details CI/CD Pipeline / Unit Tests (pull_request) Successful in 1m17s Details PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m48s Details CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 4m4s Details PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 4m26s Details ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 5s Details Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 1m17s Details CI/CD Pipeline / Validate - Security (pull_request) Successful in 22m37s Details CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped Details CI/CD Pipeline / Deploy Production (pull_request) Has been skipped Details CI/CD Pipeline / CI Gate (pull_request) Successful in 4s Details CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped Details CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped Details	2026-09-03 23:09:19 +08:00
CI Bot	a0d4f6e111	style: auto-format with black + isort + prettier [skip ci-format-check] CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped Details CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2s Details CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped Details CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s Details CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped Details CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped Details CI/CD Pipeline / PR Build API Image (pull_request) Successful in 22s Details CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped Details CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped Details CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped Details CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 10s Details CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped Details CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped Details CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped Details CI/CD Pipeline / Unit Tests (pull_request) Successful in 1m20s Details CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m27s Details CI/CD Pipeline / Validate - Style (pull_request) Failing after 1m33s Details AI Code Review / AI Code Review (pull_request) Failing after 2m11s Details Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m19s Details PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m43s Details CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 4m35s Details CI/CD Pipeline / Validate - Security (pull_request) Has been cancelled Details CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled Details CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled Details CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled Details CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled Details CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled Details CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled Details CI/CD Pipeline / CI Gate (pull_request) Has been cancelled Details PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 2m29s Details	2026-09-03 15:04:14 +00:00
xiaoxia	fac80b1f77	feat(dedup): 动态抽帧 + 滑动窗口时序匹配 (#1659 ) CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped Details CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 1s Details CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2s Details CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped Details CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped Details CI/CD Pipeline / PR Build API Image (pull_request) Successful in 22s Details CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped Details CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped Details CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped Details CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped Details CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 9s Details CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped Details CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped Details CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped Details CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped Details CI/CD Pipeline / Unit Tests (pull_request) Successful in 1m22s Details CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m25s Details AI Code Review / AI Code Review (pull_request) Failing after 1m50s Details Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m39s Details PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m44s Details CI/CD Pipeline / Validate - Style (pull_request) Has been cancelled Details CI/CD Pipeline / Validate - Security (pull_request) Has been cancelled Details CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Has been cancelled Details CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled Details CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled Details CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled Details CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled Details CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled Details CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled Details CI/CD Pipeline / CI Gate (pull_request) Has been cancelled Details PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been cancelled Details 1. 动态抽帧策略 — detect_keyframe_timestamps() - 降采样到 320x240 逐帧灰度差异检测场景切换 - 最小间隔过滤（保留差异最大的候选帧） - 数量裁剪到 [MIN_KEYFRAMES=5, MAX_KEYFRAMES=30] - 长视频（>3分钟）每 30 秒分段保底 2. 滑动窗口时序匹配 — find_duplicate_segments() - 逐帧最佳匹配 → 连续 run 检测（允许 MAX_GAP=2 间隙） - 最少 MIN_CONSECUTIVE_MATCHES=5 帧才报告 - 返回 DuplicateSegment（query/target 时间范围 + 平均距离） 3. 查重算法升级 - 均值距离 → 中位数距离（抵抗异常值） - 新增帧匹配比例条件（match_ratio >= 0.7） - Bhattacharyya 系数替代余弦相似度 - pHash + 直方图加权融合（0.7/0.3） - 判定重复后附加 duplicate_segments 字段 4. 删除旧代码 - 移除 SHORT_VIDEO_CHUNK_SEC/LONG_VIDEO_CHUNK_SEC 固定间隔 - 移除 compute_chunk_interval() - 移除 _average_histogram_similarity() 5. 测试 - 新增 test_dedup_v2.py: 34 个测试 - 更新 test_dedup_engine.py/test_duplicate_rate.py/test_dedup_pure.py - 清理 test_fingerprint_chunks.py 中旧常量测试	2026-09-03 23:00:21 +08:00