fix: 查重流程全面核实修复 — 两阶段持久化 + 重新计算查重API (#1664) #1684

Merged
auto-approve-bot merged 2 commits from fix/dedup-flow-verification-1664 into develop 2026-09-04 13:00:12 +08:00

2 Commits

Author SHA1 Message Date
CI Bot 61e9262c0a style: auto-format with black + isort + prettier [skip ci-format-check]
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 6s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 6s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 31s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 31s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 54s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 1m37s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m41s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 1m43s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 2m9s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m43s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 4m23s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 1s
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 2m25s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 10s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 23s
AI Code Review / AI Code Review (pull_request) Successful in 6m17s
2026-09-04 04:41:03 +00:00
saas-backend-agent aee29a45a4 fix: 查重流程全面核实修复 — 两阶段持久化 + 重新计算查重API (#1664)
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 24s
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 26s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 1m46s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m53s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 1m52s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 2m5s
CI/CD Pipeline / Validate - Style (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Security (pull_request) Has been cancelled
CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / CI Gate (pull_request) Has been cancelled
AI Code Review / AI Code Review (pull_request) Has been cancelled
PR Automation / Auto Approve on CI Green (pull_request) Has been cancelled
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been cancelled
问题根因:
1. dedup_helpers.py 先 commit 视频记录再算查重率 — 如果指纹计算
   或查重率计算中途异常,视频已入库但 duplicate_rate=None,且无重试
2. 没有 API 可以触发已有视频重新计算查重率
3. 前端 visual_similarity 显示 bug(后端返 0~1,前端直接 toFixed%)

修复:
- dedup_helpers.py 重构为两阶段持久化:先计算全部指纹/查重数据
  (内存),再一次性 create + commit,消除中间态
- videos.py 新增 POST /videos/recompute-dedup 端点:对缺少查重数据
  的视频触发异步 check_duplicate 任务重新计算
- 更新 test_dedup_helpers_user_id 适配新持久化模式

新增测试:
- test_dedup_two_phase_commit.py: 5 个测试覆盖两阶段提交场景
- test_recompute_dedup_api.py: 5 个测试覆盖重新计算 API
2026-09-04 12:37:52 +08:00