feat(worker): 手动查重 worker task + visual_similarity/match_count 字段 #1661

- 新增 Celery task worker.process_duplication_check:
  OSS 下载上传视频 → 动态抽帧指纹 → 跨项目与用户所有成片比对
  (compute_duplicate_rate + find_duplicate_segments) → 更新 DuplicationRecord
- 状态流转 pending → processing → completed/failed,失败重试 3 次间隔 60s,
  重试耗尽标记 failed,临时文件 finally 清理
- 片段映射:worker 分片 DuplicateSegment(ms/汉明距离) → 领域 DuplicateSegment(秒/0-100 相似度)
- DuplicationRecord 实体新增 visual_similarity/match_count(可选 None,向后兼容),
  mark_completed()/reset_for_retry() 同步
- migration 065: duplication_records 表加两列
- repository/model/schemas/routes 读写映射与响应透传
- 上传接口与重试接口 enqueue task(此前记录一直 pending 的根因)
- 11 个新单测:成功/跳过/未找到/重试/失败标记/临时文件清理/片段映射/schema
This commit is contained in:
saas-backend-agent
2026-09-04 01:11:04 +08:00
parent df164ddf75
commit c6af6dd072
9 changed files with 638 additions and 1 deletions
@@ -417,6 +417,9 @@ class DuplicationRecordModel(Base):
status = Column(String(20), nullable=False, default="pending", index=True)
duplicate_rate = Column(Float, nullable=True)
duplicate_count = Column(Integer, nullable=False, default=0)
# #1661 手动查重:视觉相似度(0~1)/ 匹配视频数
visual_similarity = Column(Float, nullable=True)
match_count = Column(Integer, nullable=True)
video_fingerprint = Column(Text, nullable=True)
error_message = Column(Text, nullable=False, default="")
created_at = Column(DateTime, nullable=False, default=lambda: datetime.now(timezone.utc))