fix(#1714): 同名兜底去重误杀新视频 + ingest 链路孤儿清理/启动恢复 #1734
Merged
auto-approve-bot
merged 2 commits from 2026-09-06 13:41:28 +08:00
fix/fallback-dedup-false-positive-1714 into develop
2 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 CI Bot
|
e1aff5609d |
style: auto-format with black + isort + prettier [skip ci-format-check]
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 1s
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 28s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 16s
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m25s
AI Code Review / AI Code Review (pull_request) Successful in 1m25s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 1m32s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 1m52s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m48s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 2m15s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m5s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 6m35s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 3s
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 3m52s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 12s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 19s
|
||
|
saas-backend-agent
|
fec656941d |
fix(#1714): 同名兜底去重误杀新视频 + ingest 链路孤儿清理/启动恢复
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 6s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 4s
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 24s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 24s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 1m41s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 1m57s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m57s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m44s
CI/CD Pipeline / Validate - Style (pull_request) Failing after 2m50s
CI/CD Pipeline / Validate - Security (pull_request) Has been cancelled
CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / CI Gate (pull_request) Has been cancelled
AI Code Review / AI Code Review (pull_request) Has been cancelled
PR Automation / Auto Approve on CI Green (pull_request) Has been cancelled
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been cancelled
根因（staging 实证 IMG_2285.MOV file_size=0 卡 processing 持续误杀）：
direct complete 不传 file_size，find_recent_active_by_library_and_name
的大小校验 if file_size>0 不生效，30 分钟内同名视频（iPhone IMG_xxxx.MOV）
即使内容全新也被同名兜底误判重复跳过。
兜底去重收紧（宁可漏判不可误杀）：
- _find_duplicate_asset：file_hash/client_upload_id 非空时不走同名兜底
（hash 已代表内容）；走到兜底必须 file_size>0 且与记录大小严格一致
- find_recent_active_by_library_and_name：file_size=0 直接返回 None，
大小条件改为 SQL 内严格等值匹配
- complete 的 _create_pending_asset 补传 file_size（之前占位记录大小永远 0）
ingest 链路孤儿兜底（此前只有 generation 链路有清理）：
- 新增 packages/application/ingest_orphan_cleanup.py：
- processing>60min / pending>90min 的 ingest_job 标 failed，关联
processing/uploading asset 联动标 error；无 job 关联超 120min 孤儿
占位 asset 也标 error（beat 每 10 分钟巡检）
- worker 启动恢复：processing 超 10 分钟的 job CAS 重置 pending 并
重新派单（Redis SET NX 锁互斥双 worker，旧消息重投由执行前守卫丢弃）
- celery 配置：task_reject_on_worker_lost=True；
broker visibility_timeout=4h（acks_late 下长转码任务不被误重投）
单测：同名不同大小放行 / hash 非空不走兜底 / file_size=0 放行 /
同名同大小 processing 才判重；孤儿清理 9 例、启动恢复 4 例、
仓储严格守卫 6 例、beat 串联 2 例。diff coverage 100%。
前端配合（前端工程师）：completeDirectUpload 请求体补 file_size=file.size。
|