xiaoxia/xiaoxia-saas
1
0
Fork 0
Code Issues 11 Pull Requests 3 Actions 3 Packages Projects Releases 7 Wiki Activity

fix(dedup): recompute任务从file_url派生OSS下载key，修复重算404 (#1702) #1705

Merged
auto-approve-bot merged 1 commits from fix/dedup-recompute-download-key-1702 into develop 2026-09-05 08:36:21 +08:00
Conversation 4 Commits 1 Files Changed 2
+39 -3
xiaoxia commented 2026-09-05 08:10:04 +08:00
Owner
Copy Link
Copy Source

Issue #1702 连带修复：recompute-dedup 任务 OSS 下载 key 错误（404）

PR #1703 合并后在 staging 验证 recompute 时发现：check_duplicate_task（recompute-dedup 的实际执行入口）从 OSS 重新下载成片时硬编码了

projects/{pid}/generated/{vid}/{vid}.mp4

但成片真实 OSS key 是生成上传时的（generation.py _upload_and_record）：

generated/projects/{pid}/tasks/{tid}/rendered_<hash>.mp4

硬编码 key 从不存在 → 所有 recompute 任务下载 404、重试 3 次后失败，查重数据永远无法重算。正常生成流程用本地渲染文件算指纹（dedup_helpers.create_video_record_and_dedup 传 video_path），不经此任务，所以一直没暴露。

修复

  • 下载 key 优先从 video.file_url 派生（download_file 内部会 normalize 完整 URL）
  • 旧 key 模式仅作 file_url 缺失时的回退并打 warning
  • 补源码级回归单测

Refs #1702

## Issue #1702 连带修复：recompute-dedup 任务 OSS 下载 key 错误（404） PR #1703 合并后在 staging 验证 recompute 时发现：`check_duplicate_task`（recompute-dedup 的实际执行入口）从 OSS 重新下载成片时硬编码了 `projects/{pid}/generated/{vid}/{vid}.mp4` 但成片真实 OSS key 是生成上传时的（`generation.py` `_upload_and_record`）： `generated/projects/{pid}/tasks/{tid}/rendered_<hash>.mp4` 硬编码 key 从不存在 → 所有 recompute 任务下载 404、重试 3 次后失败，查重数据永远无法重算。正常生成流程用本地渲染文件算指纹（`dedup_helpers.create_video_record_and_dedup` 传 `video_path`），不经此任务，所以一直没暴露。 ## 修复 - 下载 key 优先从 `video.file_url` 派生（`download_file` 内部会 normalize 完整 URL） - 旧 key 模式仅作 file_url 缺失时的回退并打 warning - 补源码级回归单测 Refs #1702
gitea-actions bot commented 2026-09-05 08:12:21 +08:00
Copy Link
Copy Source

🚀 预览环境已部署

项目 详情
PR号 #1705
预览链接 https://pr-1705.preview.xiaoxiajianji.com
API环境 staging

💡 预览环境使用 staging API 数据，请勿在预览环境中操作重要数据。

🔄 每次提交新代码后预览环境会自动更新。

🗑️ PR 关闭或合并后，预览环境会自动清理。

🚀 **预览环境已部署** | 项目 | 详情 | |------|------| | PR号 | #1705 | | 预览链接 | [https://pr-1705.preview.xiaoxiajianji.com](https://pr-1705.preview.xiaoxiajianji.com) | | API环境 | staging | > 💡 预览环境使用 staging API 数据，请勿在预览环境中操作重要数据。 > > 🔄 每次提交新代码后预览环境会自动更新。 > > 🗑️ PR 关闭或合并后，预览环境会自动清理。
auto-approve-bot commented 2026-09-05 08:17:49 +08:00
Collaborator
Copy Link
Copy Source

Auto merge skipped after multiple 405 errors: PR may have conflicts or unresolved checks. Please review manually. This is not a CI failure.

Auto merge skipped after multiple 405 errors: PR may have conflicts or unresolved checks. Please review manually. This is not a CI failure.
xiaoxia added 1 commit 2026-09-05 08:31:14 +08:00
fix(dedup): recompute任务从file_url派生OSS下载key (Issue #1702连带修复)
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
Details
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2s
Details
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
Details
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
Details
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
Details
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
Details
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
Details
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
Details
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
Details
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 35s
Details
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 34s
Details
CI/CD Pipeline / Unit Tests (pull_request) Successful in 1m46s
Details
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m56s
Details
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m54s
Details
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 1m57s
Details
CI/CD Pipeline / Validate - Style (pull_request) Successful in 2m21s
Details
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m57s
Details
AI Code Review / AI Code Review (pull_request) Successful in 3m42s
Details
CI/CD Pipeline / Validate - Security (pull_request) Successful in 4m20s
Details
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
Details
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
Details
CI/CD Pipeline / CI Gate (pull_request) Successful in 3s
Details
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
Details
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 2m8s
Details
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 9s
Details
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 18s
Details
46325f55ca 
check_duplicate_task(recompute-dedup的执行入口)硬编码下载
projects/{pid}/generated/{vid}/{vid}.mp4，但成片真实OSS key是生成时的
generated/projects/{pid}/tasks/{tid}/rendered_*.mp4(generation.py
_upload_and_record),导致所有recompute任务下载404、查重数据无法重算。
正常生成流程用本地渲染文件算指纹不受影响,仅recompute路径中招。
改为优先从video.file_url解析下载key,旧key模式仅作回退。
补源码级回归单测。
xiaoxia force-pushed fix/dedup-recompute-download-key-1702 from 8ba7108edd to 46325f55ca 2026-09-05 08:31:14 +08:00 Compare
auto-approve-bot commented 2026-09-05 08:34:57 +08:00
Collaborator
Copy Link
Copy Source

代码审查结果 - PR #1705

⚠️ 问题（0个需要修改）

无

💡 建议（1个可选）

  1. tests/unit/test_dedup_1702_zero_rate_fix.py 第373行：测试方法使用了 inspect.getsource 进行静态字符串匹配来验证逻辑。这种方式非常脆弱，一旦代码格式化、变量重命名或逻辑重构（即使逻辑正确），测试就会失败。建议改为单元测试行为：Mock video 对象和 storage_service，调用 check_duplicate_task，然后验证 storage_service.download_file 是否被传入了正确的参数（即 video.file_url 的值）。

✅ 格式检查通过 | ✅ 逻辑审查通过 | ✅ 性能无明显问题

🤖 由 AI 代码审查机器人自动生成 | 2026-09-05 00:34:57 | 模型:

## 代码审查结果 - PR #1705 ### ⚠️ 问题（0个需要修改） 无 ### 💡 建议（1个可选） 1. **tests/unit/test_dedup_1702_zero_rate_fix.py 第373行**：测试方法使用了 `inspect.getsource` 进行静态字符串匹配来验证逻辑。这种方式非常脆弱，一旦代码格式化、变量重命名或逻辑重构（即使逻辑正确），测试就会失败。建议改为单元测试行为：Mock `video` 对象和 `storage_service`，调用 `check_duplicate_task`，然后验证 `storage_service.download_file` 是否被传入了正确的参数（即 `video.file_url` 的值）。 --- ✅ 格式检查通过 | ✅ 逻辑审查通过 | ✅ 性能无明显问题 --- <sub>🤖 由 AI 代码审查机器人自动生成 | 2026-09-05 00:34:57 | 模型: </sub> <!-- AI_CODE_REVIEW_AUTO_COMMENT -->
auto-approve-bot merged commit 7a4aa27f71 into develop 2026-09-05 08:36:21 +08:00
auto-approve-bot deleted branch fix/dedup-recompute-download-key-1702 2026-09-05 08:36:22 +08:00
gitea-actions bot commented 2026-09-05 08:36:39 +08:00
Copy Link
Copy Source

🗑️ 预览环境已清理

PR #1705 已关闭或合并，对应的预览环境已被清理。

如有需要，可以重新打开 PR 来重新生成预览环境。

🗑️ **预览环境已清理** PR #1705 已关闭或合并，对应的预览环境已被清理。 > 如有需要，可以重新打开 PR 来重新生成预览环境。
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
P0
阻塞性，必须立即处理
 P1
高优先级
 P2
中优先级
 P3
低优先级/后考虑
 architecture
架构设计
 backend
后端相关
 blocked
阻塞中
 bug
Bug修复
 ci-bug
CI本身的bug
 ci-optimization
CI优化类任务
 ci-pipeline
CI流水线/workflow相关
 cleanup
代码清理
 devops
运维/部署/CI
 documentation
文档相关
 feature
新功能
 frontend
前端相关
 question
需要讨论确认
 ready
PR CI通过，可合并
 refactor
代码重构
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
auto-approve-bot xiaoxia
No Assignees
3 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: xiaoxia/xiaoxia-saas#1705
Delete Branch "fix/dedup-recompute-download-key-1702"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?