xiaoxia/xiaoxia-saas
1
0
Fork 0
Code Issues 9 Pull Requests 2 Actions Packages Projects Releases 7 Wiki Activity

fix(worker): download_asset支持HTTP URL下载，兼容预签名file_url #487

Merged
auto-approve-bot merged 1 commits from fix/download-asset-http-fallback into develop 2026-07-17 20:46:29 +08:00
Conversation 2 Commits 1 Files Changed 2
+218 -1
xiaoxia commented 2026-07-17 20:42:03 +08:00
Owner
Copy Link
Copy Source

问题

存量数据 file_url 存的是完整预签名 URL（https:// 开头），#485 fallback 到 file_url 后，download_asset 仍然走 oss2 SDK，把完整 URL 当 OSS key 传，下载到的是 XML 错误页。

修复

oss_helpers.download_asset 自动识别输入类型：

  • http(s):// 开头 → 走 requests.get HTTP 下载（支持预签名 URL）
  • 普通路径 → 走 oss2 SDK 下载

HTTP 下载使用流式 + 8MB 分片 + 300s 超时，避免大文件内存溢出。

影响范围

只改了 apps/worker/video_processing/oss_helpers.py 的 download_asset 函数，所有调用方（render_adapter主链路+BGM、legacy路径、generation.py BGM）自动受益。

测试

  • 新增 tests/unit/test_oss_helpers_download.py，10个测试覆盖：
    • HTTP下载成功 / 403失败 / 连接错误 / 流式分片
    • OSS SDK路径正常 / bucket为空 / SDK异常
    • 路由选择（https/http/普通路径）
  • 原有 49个 相关测试全部通过
  • black / isort / ruff 全过
## 问题 存量数据 `file_url` 存的是完整预签名 URL（https:// 开头），#485 fallback 到 file_url 后，`download_asset` 仍然走 oss2 SDK，把完整 URL 当 OSS key 传，下载到的是 XML 错误页。 ## 修复 `oss_helpers.download_asset` 自动识别输入类型： - `http(s)://` 开头 → 走 `requests.get` HTTP 下载（支持预签名 URL） - 普通路径 → 走 oss2 SDK 下载 HTTP 下载使用流式 + 8MB 分片 + 300s 超时，避免大文件内存溢出。 ## 影响范围 只改了 `apps/worker/video_processing/oss_helpers.py` 的 `download_asset` 函数，所有调用方（render_adapter主链路+BGM、legacy路径、generation.py BGM）自动受益。 ## 测试 - 新增 `tests/unit/test_oss_helpers_download.py`，10个测试覆盖： - HTTP下载成功 / 403失败 / 连接错误 / 流式分片 - OSS SDK路径正常 / bucket为空 / SDK异常 - 路由选择（https/http/普通路径） - 原有 49个 相关测试全部通过 - black / isort / ruff 全过
xiaoxia added 1 commit 2026-07-17 20:42:03 +08:00
fix(worker): download_asset支持HTTP URL下载，兼容预签名file_url
CI Build & Deploy Pipeline / Build Staging API Image (pull_request) Has been skipped
Details
CI Build & Deploy Pipeline / Build Staging Web Image (pull_request) Has been skipped
Details
CI Build & Deploy Pipeline / Build Staging Worker Image (pull_request) Has been skipped
Details
CI Build & Deploy Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
Details
CI Build & Deploy Pipeline / Build Production API Image (pull_request) Has been skipped
Details
CI Build & Deploy Pipeline / Build Production Web Image (pull_request) Has been skipped
Details
CI Build & Deploy Pipeline / Build Production Worker Image (pull_request) Has been skipped
Details
CI Build & Deploy Pipeline / Staging E2E Tests (pull_request) Has been skipped
Details
CI Build & Deploy Pipeline / Staging API Integration Tests (pull_request) Has been skipped
Details
CI Build & Deploy Pipeline / Deploy Production (pull_request) Has been skipped
Details
CI Build & Deploy Pipeline / Production Browser E2E (pull_request) Has been skipped
Details
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 20s
Details
AI Code Review / AI Code Review (pull_request) Failing after 34s
Details
CI/CD Pipeline / Unit Tests (pull_request) Successful in 28s
Details
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 54s
Details
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Successful in 2m22s
Details
Auto Approve CI PRs / Auto Approve on CI Green (pull_request) Successful in 2m54s
Details
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m28s
Details
Auto Merge CI PRs / Auto Merge on CI Green + Approved (pull_request) Successful in 4m26s
Details
479be6ade7 
- download_asset自动识别：http(s)://开头走requests HTTP下载，OSS key走oss2 SDK
- HTTP下载支持流式分片（8MB chunk）+ 300s超时，避免大文件OOM
- 修复存量数据file_url为完整预签名URL时，oss2 SDK把URL当key用下载到XML错误页的问题
- 新增10个单元测试覆盖HTTP下载成功/失败/403/超时、OSS SDK路径、路由选择逻辑
auto-approve-bot approved these changes 2026-07-17 20:44:57 +08:00
auto-approve-bot left a comment
Collaborator
Copy Link
Copy Source

CI全绿，自动审批通过。

CI全绿，自动审批通过。
auto-approve-bot approved these changes 2026-07-17 20:44:57 +08:00
auto-approve-bot left a comment
Collaborator
Copy Link
Copy Source

CI全绿，自动审批通过。

CI全绿，自动审批通过。
auto-approve-bot merged commit 840fd43cb3 into develop 2026-07-17 20:46:29 +08:00
auto-approve-bot deleted branch fix/download-asset-http-fallback 2026-07-17 20:46:29 +08:00
Sign in to join this conversation.
Reviewers
No Reviewers
auto-approve-bot
Labels
Clear labels
P0
阻塞性，必须立即处理
 P1
高优先级
 P2
中优先级
 P3
低优先级/后考虑
 architecture
架构设计
 backend
后端相关
 blocked
阻塞中
 bug
Bug修复
 ci-bug
CI本身的bug
 ci-optimization
CI优化类任务
 ci-pipeline
CI流水线/workflow相关
 cleanup
代码清理
 devops
运维/部署/CI
 documentation
文档相关
 feature
新功能
 frontend
前端相关
 question
需要讨论确认
 ready
PR CI通过，可合并
 refactor
代码重构
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
auto-approve-bot xiaoxia
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: xiaoxia/xiaoxia-saas#487
Delete Branch "fix/download-asset-http-fallback"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?