xiaoxia/xiaoxia-saas
1
0
Fork 0
Code Issues 11 Pull Requests 2 Actions Packages Projects Releases 7 Wiki Activity

test(wave128): url_security纯逻辑抽离 + 110单测 #1031

Closed
xiaoxia wants to merge 1 commits from test/wave128-url-security into develop
pull from: test/wave128-url-security
merge into: xiaoxia:develop
Conversation 3 Commits 1 Files Changed 3
+848 -362
xiaoxia commented 2026-07-27 13:34:03 +08:00
Owner
Copy Link
Copy Source

wave128: URL安全校验纯逻辑抽离与单测覆盖

变更内容

  • 新建 packages/domain/url_security.py：SSRF防护纯逻辑模块，无IO无环境变量
    • URL基础校验：scheme白名单/端口白名单/主机名/内部域名/IP格式SSRF/可信域名白名单
    • 魔数校验纯函数（接收bytes而非文件路径）
    • 常量定义、异常类、辅助判断函数
  • 重构 packages/shared/url_security.py：薄包装层
    • 纯逻辑全部委托给domain层
    • 保留DNS解析、文件下载等有副作用逻辑
    • 完全向后兼容，worker/tts/voice_clone等引用方无需修改
  • 新增 tests/unit/test_url_security_domain.py：110个单测

测试覆盖

模块 用例数 覆盖场景
常量校验 4 schemes/ports/max_length/magic_numbers
内部主机名检查 13 精确匹配+后缀匹配+大小写+正常域名
可信域名匹配 6 精确/子域名/大小写/空集合
SSRF IP检查 20 回环/私有/链路本地/组播/未指定/保留/公网
IP地址判断 10 IPv4/IPv6/域名
URL基础校验 23 正常/空/过长/scheme/hostname/端口/直连IP/可信域名
is_url_basic_safe 4 安全/不安全/空/白名单
魔数校验 20 MP3/WAV/PNG/JPEG/GIF/MP4/OGG/FLAC/WEBP/BMP/空文件/不匹配/跳过
合计 110

关键修复

  • check_ssrf_ip 检查顺序调整：link_local/unspecified 放在 private 之前，避免 169.254.x.x/0.0.0.0 被误报为"内网地址"而非更准确的分类
## wave128: URL安全校验纯逻辑抽离与单测覆盖 ### 变更内容 - **新建** `packages/domain/url_security.py`：SSRF防护纯逻辑模块，无IO无环境变量 - URL基础校验：scheme白名单/端口白名单/主机名/内部域名/IP格式SSRF/可信域名白名单 - 魔数校验纯函数（接收bytes而非文件路径） - 常量定义、异常类、辅助判断函数 - **重构** `packages/shared/url_security.py`：薄包装层 - 纯逻辑全部委托给domain层 - 保留DNS解析、文件下载等有副作用逻辑 - 完全向后兼容，worker/tts/voice_clone等引用方无需修改 - **新增** `tests/unit/test_url_security_domain.py`：110个单测 ### 测试覆盖 | 模块 | 用例数 | 覆盖场景 | |------|--------|----------| | 常量校验 | 4 | schemes/ports/max_length/magic_numbers | | 内部主机名检查 | 13 | 精确匹配+后缀匹配+大小写+正常域名 | | 可信域名匹配 | 6 | 精确/子域名/大小写/空集合 | | SSRF IP检查 | 20 | 回环/私有/链路本地/组播/未指定/保留/公网 | | IP地址判断 | 10 | IPv4/IPv6/域名 | | URL基础校验 | 23 | 正常/空/过长/scheme/hostname/端口/直连IP/可信域名 | | is_url_basic_safe | 4 | 安全/不安全/空/白名单 | | 魔数校验 | 20 | MP3/WAV/PNG/JPEG/GIF/MP4/OGG/FLAC/WEBP/BMP/空文件/不匹配/跳过 | | **合计** | **110** | | ### 关键修复 - `check_ssrf_ip` 检查顺序调整：link_local/unspecified 放在 private 之前，避免 169.254.x.x/0.0.0.0 被误报为"内网地址"而非更准确的分类
gitea-actions bot commented 2026-07-27 16:22:52 +08:00
Copy Link
Copy Source

🚀 预览环境已部署

项目 详情
PR号 #1031
预览链接 https://pr-1031.preview.xiaoxiajianji.com
API环境 staging

💡 预览环境使用 staging API 数据，请勿在预览环境中操作重要数据。

🔄 每次提交新代码后预览环境会自动更新。

🗑️ PR 关闭或合并后，预览环境会自动清理。

🚀 **预览环境已部署** | 项目 | 详情 | |------|------| | PR号 | #1031 | | 预览链接 | [https://pr-1031.preview.xiaoxiajianji.com](https://pr-1031.preview.xiaoxiajianji.com) | | API环境 | staging | > 💡 预览环境使用 staging API 数据，请勿在预览环境中操作重要数据。 > > 🔄 每次提交新代码后预览环境会自动更新。 > > 🗑️ PR 关闭或合并后，预览环境会自动清理。
auto-approve-bot commented 2026-07-27 16:23:54 +08:00
Collaborator
Copy Link
Copy Source

代码审查结果 - PR #1031

⚠️ 问题（2个需要修改）

  1. packages/domain/url_security.py 第85行：域名后缀匹配存在安全漏洞，可能导致绕过白名单。

    • 问题描述：使用 hostname_lower.endswith("." + domain.lower()) 进行子域名匹配时，如果白名单包含 example.com，攻击者可以使用 example.com.evil.com 进行绕过，因为该字符串确实以 .example.com 结尾。
    • 后果：攻击者可利用此漏洞访问未授权的恶意域名，破坏白名单防护机制。
    • 修改建议：应通过分割域名组件进行比较，确保匹配的是真正的子域。例如： 
      parts = hostname_lower.split('.')
domain_parts = domain.lower().split('.')
return parts[-len(domain_parts):] == domain_parts

  2. packages/shared/url_security.py 第209行：解析 Content-Length 时未处理异常，可能导致程序崩溃。

    • 问题描述：代码直接调用 int(content_length)，如果服务器返回的 Content-Length 头部格式非法（如非数字字符串、包含逗号等），会抛出 ValueError，导致下载任务异常终止，而不是抛出统一的 UrlSecurityError。
    • 后果：恶意服务器可以通过发送畸形头部拒绝服务（DoS），影响服务稳定性。
    • 修改建议：增加 try-except 块捕获数值转换异常，或使用正则预校验格式。

💡 建议（3个可选）

  1. packages/domain/url_security.py 第78-85行：is_trusted_domain 函数存在性能隐患。

    • 具体内容：每次调用函数时都会执行 {d.lower() for d in trusted_domains} 创建一个新的 Set。如果白名单较大且该函数被频繁调用（如高并发下载或多次重定向），会产生不必要的 CPU 开销。建议在调用方（shared 层）预先处理好大小写，或确保传入的 trusted_domains 已经是归一化的。

  2. packages/shared/url_security.py 第116行：函数内部导入模块不符合常规风格。

    • 具体内容：在 validate_url_safety 函数内部使用 from urllib.parse import urlparse。虽然 Python 允许这样做，但通常应将所有 import 语句放在文件顶部，除非是为了解决循环依赖或极其特殊的懒加载需求。建议移至文件顶部。

  3. packages/shared/url_security.py：DNS 解析与实际请求之间存在 TOCTOU（Time-of-check to time-of-use）竞态条件风险。

    • 具体内容：代码先在 _check_ssrf_domain 中解析 DNS 并校验 IP，随后在 safe_download_file 中发起请求。如果攻击者在 DNS 校验与实际连接之间改变 DNS 解析结果（DNS Rebinding 攻击），仍可能绕过 SSRF 防护。虽然使用标准库 urllib 很难完美解决此问题（需要绑定 IP 连接），但建议知晓此风险，或在安全要求极高的场景下改用支持 IP 绑定的 HTTP 客户端。

✅ 格式检查通过 | ❌ 逻辑审查需修改 | ⚠️ 建议关注性能

🤖 由 AI 代码审查机器人自动生成 | 2026-07-27 08:23:54 | 模型:

## 代码审查结果 - PR #1031 ### ⚠️ 问题（2个需要修改） 1. **packages/domain/url_security.py 第85行**：域名后缀匹配存在安全漏洞，可能导致绕过白名单。 - **问题描述**：使用 `hostname_lower.endswith("." + domain.lower())` 进行子域名匹配时，如果白名单包含 `example.com`，攻击者可以使用 `example.com.evil.com` 进行绕过，因为该字符串确实以 `.example.com` 结尾。 - **后果**：攻击者可利用此漏洞访问未授权的恶意域名，破坏白名单防护机制。 - **修改建议**：应通过分割域名组件进行比较，确保匹配的是真正的子域。例如： ```python parts = hostname_lower.split('.') domain_parts = domain.lower().split('.') return parts[-len(domain_parts):] == domain_parts ``` 2. **packages/shared/url_security.py 第209行**：解析 `Content-Length` 时未处理异常，可能导致程序崩溃。 - **问题描述**：代码直接调用 `int(content_length)`，如果服务器返回的 `Content-Length` 头部格式非法（如非数字字符串、包含逗号等），会抛出 `ValueError`，导致下载任务异常终止，而不是抛出统一的 `UrlSecurityError`。 - **后果**：恶意服务器可以通过发送畸形头部拒绝服务（DoS），影响服务稳定性。 - **修改建议**：增加 try-except 块捕获数值转换异常，或使用正则预校验格式。 ### 💡 建议（3个可选） 1. **packages/domain/url_security.py 第78-85行**：`is_trusted_domain` 函数存在性能隐患。 - **具体内容**：每次调用函数时都会执行 `{d.lower() for d in trusted_domains}` 创建一个新的 Set。如果白名单较大且该函数被频繁调用（如高并发下载或多次重定向），会产生不必要的 CPU 开销。建议在调用方（`shared` 层）预先处理好大小写，或确保传入的 `trusted_domains` 已经是归一化的。 2. **packages/shared/url_security.py 第116行**：函数内部导入模块不符合常规风格。 - **具体内容**：在 `validate_url_safety` 函数内部使用 `from urllib.parse import urlparse`。虽然 Python 允许这样做，但通常应将所有 import 语句放在文件顶部，除非是为了解决循环依赖或极其特殊的懒加载需求。建议移至文件顶部。 3. **packages/shared/url_security.py**：DNS 解析与实际请求之间存在 TOCTOU（Time-of-check to time-of-use）竞态条件风险。 - **具体内容**：代码先在 `_check_ssrf_domain` 中解析 DNS 并校验 IP，随后在 `safe_download_file` 中发起请求。如果攻击者在 DNS 校验与实际连接之间改变 DNS 解析结果（DNS Rebinding 攻击），仍可能绕过 SSRF 防护。虽然使用标准库 `urllib` 很难完美解决此问题（需要绑定 IP 连接），但建议知晓此风险，或在安全要求极高的场景下改用支持 IP 绑定的 HTTP 客户端。 --- ✅ 格式检查通过 | ❌ 逻辑审查需修改 | ⚠️ 建议关注性能 --- <sub>🤖 由 AI 代码审查机器人自动生成 | 2026-07-27 08:23:54 | 模型: </sub> <!-- AI_CODE_REVIEW_AUTO_COMMENT -->
xiaoxia added 1 commit 2026-07-27 18:25:55 +08:00
test(wave128): url_security纯逻辑抽离+110单测
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 5s
Details
CI/CD Pipeline / Validate - Type Check (mypy) (pull_request) Successful in 1m2s
Details
CI/CD Pipeline / Validate - Code Quality (pull_request) Failing after 1m37s
Details
CI/CD Pipeline / Frontend Lint (pull_request) Failing after 36s
Details
CI/CD Pipeline / Validate - Migration (alembic) (pull_request) Successful in 1m6s
Details
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 1m59s
Details
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
Details
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 3m5s
Details
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 1m36s
Details
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 45s
Details
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m0s
Details
AI Code Review / AI Code Review (pull_request) Successful in 6m24s
Details
CI/CD Pipeline / Unit Tests (pull_request) Failing after 0s
Details
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
Details
CI/CD Pipeline / Integration Tests (pull_request) Failing after 1s
Details
CI/CD Pipeline / CI Gate (pull_request) Failing after 0s
Details
ACR Cleanup / ACR Image Cleanup (pull_request_target) Failing after 0s
Details
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 46m23s
Details
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
Details
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 41s
Details
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
Details
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
Details
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
Details
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
Details
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
Details
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
Details
c7a7c019a3 
- 新建 packages/domain/url_security.py：SSRF防护纯逻辑模块
  - URL基础校验（scheme/port/hostname/internal/IP-SSRF/trusted-domain）
  - 魔数校验纯函数（接收bytes）
  - 常量、异常类、辅助函数
- packages/shared/url_security.py 改为薄包装
  - 保留DNS解析、文件下载等有副作用逻辑
  - 纯逻辑委托给domain层，完全向后兼容
- 110个单测全覆盖：常量/内部主机名/可信域名/SSRF IP/IP判断/URL校验/魔数校验
- 修复check_ssrf_ip检查顺序：link_local/unspecified放private前
xiaoxia force-pushed test/wave128-url-security from 019201fb5c to c7a7c019a3 2026-07-27 18:25:55 +08:00 Compare
xiaoxia closed this pull request 2026-07-27 20:21:09 +08:00
gitea-actions bot commented 2026-07-27 21:01:46 +08:00
Copy Link
Copy Source

🗑️ 预览环境已清理

PR #1031 已关闭或合并，对应的预览环境已被清理。

如有需要，可以重新打开 PR 来重新生成预览环境。

🗑️ **预览环境已清理** PR #1031 已关闭或合并，对应的预览环境已被清理。 > 如有需要，可以重新打开 PR 来重新生成预览环境。
Some checks are pending
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 5s
Details
CI/CD Pipeline / Validate - Type Check (mypy) (pull_request) Successful in 1m2s
Details
CI/CD Pipeline / Validate - Code Quality (pull_request) Failing after 1m37s
Details
CI/CD Pipeline / Frontend Lint (pull_request) Failing after 36s
Details
CI/CD Pipeline / Validate - Migration (alembic) (pull_request) Successful in 1m6s
Details
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 1m59s
Details
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
Details
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 3m5s
Details
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 1m36s
Details
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 45s
Details
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m0s
Details
AI Code Review / AI Code Review (pull_request) Successful in 6m24s
Details
CI/CD Pipeline / Unit Tests (pull_request) Failing after 0s
Details
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
Details
CI/CD Pipeline / Integration Tests (pull_request) Failing after 1s
Details
CI/CD Pipeline / CI Gate (pull_request) Failing after 0s
Details
ACR Cleanup / ACR Image Cleanup (pull_request_target) Failing after 0s
Details
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 46m23s
Details
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
Details
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
Details
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 41s
Details
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
Details
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
Details
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
Details
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
Details
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
Details
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
Details

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
P0
阻塞性，必须立即处理
 P1
高优先级
 P2
中优先级
 P3
低优先级/后考虑
 architecture
架构设计
 backend
后端相关
 blocked
阻塞中
 bug
Bug修复
 ci-bug
CI本身的bug
 ci-optimization
CI优化类任务
 ci-pipeline
CI流水线/workflow相关
 cleanup
代码清理
 devops
运维/部署/CI
 documentation
文档相关
 feature
新功能
 frontend
前端相关
 question
需要讨论确认
 ready
PR CI通过，可合并
 refactor
代码重构
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
auto-approve-bot xiaoxia
No Assignees
3 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: xiaoxia/xiaoxia-saas#1031
Delete Branch "test/wave128-url-security"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?