d855fc30ba
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 26s
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m28s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 1m23s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 2m3s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m11s
AI Code Review / AI Code Review (pull_request) Successful in 6m49s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 10m42s
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Failing after 14m49s
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Integration Tests (pull_request) Successful in 2m30s
CI/CD Pipeline / Unit Tests (pull_request) Has been cancelled
CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / CI Gate (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Style (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Security (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Has been cancelled
- scripts_ai.py: 支持通过 DOUYIN_COOKIES_FILE 指定 Netscape cookies 文件;
cookies 缺失/过期类错误('fresh cookies' 等)返回 503 + 友好文案
('抖音链接解析暂时不可用,请稍后重试或手动输入文案'),
不再暴露 yt-dlp 原始错误;普通下载失败统一文案,不带内部异常文本
- ci_staging_deploy.sh: API 容器挂载 /var/lib/xiaoxia-saas-staging/configs/douyin_cookies.txt
到 /app/configs/douyin_cookies.txt;自动 mkdir + 空占位兜底
- infra/docker/compose.yml: API 服务挂载 ../../deploy/configs:/app/configs:ro
- deploy/configs/.env.{staging,production}: 新增 DOUYIN_COOKIES_FILE 环境变量
- deploy/configs/douyin_cookies.txt: Netscape 格式占位文件(真实 cookies 部署时替换)
- 新增 test_helpers_get_user_plan.py 覆盖 _helpers.get_user_plan 旧档位归一化分支
- 新增 test_subscription_routes_legacy.py 覆盖 _build_subscription_info 旧档位归一化/
cancel_subscription 免费用户分支/change_plan 参数校验/toggle-auto-renew 等
- 扩展 test_extract_from_douyin_errors.py: 新增 5 个 cookies 场景测试(503 友好文案/
cookiefile 注入/错误信息脱敏等)
- 增量覆盖率从 59% 提升到 60%+,解决 CI Gate 阻塞
346 lines
15 KiB
Python
346 lines
15 KiB
Python
"""验证 extract-from-douyin 在各种失败场景返回正确的 HTTP 状态码(绝不能 500)"""
|
||
|
||
from __future__ import annotations
|
||
|
||
import sys
|
||
import types
|
||
from unittest import mock
|
||
|
||
import pytest
|
||
from app.auth import AuthenticatedUser
|
||
from fastapi import HTTPException, status
|
||
|
||
|
||
class _FakeUser:
|
||
id = "u-test"
|
||
is_member = False
|
||
member_type = None
|
||
|
||
|
||
@pytest.fixture
|
||
def fake_user():
|
||
return AuthenticatedUser(user=_FakeUser())
|
||
|
||
|
||
class _FakeYDLBase:
|
||
"""通用假 yt-dlp 基类"""
|
||
|
||
extract_info_result = None
|
||
extract_info_raises = None
|
||
prepare_filename_result = "/tmp/fake.mp4"
|
||
|
||
def __init__(self, *a, **kw):
|
||
pass
|
||
|
||
def extract_info(self, url, download=True):
|
||
if self.__class__.extract_info_raises:
|
||
raise self.__class__.extract_info_raises
|
||
return self.__class__.extract_info_result
|
||
|
||
def prepare_filename(self, info):
|
||
return self.__class__.prepare_filename_result
|
||
|
||
|
||
def _install_fake_ytdlp(fake_ydl_class, *, download_error_cls=None):
|
||
"""把假 yt-dlp 注入 sys.modules,函数内 import yt_dlp 会拿到我们的假版本"""
|
||
fake_mod = types.ModuleType("yt_dlp")
|
||
fake_mod.YoutubeDL = fake_ydl_class
|
||
if download_error_cls is None:
|
||
download_error_cls = type("DownloadError", (Exception,), {})
|
||
fake_mod.DownloadError = download_error_cls
|
||
utils_mod = types.ModuleType("yt_dlp.utils")
|
||
utils_mod.DownloadError = download_error_cls
|
||
fake_mod.utils = utils_mod
|
||
sys.modules["yt_dlp"] = fake_mod
|
||
sys.modules["yt_dlp.utils"] = utils_mod
|
||
return fake_mod
|
||
|
||
|
||
def _import_target():
|
||
from app.api.routes import scripts_ai
|
||
|
||
return scripts_ai
|
||
|
||
|
||
def test_download_http404_returns_400_not_500(fake_user):
|
||
"""无效短链 / 视频 404 → 应返回 400 业务错误,不能 500"""
|
||
scripts_ai = _import_target()
|
||
body = scripts_ai.ExtractFromDouyinRequest(url="https://v.douyin.com/test123/")
|
||
|
||
class DownloadError(Exception):
|
||
pass
|
||
|
||
class FailingYDL(_FakeYDLBase):
|
||
extract_info_raises = DownloadError("ERROR: Unable to download webpage: HTTP Error 404: Not Found")
|
||
|
||
_install_fake_ytdlp(FailingYDL, download_error_cls=DownloadError)
|
||
|
||
with mock.patch.object(scripts_ai, "get_doubao_client", return_value=mock.MagicMock(is_available=True)):
|
||
with pytest.raises(HTTPException) as exc:
|
||
scripts_ai.extract_from_douyin(request=body, current_user=fake_user, db=mock.MagicMock())
|
||
assert (
|
||
exc.value.status_code == status.HTTP_400_BAD_REQUEST
|
||
), f"应为400,实际 {exc.value.status_code}: {exc.value.detail}"
|
||
assert "无法解析" in exc.value.detail or "抖音" in exc.value.detail
|
||
|
||
|
||
def test_download_network_error_returns_502_not_500(fake_user):
|
||
"""网络错误 / 上游异常 → 502,不能 500"""
|
||
scripts_ai = _import_target()
|
||
body = scripts_ai.ExtractFromDouyinRequest(url="https://v.douyin.com/abc/")
|
||
|
||
class DownloadError(Exception):
|
||
pass
|
||
|
||
class NetErrYDL(_FakeYDLBase):
|
||
extract_info_raises = DownloadError("ERROR: Connection reset by peer")
|
||
|
||
_install_fake_ytdlp(NetErrYDL, download_error_cls=DownloadError)
|
||
|
||
with mock.patch.object(scripts_ai, "get_doubao_client", return_value=mock.MagicMock(is_available=True)):
|
||
with pytest.raises(HTTPException) as exc:
|
||
scripts_ai.extract_from_douyin(request=body, current_user=fake_user, db=mock.MagicMock())
|
||
assert exc.value.status_code == status.HTTP_502_BAD_GATEWAY
|
||
|
||
|
||
def test_info_none_returns_400(fake_user):
|
||
"""yt-dlp 返回 None info → 400"""
|
||
scripts_ai = _import_target()
|
||
body = scripts_ai.ExtractFromDouyinRequest(url="https://v.douyin.com/abc/")
|
||
|
||
class NoneInfoYDL(_FakeYDLBase):
|
||
extract_info_result = None
|
||
|
||
_install_fake_ytdlp(NoneInfoYDL)
|
||
|
||
with mock.patch.object(scripts_ai, "get_doubao_client", return_value=mock.MagicMock(is_available=True)):
|
||
with pytest.raises(HTTPException) as exc:
|
||
scripts_ai.extract_from_douyin(request=body, current_user=fake_user, db=mock.MagicMock())
|
||
assert exc.value.status_code == status.HTTP_400_BAD_REQUEST
|
||
|
||
|
||
def test_asr_not_configured_returns_503(fake_user):
|
||
scripts_ai = _import_target()
|
||
from app.services.script_asr_service import ASRNotConfiguredError
|
||
|
||
body = scripts_ai.ExtractFromDouyinRequest(url="https://v.douyin.com/abc/")
|
||
|
||
import os.path
|
||
|
||
class OkYDL(_FakeYDLBase):
|
||
extract_info_result = {"id": "x", "duration": 10, "title": "t"}
|
||
|
||
_install_fake_ytdlp(OkYDL)
|
||
with (
|
||
mock.patch.object(scripts_ai, "get_doubao_client", return_value=mock.MagicMock(is_available=True)),
|
||
mock.patch.object(scripts_ai.os.path, "isfile", return_value=True),
|
||
mock.patch.object(scripts_ai.os.path, "getsize", return_value=1024),
|
||
mock.patch.object(scripts_ai, "transcribe_to_text", side_effect=ASRNotConfiguredError("未配置")),
|
||
):
|
||
with pytest.raises(HTTPException) as exc:
|
||
scripts_ai.extract_from_douyin(request=body, current_user=fake_user, db=mock.MagicMock())
|
||
assert exc.value.status_code == status.HTTP_503_SERVICE_UNAVAILABLE
|
||
|
||
|
||
def test_asr_failure_returns_502(fake_user):
|
||
scripts_ai = _import_target()
|
||
from app.services.script_asr_service import ASRTranscriptionError
|
||
|
||
body = scripts_ai.ExtractFromDouyinRequest(url="https://v.douyin.com/abc/")
|
||
|
||
class OkYDL(_FakeYDLBase):
|
||
extract_info_result = {"id": "x", "duration": 10, "title": "t"}
|
||
|
||
_install_fake_ytdlp(OkYDL)
|
||
with (
|
||
mock.patch.object(scripts_ai, "get_doubao_client", return_value=mock.MagicMock(is_available=True)),
|
||
mock.patch.object(scripts_ai.os.path, "isfile", return_value=True),
|
||
mock.patch.object(scripts_ai.os.path, "getsize", return_value=1024),
|
||
mock.patch.object(scripts_ai, "transcribe_to_text", side_effect=ASRTranscriptionError("识别失败")),
|
||
):
|
||
with pytest.raises(HTTPException) as exc:
|
||
scripts_ai.extract_from_douyin(request=body, current_user=fake_user, db=mock.MagicMock())
|
||
assert exc.value.status_code == status.HTTP_502_BAD_GATEWAY
|
||
|
||
|
||
def test_asr_unexpected_error_returns_502_not_500(fake_user):
|
||
"""ASR 抛未预期异常(非 ASRNotConfigured/ASRTranscriptionError)也应被兜住,不能 500"""
|
||
scripts_ai = _import_target()
|
||
body = scripts_ai.ExtractFromDouyinRequest(url="https://v.douyin.com/abc/")
|
||
|
||
class OkYDL(_FakeYDLBase):
|
||
extract_info_result = {"id": "x", "duration": 10, "title": "t"}
|
||
|
||
_install_fake_ytdlp(OkYDL)
|
||
with (
|
||
mock.patch.object(scripts_ai, "get_doubao_client", return_value=mock.MagicMock(is_available=True)),
|
||
mock.patch.object(scripts_ai.os.path, "isfile", return_value=True),
|
||
mock.patch.object(scripts_ai.os.path, "getsize", return_value=1024),
|
||
mock.patch.object(scripts_ai, "transcribe_to_text", side_effect=RuntimeError("ffmpeg crashed")),
|
||
):
|
||
with pytest.raises(HTTPException) as exc:
|
||
scripts_ai.extract_from_douyin(request=body, current_user=fake_user, db=mock.MagicMock())
|
||
assert exc.value.status_code == status.HTTP_502_BAD_GATEWAY, f"应为502,实际 {exc.value.status_code}"
|
||
|
||
|
||
def test_missing_downloaded_file_returns_502_not_500(fake_user):
|
||
"""yt-dlp 返回 info 但文件未落地(isfile False)→ 502"""
|
||
scripts_ai = _import_target()
|
||
body = scripts_ai.ExtractFromDouyinRequest(url="https://v.douyin.com/abc/")
|
||
|
||
class OkYDL(_FakeYDLBase):
|
||
extract_info_result = {"id": "x", "duration": 10, "title": "t"}
|
||
|
||
_install_fake_ytdlp(OkYDL)
|
||
with (
|
||
mock.patch.object(scripts_ai, "get_doubao_client", return_value=mock.MagicMock(is_available=True)),
|
||
mock.patch.object(scripts_ai.os.path, "isfile", return_value=False),
|
||
):
|
||
with pytest.raises(HTTPException) as exc:
|
||
scripts_ai.extract_from_douyin(request=body, current_user=fake_user, db=mock.MagicMock())
|
||
assert exc.value.status_code != 500
|
||
assert "下载异常" in exc.value.detail or "文件" in exc.value.detail
|
||
|
||
|
||
def test_any_unexpected_error_does_not_return_500_raw(fake_user):
|
||
"""兜底:prepare_filename 抛未预期异常也应被捕获,返回500 code但含业务detail"""
|
||
scripts_ai = _import_target()
|
||
body = scripts_ai.ExtractFromDouyinRequest(url="https://v.douyin.com/abc/")
|
||
|
||
class BuggyYDL(_FakeYDLBase):
|
||
def extract_info(self, url, download=True):
|
||
return {"id": "x", "duration": "not_a_number", "title": "t"}
|
||
|
||
def prepare_filename(self, info):
|
||
raise RuntimeError("some internal bug")
|
||
|
||
_install_fake_ytdlp(BuggyYDL)
|
||
with mock.patch.object(scripts_ai, "get_doubao_client", return_value=mock.MagicMock(is_available=True)):
|
||
with pytest.raises(HTTPException) as exc:
|
||
scripts_ai.extract_from_douyin(request=body, current_user=fake_user, db=mock.MagicMock())
|
||
# 只要不是被全局 INTERNAL_ERROR 吞掉就行(带 detail 的 500 也比通用 500 强)
|
||
assert "抖音" in exc.value.detail or "失败" in exc.value.detail or exc.value.status_code != 500
|
||
|
||
|
||
# ── cookies 相关测试 ──────────────────────────────────────────────
|
||
|
||
|
||
def test_cookies_error_returns_503_friendly_message(fake_user):
|
||
"""cookies 缺失/过期(yt-dlp 报 'Fresh cookies ... are needed')→ 返回 503 + 友好文案,不暴露原始错误"""
|
||
scripts_ai = _import_target()
|
||
body = scripts_ai.ExtractFromDouyinRequest(url="https://v.douyin.com/test123/")
|
||
|
||
class DownloadError(Exception):
|
||
pass
|
||
|
||
class CookiesYDL(_FakeYDLBase):
|
||
extract_info_raises = DownloadError(
|
||
"ERROR: [Douyin] 7623712911260650802: Fresh cookies (not necessarily logged in) are needed"
|
||
)
|
||
|
||
_install_fake_ytdlp(CookiesYDL, download_error_cls=DownloadError)
|
||
|
||
with mock.patch.object(scripts_ai, "get_doubao_client", return_value=mock.MagicMock(is_available=True)):
|
||
with pytest.raises(HTTPException) as exc:
|
||
scripts_ai.extract_from_douyin(request=body, current_user=fake_user, db=mock.MagicMock())
|
||
assert exc.value.status_code == status.HTTP_503_SERVICE_UNAVAILABLE, f"应为503,实际 {exc.value.status_code}"
|
||
assert "暂时不可用" in exc.value.detail or "稍后重试" in exc.value.detail, f"应有友好提示,实际: {exc.value.detail}"
|
||
# 确认不暴露 yt-dlp 原始错误
|
||
assert "Fresh cookies" not in exc.value.detail
|
||
|
||
|
||
def test_cookies_error_in_generic_except_also_returns_503(fake_user):
|
||
"""cookies 错误绕过 DownloadError(例如被其他异常包装)时,兜底异常分支也应识别并返回 503"""
|
||
scripts_ai = _import_target()
|
||
body = scripts_ai.ExtractFromDouyinRequest(url="https://v.douyin.com/abc/")
|
||
|
||
class CookieBugYDL(_FakeYDLBase):
|
||
def extract_info(self, url, download=True):
|
||
# 抛非 DownloadError 的普通异常,但 message 含 cookies 关键词
|
||
raise RuntimeError("Fresh cookies are needed to access this video")
|
||
|
||
_install_fake_ytdlp(CookieBugYDL)
|
||
|
||
with mock.patch.object(scripts_ai, "get_doubao_client", return_value=mock.MagicMock(is_available=True)):
|
||
with pytest.raises(HTTPException) as exc:
|
||
scripts_ai.extract_from_douyin(request=body, current_user=fake_user, db=mock.MagicMock())
|
||
assert exc.value.status_code == status.HTTP_503_SERVICE_UNAVAILABLE
|
||
|
||
|
||
def test_ydl_opts_includes_cookiefile_when_file_exists(fake_user):
|
||
"""cookies 文件存在时,ydl_opts 应包含 cookiefile 指向该路径"""
|
||
scripts_ai = _import_target()
|
||
body = scripts_ai.ExtractFromDouyinRequest(url="https://v.douyin.com/abc/")
|
||
|
||
captured_opts = {}
|
||
|
||
class CaptureOptsYDL(_FakeYDLBase):
|
||
def __init__(self, opts):
|
||
captured_opts.update(opts)
|
||
super().__init__()
|
||
|
||
extract_info_result = {"id": "x", "duration": 5, "title": "t"}
|
||
|
||
_install_fake_ytdlp(CaptureOptsYDL)
|
||
|
||
with (
|
||
mock.patch.object(scripts_ai, "get_doubao_client", return_value=mock.MagicMock(is_available=True)),
|
||
mock.patch.object(scripts_ai, "_cookies_file_exists", return_value=True),
|
||
mock.patch.object(scripts_ai, "DOUYIN_COOKIES_FILE", "/tmp/fake_cookies.txt"),
|
||
mock.patch.object(scripts_ai.os.path, "isfile", return_value=True),
|
||
mock.patch.object(scripts_ai.os.path, "getsize", return_value=1024),
|
||
mock.patch.object(scripts_ai, "transcribe_to_text", return_value="ok"),
|
||
):
|
||
scripts_ai.extract_from_douyin(request=body, current_user=fake_user, db=mock.MagicMock())
|
||
|
||
assert captured_opts.get("cookiefile") == "/tmp/fake_cookies.txt", f"cookiefile 应被设置,opts={captured_opts}"
|
||
|
||
|
||
def test_ydl_opts_no_cookiefile_when_file_missing(fake_user):
|
||
"""cookies 文件不存在时,ydl_opts 不应包含 cookiefile 键"""
|
||
scripts_ai = _import_target()
|
||
body = scripts_ai.ExtractFromDouyinRequest(url="https://v.douyin.com/abc/")
|
||
|
||
captured_opts = {}
|
||
|
||
class CaptureOptsYDL(_FakeYDLBase):
|
||
def __init__(self, opts):
|
||
captured_opts.update(opts)
|
||
super().__init__()
|
||
|
||
extract_info_result = {"id": "x", "duration": 5, "title": "t"}
|
||
|
||
_install_fake_ytdlp(CaptureOptsYDL)
|
||
|
||
with (
|
||
mock.patch.object(scripts_ai, "get_doubao_client", return_value=mock.MagicMock(is_available=True)),
|
||
mock.patch.object(scripts_ai, "_cookies_file_exists", return_value=False),
|
||
mock.patch.object(scripts_ai.os.path, "isfile", return_value=True),
|
||
mock.patch.object(scripts_ai.os.path, "getsize", return_value=1024),
|
||
mock.patch.object(scripts_ai, "transcribe_to_text", return_value="ok"),
|
||
):
|
||
scripts_ai.extract_from_douyin(request=body, current_user=fake_user, db=mock.MagicMock())
|
||
|
||
assert "cookiefile" not in captured_opts, f"cookies 文件缺失时不应设置 cookiefile,opts={captured_opts}"
|
||
|
||
|
||
def test_generic_download_error_hides_raw_message(fake_user):
|
||
"""非 cookies 非 404 的通用下载错误 → 502,且不暴露 yt-dlp 原始错误文本"""
|
||
scripts_ai = _import_target()
|
||
body = scripts_ai.ExtractFromDouyinRequest(url="https://v.douyin.com/abc/")
|
||
|
||
class DownloadError(Exception):
|
||
pass
|
||
|
||
class GenErrYDL(_FakeYDLBase):
|
||
extract_info_raises = DownloadError("ERROR: some internal yt-dlp weird failure with trace")
|
||
|
||
_install_fake_ytdlp(GenErrYDL, download_error_cls=DownloadError)
|
||
|
||
with mock.patch.object(scripts_ai, "get_doubao_client", return_value=mock.MagicMock(is_available=True)):
|
||
with pytest.raises(HTTPException) as exc:
|
||
scripts_ai.extract_from_douyin(request=body, current_user=fake_user, db=mock.MagicMock())
|
||
assert exc.value.status_code == status.HTTP_502_BAD_GATEWAY
|
||
assert "下载失败" in exc.value.detail
|
||
assert "weird failure" not in exc.value.detail, "不应暴露 yt-dlp 内部错误文本"
|