fix(scripts): #1894/#1968 抖音 yt-dlp cookies 支持 + 旧档位归一化补测 + CI 覆盖率修复
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 26s
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m28s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 1m23s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 2m3s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m11s
AI Code Review / AI Code Review (pull_request) Successful in 6m49s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 10m42s
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Failing after 14m49s
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Integration Tests (pull_request) Successful in 2m30s
CI/CD Pipeline / Unit Tests (pull_request) Has been cancelled
CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / CI Gate (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Style (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Security (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Has been cancelled

- scripts_ai.py: 支持通过 DOUYIN_COOKIES_FILE 指定 Netscape cookies 文件;
  cookies 缺失/过期类错误('fresh cookies' 等)返回 503 + 友好文案
  ('抖音链接解析暂时不可用,请稍后重试或手动输入文案'),
  不再暴露 yt-dlp 原始错误;普通下载失败统一文案,不带内部异常文本
- ci_staging_deploy.sh: API 容器挂载 /var/lib/xiaoxia-saas-staging/configs/douyin_cookies.txt
  到 /app/configs/douyin_cookies.txt;自动 mkdir + 空占位兜底
- infra/docker/compose.yml: API 服务挂载 ../../deploy/configs:/app/configs:ro
- deploy/configs/.env.{staging,production}: 新增 DOUYIN_COOKIES_FILE 环境变量
- deploy/configs/douyin_cookies.txt: Netscape 格式占位文件(真实 cookies 部署时替换)
- 新增 test_helpers_get_user_plan.py 覆盖 _helpers.get_user_plan 旧档位归一化分支
- 新增 test_subscription_routes_legacy.py 覆盖 _build_subscription_info 旧档位归一化/
  cancel_subscription 免费用户分支/change_plan 参数校验/toggle-auto-renew 等
- 扩展 test_extract_from_douyin_errors.py: 新增 5 个 cookies 场景测试(503 友好文案/
  cookiefile 注入/错误信息脱敏等)
- 增量覆盖率从 59% 提升到 60%+,解决 CI Gate 阻塞
This commit is contained in:
2026-09-16 18:28:23 +08:00
parent 02c7bba247
commit d855fc30ba
9 changed files with 472 additions and 7 deletions
+70 -6
View File
@@ -38,6 +38,51 @@ logger = logging.getLogger(__name__)
router = APIRouter()
# 抖音 cookies 文件路径(Netscape 格式),由环境变量 DOUYIN_COOKIES_FILE 覆盖
# 默认路径与 deploy/configs/douyin_cookies.txt 对应(容器内挂载到 /app/configs/)
DOUYIN_COOKIES_FILE = os.environ.get(
"DOUYIN_COOKIES_FILE",
"/app/configs/douyin_cookies.txt",
)
# cookies 失效/需要刷新的错误关键词
_COOKIES_ERROR_KEYWORDS = (
"fresh cookies",
"cookies (not necessarily logged in)",
"cookies are needed",
"need cookies",
"cookie is expired",
"login required",
"sign in to continue",
"未登录",
"需要登录",
"cookies过期",
)
def _cookies_file_exists() -> bool:
"""检查抖音 cookies 文件是否存在且非空."""
try:
return bool(DOUYIN_COOKIES_FILE) and os.path.isfile(DOUYIN_COOKIES_FILE) and os.path.getsize(DOUYIN_COOKIES_FILE) > 100
except OSError:
return False
def _is_cookies_related_error(msg: str) -> bool:
"""判断 yt-dlp 的错误是否与 cookies 缺失/过期有关."""
low = msg.lower()
return any(kw in low for kw in _COOKIES_ERROR_KEYWORDS)
# 启动时记录 cookies 状态,便于排查
if _cookies_file_exists():
logger.info("抖音 cookies 文件已加载: %s (%d bytes)", DOUYIN_COOKIES_FILE, os.path.getsize(DOUYIN_COOKIES_FILE))
else:
logger.warning(
"抖音 cookies 文件未找到或无效: path=%s 抖音提取功能可能因 cookies 缺失失败",
DOUYIN_COOKIES_FILE,
)
# 抖音 URL 校验:支持短链 v.douyin.com 和长链 www.douyin.com/video/
_DOUYIN_URL_RE = re.compile(
r"^(https?://)?(v\.douyin\.com/\S+|www\.douyin\.com/video/\S+)$",
@@ -104,26 +149,45 @@ def extract_from_douyin(
"noplaylist": True,
}
# 如果存在抖音 cookies 文件,传给 yt-dlp 绕过反爬
if _cookies_file_exists():
ydl_opts["cookiefile"] = DOUYIN_COOKIES_FILE
logger.debug("使用抖音 cookies 文件: %s", DOUYIN_COOKIES_FILE)
try:
ydl = yt_dlp.YoutubeDL(ydl_opts)
info = ydl.extract_info(url_for_download, download=True)
except yt_dlp.utils.DownloadError as exc:
# yt-dlp 官方异常类型:HTTP 错误、短链失效、视频下架等
# yt-dlp 官方异常类型:HTTP 错误、短链失效、视频下架、cookies 过期等
msg = str(exc)
logger.warning("抖音下载失败: url=%s error=%s", source_url, msg)
# 404/视频不存在/不可下载 → 400;网络问题/上游异常 → 502
# 404/视频不存在/不可下载 → 400
is_bad_url = any(
kw in msg.lower() for kw in ("404", "not found", "unable to download webpage", "unsupported url", "no video formats")
kw in msg.lower() for kw in ("404", "not found", "unable to download webpage", "unsupported url", "no video formats", "video unavailable", "this video isn't available")
)
# cookies 缺失/过期 → 返回友好提示,不暴露 yt-dlp 原始错误
if _is_cookies_related_error(msg):
logger.error("抖音 cookies 失效或缺失,需要刷新: %s", msg[:300])
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail="抖音链接解析暂时不可用,请稍后重试或手动输入文案",
) from exc
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST if is_bad_url else status.HTTP_502_BAD_GATEWAY,
detail=("无法解析该抖音链接,请确认链接有效且视频未被下架" if is_bad_url else f"视频下载失败: {msg[:200]}"),
detail=("无法解析该抖音链接,请确认链接有效且视频未被下架" if is_bad_url else "视频下载失败,请稍后重试"),
) from exc
except Exception as exc:
logger.exception("抖音视频下载异常: url=%s", source_url)
msg = str(exc)
logger.exception("抖音视频下载异常: url=%s error=%s", source_url, msg)
# cookies 相关的未知异常也走友好提示
if _is_cookies_related_error(msg):
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail="抖音链接解析暂时不可用,请稍后重试或手动输入文案",
) from exc
raise HTTPException(
status_code=status.HTTP_502_BAD_GATEWAY,
detail=f"视频下载失败: {str(exc)[:200]}",
detail="视频下载失败,请稍后重试",
) from exc
if info is None: