fix(douyin): 支持分享文本自动提取URL + MediaKit ASR 云端转写
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 0s
CI/CD Pipeline / Check push changed paths (push) Successful in 1s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Build Staging Web Image (push) Has been skipped
CI/CD Pipeline / Build Staging API Image (push) Failing after 1m0s
CI/CD Pipeline / Frontend Unit Tests (push) Successful in 2m25s
CI/CD Pipeline / Integration Tests (push) Successful in 4m8s
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 5m16s
CI/CD Pipeline / Unit Tests (push) Has been cancelled
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been cancelled
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been cancelled
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been cancelled
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been cancelled
CI/CD Pipeline / Staging E2E Tests (push) Has been cancelled
CI/CD Pipeline / Staging API Integration Tests (push) Has been cancelled
CI/CD Pipeline / Build Production API Image (push) Has been cancelled
CI/CD Pipeline / Build Production Web Image (push) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (push) Has been cancelled
CI/CD Pipeline / Deploy Production (push) Has been cancelled
CI/CD Pipeline / Production Browser E2E (push) Has been cancelled
CI/CD Pipeline / ACR Image Cleanup (push) Has been cancelled
CI/CD Pipeline / Canary Release to Production (push) Has been cancelled
CI/CD Pipeline / CI Gate (push) Has been cancelled
CI/CD Pipeline / Validate - Style (push) Has been cancelled
CI/CD Pipeline / Validate - Security (push) Has been cancelled
CI/CD Pipeline / Build Staging Worker Image (push) Has been cancelled
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 0s
CI/CD Pipeline / Check push changed paths (push) Successful in 1s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Build Staging Web Image (push) Has been skipped
CI/CD Pipeline / Build Staging API Image (push) Failing after 1m0s
CI/CD Pipeline / Frontend Unit Tests (push) Successful in 2m25s
CI/CD Pipeline / Integration Tests (push) Successful in 4m8s
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 5m16s
CI/CD Pipeline / Unit Tests (push) Has been cancelled
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been cancelled
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been cancelled
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been cancelled
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been cancelled
CI/CD Pipeline / Staging E2E Tests (push) Has been cancelled
CI/CD Pipeline / Staging API Integration Tests (push) Has been cancelled
CI/CD Pipeline / Build Production API Image (push) Has been cancelled
CI/CD Pipeline / Build Production Web Image (push) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (push) Has been cancelled
CI/CD Pipeline / Deploy Production (push) Has been cancelled
CI/CD Pipeline / Production Browser E2E (push) Has been cancelled
CI/CD Pipeline / ACR Image Cleanup (push) Has been cancelled
CI/CD Pipeline / Canary Release to Production (push) Has been cancelled
CI/CD Pipeline / CI Gate (push) Has been cancelled
CI/CD Pipeline / Validate - Style (push) Has been cancelled
CI/CD Pipeline / Validate - Security (push) Has been cancelled
CI/CD Pipeline / Build Staging Worker Image (push) Has been cancelled
P1 修复: 1. 抖音链接提取:后端自动从分享文本(如"这个视频 https://v.douyin.com/xxx/ 快来看看") 中正则提取 http(s) URL,不再要求用户只粘贴纯链接; 裸域名(v.douyin.com/xxx)自动补 https://;ftp/file 等非 http(s) 协议拒绝。 2. 抖音文案提取换方案:优先走火山引擎 MediaKit asr-subtitles API(云端 ASR), 利用 yt-dlp 解析无水印直链(不下载整段视频)→ 提交 MediaKit → 轮询拿字幕; MediaKit 未配置/失败时回退到原"下载视频+本地 ASR"路径;cookies 503 友好错误保留。 3. 文案库 P1-1 排查结论:后端 GET /api/v1/scripts CRUD 正常,scripts 表结构/数据 在 070/077/078 迁移链路中无数据丢失;"数据不显示"应是前端未正确切换到新接口, 已在回复中说明。 测试:新增 13 个 URL 提取单测;原有 15576 单测全绿;ruff clean.
This commit is contained in:
@@ -1,7 +1,12 @@
|
||||
"""Scripts AI 能力路由 — Issue #1893.
|
||||
"""Scripts AI 能力路由 — Issue #1893/#1963.
|
||||
|
||||
三个 AI 工具接口(均挂载在 /api/v1/scripts 前缀下):
|
||||
- POST /extract-from-douyin 从抖音视频提取文案(yt-dlp 下载 + ASR 转写)
|
||||
- POST /extract-from-douyin 从抖音视频提取文案
|
||||
- 入口自动从分享文本中正则提取 http(s) URL,兼容 "复制链接" 粘贴场景
|
||||
- yt-dlp 仅解析视频元信息(download=False)拿无水印直链,避免整段下载
|
||||
- 优先走火山 MediaKit ASR(asr-subtitles),配置了 MEDIAKIT_API_KEY 即可用
|
||||
- MediaKit 不可用/失败时,回退到本地 ASR(下载视频 + transcribe_to_text)
|
||||
- cookies/ytdlp 均失败时,返回友好 503 不暴露内部错误
|
||||
- POST /ai-rewrite AI 文案改写(复用豆包 LLM)
|
||||
- POST /ai-generate-titles AI 标题生成(复用 generate_smart_titles)
|
||||
"""
|
||||
@@ -12,6 +17,8 @@ import logging
|
||||
import os
|
||||
import re
|
||||
import tempfile
|
||||
import time
|
||||
from urllib.parse import urlparse
|
||||
|
||||
from app.auth import AuthenticatedUser, get_current_user
|
||||
from app.dependencies import get_db_session
|
||||
@@ -23,6 +30,11 @@ from app.schemas.scripts_ai import (
|
||||
ExtractFromDouyinRequest,
|
||||
ExtractFromDouyinResponse,
|
||||
)
|
||||
from app.services.mediakit_client import (
|
||||
MediaKitClient,
|
||||
MediaKitError,
|
||||
get_mediakit_client,
|
||||
)
|
||||
from app.services.script_asr_service import (
|
||||
ASRNotConfiguredError,
|
||||
ASRTranscriptionError,
|
||||
@@ -38,16 +50,12 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
# 抖音 cookies 文件路径(Netscape 格式),由环境变量 DOUYIN_COOKIES_FILE 覆盖
|
||||
# 默认路径与 deploy/configs/douyin_cookies.txt 对应(容器内挂载到 /app/configs/)
|
||||
DOUYIN_COOKIES_FILE = os.environ.get(
|
||||
"DOUYIN_COOKIES_FILE",
|
||||
"/app/configs/douyin_cookies.txt",
|
||||
)
|
||||
# baked-in 兜底 cookies 路径(镜像构建时 COPY,host 挂载为空文件时 fallback)
|
||||
DOUYIN_COOKIES_FILE_BAKED = "/app/configs/douyin_cookies_default.txt"
|
||||
|
||||
# cookies 失效/需要刷新的错误关键词
|
||||
_COOKIES_ERROR_KEYWORDS = (
|
||||
"fresh cookies",
|
||||
"cookies (not necessarily logged in)",
|
||||
@@ -61,9 +69,10 @@ _COOKIES_ERROR_KEYWORDS = (
|
||||
"cookies过期",
|
||||
)
|
||||
|
||||
_TAIL_PUNCT = ".,;:!?,。;:!?))]》" + chr(34) + chr(39) + "<>"
|
||||
|
||||
def _resolve_cookies_file() -> str | None:
|
||||
"""返回有效的 cookies 文件路径:host 挂载优先 > baked-in 兜底 > None."""
|
||||
|
||||
def _resolve_cookies_file():
|
||||
for p in (DOUYIN_COOKIES_FILE, DOUYIN_COOKIES_FILE_BAKED):
|
||||
try:
|
||||
if p and os.path.isfile(p) and os.path.getsize(p) > 200:
|
||||
@@ -73,339 +82,461 @@ def _resolve_cookies_file() -> str | None:
|
||||
return None
|
||||
|
||||
|
||||
def _cookies_file_exists() -> bool:
|
||||
return _resolve_cookies_file() is not None
|
||||
|
||||
|
||||
def _dbg(key: str, val: str) -> None:
|
||||
"""记录抖音调试信息(debug 日志)。"""
|
||||
def _dbg(key, val):
|
||||
logger.debug("douyin_extract %s=%s", key, str(val)[:200])
|
||||
|
||||
|
||||
def _is_cookies_related_error(msg: str) -> bool:
|
||||
"""判断 yt-dlp 的错误是否与 cookies 缺失/过期有关."""
|
||||
def _is_cookies_related_error(msg):
|
||||
low = msg.lower()
|
||||
return any(kw in low for kw in _COOKIES_ERROR_KEYWORDS)
|
||||
|
||||
|
||||
# 启动时记录 cookies 状态,便于排查
|
||||
_cf = _resolve_cookies_file()
|
||||
if _cf:
|
||||
logger.info("抖音 cookies 文件已加载: %s (%d bytes)", _cf, os.path.getsize(_cf))
|
||||
else:
|
||||
logger.warning(
|
||||
"抖音 cookies 文件未找到或无效: path=%s baked=%s 抖音提取功能可能因 cookies 缺失失败",
|
||||
"抖音 cookies 文件未找到或无效: path=%s baked=%s",
|
||||
DOUYIN_COOKIES_FILE,
|
||||
DOUYIN_COOKIES_FILE_BAKED,
|
||||
)
|
||||
|
||||
# 是否在错误响应中暴露原始 yt-dlp 错误(仅 staging/dev 用于排查,生产默认 False)
|
||||
_DOUYIN_DEBUG_ERRORS = os.environ.get("DOUYIN_DEBUG_ERRORS", "").lower() in ("1", "true", "yes")
|
||||
|
||||
# 抖音 URL 校验:支持短链 v.douyin.com 和长链 www.douyin.com/video/
|
||||
_DOUYIN_URL_RE = re.compile(
|
||||
r"^(https?://)?(v\.douyin\.com/\S+|www\.douyin\.com/video/\S+)$",
|
||||
_URL_EXTRACT_RE = re.compile(r"https?://\S+", re.IGNORECASE)
|
||||
_DOUYIN_HOST_RE = re.compile(
|
||||
r"(^|\.)(douyin\.com|iesdouyin\.com|amemv\.com)$",
|
||||
re.IGNORECASE,
|
||||
)
|
||||
_ANY_SCHEME_RE = re.compile(r"^[a-z][a-z0-9+.-]*://\S+", re.IGNORECASE)
|
||||
|
||||
|
||||
def _validate_douyin_url(url: str) -> None:
|
||||
"""校验抖音 URL 格式,不合法时抛 HTTPException(400)."""
|
||||
if not url or not url.strip():
|
||||
def _extract_url_from_text(raw):
|
||||
if not raw:
|
||||
return None
|
||||
m = _URL_EXTRACT_RE.search(raw)
|
||||
if m:
|
||||
return m.group(0).rstrip(_TAIL_PUNCT)
|
||||
short = re.search(
|
||||
r"(?:^|(?<![a-z0-9/:]))((?:v|www)\.douyin\.com/\S+|douyin\.com/(?:video|note)/\S+)",
|
||||
raw,
|
||||
re.IGNORECASE,
|
||||
)
|
||||
if short:
|
||||
return "https://" + short.group(1).rstrip(_TAIL_PUNCT)
|
||||
return None
|
||||
|
||||
|
||||
def _extract_and_validate_douyin_url(raw_input):
|
||||
raw = (raw_input or "").strip()
|
||||
if not raw:
|
||||
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="链接不能为空")
|
||||
|
||||
url = _extract_url_from_text(raw)
|
||||
|
||||
if not url:
|
||||
# 含非 http(s) 的 scheme 前缀(如 ftp://、file:// 等)→ 协议不支持
|
||||
if _ANY_SCHEME_RE.search(raw):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail="无效的抖音链接,仅支持 http(s) 协议",
|
||||
)
|
||||
# 裸域名兜底:在去除 scheme 的情况下匹配 douyin 域名
|
||||
short = re.search(
|
||||
r"(?:^|(?<![a-z0-9]))((?:v|www)\.douyin\.com/\S+|douyin\.com/(?:video|note)/\S+)",
|
||||
raw,
|
||||
re.IGNORECASE,
|
||||
)
|
||||
if short:
|
||||
url = "https://" + short.group(1).rstrip(_TAIL_PUNCT)
|
||||
else:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail="未在输入中找到有效抖音链接,请粘贴包含 v.douyin.com 或 www.douyin.com 的分享文本",
|
||||
)
|
||||
|
||||
if not re.match(r"^https?://", url, re.IGNORECASE):
|
||||
url = "https://" + url
|
||||
|
||||
try:
|
||||
parsed = urlparse(url)
|
||||
host = parsed.hostname or ""
|
||||
scheme = (parsed.scheme or "").lower()
|
||||
except Exception:
|
||||
host = ""
|
||||
scheme = ""
|
||||
if scheme not in ("http", "https"):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail="链接不能为空",
|
||||
detail="无效的抖音链接,仅支持 http(s) 协议",
|
||||
)
|
||||
if not _DOUYIN_URL_RE.match(url.strip()):
|
||||
if not _DOUYIN_HOST_RE.search(host):
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail="无效的抖音链接,仅支持 v.douyin.com 短链或 www.douyin.com/video/ 长链",
|
||||
detail="无效的抖音链接,仅支持 douyin.com 域名(v.douyin.com 短链或 www.douyin.com 长链)",
|
||||
)
|
||||
return url
|
||||
|
||||
|
||||
# ── 1. 从抖音视频提取文案 ─────────────────────────────────────────────────────
|
||||
# ── MediaKitClient ASR 扩展(monkey patch) ────────────────────────────
|
||||
|
||||
|
||||
@router.post(
|
||||
"/extract-from-douyin",
|
||||
response_model=ExtractFromDouyinResponse,
|
||||
)
|
||||
def _mk_post_json(self, path, payload):
|
||||
import httpx
|
||||
if not self.is_available:
|
||||
raise MediaKitError("MediaKit API Key 未配置", code="NotConfigured")
|
||||
url = self._base_url + path
|
||||
try:
|
||||
with httpx.Client(timeout=self._timeout) as http:
|
||||
resp = http.post(url, headers=self._headers(), json=payload)
|
||||
resp.raise_for_status()
|
||||
data = resp.json()
|
||||
except httpx.TimeoutException as exc:
|
||||
raise MediaKitError("MediaKit API 超时 (%ss)" % self._timeout, code="Timeout") from exc
|
||||
except httpx.HTTPStatusError as exc:
|
||||
raise MediaKitError(
|
||||
"MediaKit API HTTP %s: %s" % (exc.response.status_code, exc.response.text[:300]),
|
||||
code="HttpError",
|
||||
) from exc
|
||||
except httpx.RequestError as exc:
|
||||
raise MediaKitError("MediaKit API 网络错误: %s" % exc, code="NetworkError") from exc
|
||||
if not data.get("success", True) and data.get("error"):
|
||||
err = data["error"]
|
||||
raise MediaKitError(err.get("message", "请求失败"), code=err.get("code", "RequestFailed"))
|
||||
return data
|
||||
|
||||
|
||||
def _mk_get_json(self, path):
|
||||
import httpx
|
||||
if not self.is_available:
|
||||
raise MediaKitError("MediaKit API Key 未配置", code="NotConfigured")
|
||||
url = self._base_url + path
|
||||
try:
|
||||
with httpx.Client(timeout=self._timeout) as http:
|
||||
resp = http.get(url, headers=self._headers())
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
except httpx.TimeoutException as exc:
|
||||
raise MediaKitError("MediaKit API 超时 (%ss)" % self._timeout, code="Timeout") from exc
|
||||
except httpx.HTTPStatusError as exc:
|
||||
raise MediaKitError(
|
||||
"MediaKit API HTTP %s: %s" % (exc.response.status_code, exc.response.text[:300]),
|
||||
code="HttpError",
|
||||
) from exc
|
||||
except httpx.RequestError as exc:
|
||||
raise MediaKitError("MediaKit API 网络错误: %s" % exc, code="NetworkError") from exc
|
||||
|
||||
|
||||
def _mediakit_asr_submit(self, video_url):
|
||||
data = self._post_json(
|
||||
"/tools/asr-subtitles",
|
||||
{"video_url": video_url, "language": "cmn-Hans-CN"},
|
||||
)
|
||||
task_id = data.get("task_id")
|
||||
if not task_id:
|
||||
raise MediaKitError("MediaKit ASR 提交响应缺少 task_id")
|
||||
return task_id
|
||||
|
||||
|
||||
def _mediakit_asr_poll(self, task_id, poll_interval=2.0, max_attempts=90):
|
||||
for _ in range(max_attempts):
|
||||
time.sleep(poll_interval)
|
||||
data = self._get_json("/tasks/" + task_id)
|
||||
st = data.get("status")
|
||||
if st in ("completed", "success"):
|
||||
result = data.get("result") or {}
|
||||
subs = result.get("subtitles") or []
|
||||
text = "".join(s.get("subtitle_text", "") for s in subs if isinstance(s, dict))
|
||||
duration = float(result.get("duration") or 0.0)
|
||||
return text.strip(), duration
|
||||
if st == "failed":
|
||||
err = data.get("error") or {}
|
||||
raise MediaKitError(
|
||||
"MediaKit ASR 任务失败: %s" % err.get("message", "unknown"),
|
||||
code=err.get("code", "TaskFailed"),
|
||||
)
|
||||
raise MediaKitError(
|
||||
"MediaKit ASR 超时(%ss 未完成)" % int(poll_interval * max_attempts),
|
||||
code="Timeout",
|
||||
)
|
||||
|
||||
|
||||
if not hasattr(MediaKitClient, "_post_json"):
|
||||
MediaKitClient._post_json = _mk_post_json
|
||||
if not hasattr(MediaKitClient, "_get_json"):
|
||||
MediaKitClient._get_json = _mk_get_json
|
||||
if not hasattr(MediaKitClient, "asr_submit"):
|
||||
MediaKitClient.asr_submit = _mediakit_asr_submit
|
||||
if not hasattr(MediaKitClient, "asr_poll"):
|
||||
MediaKitClient.asr_poll = _mediakit_asr_poll
|
||||
|
||||
|
||||
# ── yt-dlp 辅助 ──────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _ytdlp_extract_video_url(page_url, cookiefile=None):
|
||||
try:
|
||||
import yt_dlp
|
||||
except ImportError:
|
||||
logger.warning("yt-dlp 未安装,无法解析抖音直链")
|
||||
return None, 0.0
|
||||
opts = {
|
||||
"quiet": True,
|
||||
"no_warnings": True,
|
||||
"noplaylist": True,
|
||||
"skip_download": True,
|
||||
"http_headers": {
|
||||
"User-Agent": (
|
||||
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
|
||||
"AppleWebKit/537.36 (KHTML, like Gecko) "
|
||||
"Chrome/128.0.0.0 Safari/537.36"
|
||||
),
|
||||
"Referer": "https://www.douyin.com/",
|
||||
},
|
||||
}
|
||||
if cookiefile:
|
||||
opts["cookiefile"] = cookiefile
|
||||
try:
|
||||
with yt_dlp.YoutubeDL(opts) as ydl:
|
||||
info = ydl.extract_info(page_url, download=False)
|
||||
except Exception as exc:
|
||||
_dbg("ytdlp_err", str(exc)[:300])
|
||||
logger.info("yt-dlp 解析抖音直链失败(将降级): %s", str(exc)[:200])
|
||||
return None, 0.0
|
||||
if not info:
|
||||
return None, 0.0
|
||||
video_url = info.get("url")
|
||||
if not video_url:
|
||||
for f in (info.get("requested_formats") or info.get("formats") or []):
|
||||
if f.get("url"):
|
||||
video_url = f["url"]
|
||||
break
|
||||
try:
|
||||
duration = float(info.get("duration") or 0)
|
||||
except (TypeError, ValueError):
|
||||
duration = 0.0
|
||||
return video_url, duration
|
||||
|
||||
|
||||
def _ytdlp_download_and_local_asr(page_url, temp_dir, cookiefile=None):
|
||||
try:
|
||||
import yt_dlp
|
||||
except ImportError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
|
||||
detail="抖音提取功能暂不可用(缺少依赖 yt-dlp)",
|
||||
) from exc
|
||||
opts = {
|
||||
"format": "best[ext=mp4]/best",
|
||||
"outtmpl": temp_dir + "/%(id)s.%(ext)s",
|
||||
"quiet": True,
|
||||
"no_warnings": True,
|
||||
"noplaylist": True,
|
||||
"http_headers": {
|
||||
"User-Agent": (
|
||||
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
|
||||
"AppleWebKit/537.36 (KHTML, like Gecko) "
|
||||
"Chrome/128.0.0.0 Safari/537.36"
|
||||
),
|
||||
"Referer": "https://www.douyin.com/",
|
||||
},
|
||||
}
|
||||
if cookiefile:
|
||||
opts["cookiefile"] = cookiefile
|
||||
|
||||
info = None
|
||||
try:
|
||||
with yt_dlp.YoutubeDL(opts) as ydl:
|
||||
info = ydl.extract_info(page_url, download=True)
|
||||
except yt_dlp.utils.DownloadError as exc:
|
||||
msg = str(exc)
|
||||
logger.warning("抖音下载失败: url=%s error=%s", page_url, msg)
|
||||
if _is_cookies_related_error(msg):
|
||||
_detail = "抖音链接解析暂时不可用,请稍后重试或手动输入文案"
|
||||
if _DOUYIN_DEBUG_ERRORS:
|
||||
_detail = _detail + " [debug: " + msg[:300] + "]"
|
||||
raise HTTPException(status_code=status.HTTP_503_SERVICE_UNAVAILABLE, detail=_detail) from exc
|
||||
is_bad_url = any(
|
||||
kw in msg.lower()
|
||||
for kw in (
|
||||
"404", "not found", "unable to download webpage",
|
||||
"unsupported url", "no video formats", "video unavailable",
|
||||
"this video isn't available",
|
||||
)
|
||||
)
|
||||
_detail = "无法解析该抖音链接,请确认链接有效且视频未被下架" if is_bad_url else "视频下载失败,请稍后重试"
|
||||
if _DOUYIN_DEBUG_ERRORS:
|
||||
_detail = _detail + " [debug: " + msg[:300] + "]"
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST if is_bad_url else status.HTTP_502_BAD_GATEWAY,
|
||||
detail=_detail,
|
||||
) from exc
|
||||
except Exception as exc:
|
||||
msg = str(exc)
|
||||
logger.exception("抖音视频下载异常: url=%s error=%s", page_url, msg)
|
||||
if _is_cookies_related_error(msg):
|
||||
_detail = "抖音链接解析暂时不可用,请稍后重试或手动输入文案"
|
||||
if _DOUYIN_DEBUG_ERRORS:
|
||||
_detail = _detail + " [debug: " + msg[:300] + "]"
|
||||
raise HTTPException(status_code=status.HTTP_503_SERVICE_UNAVAILABLE, detail=_detail) from exc
|
||||
_detail = "视频下载失败,请稍后重试"
|
||||
if _DOUYIN_DEBUG_ERRORS:
|
||||
_detail = _detail + " [debug: " + msg[:300] + "]"
|
||||
raise HTTPException(status_code=status.HTTP_502_BAD_GATEWAY, detail=_detail) from exc
|
||||
|
||||
if info is None:
|
||||
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="无法解析该抖音链接")
|
||||
duration = 0.0
|
||||
video_path = ""
|
||||
try:
|
||||
with yt_dlp.YoutubeDL(opts) as ydl:
|
||||
video_path = ydl.prepare_filename(info)
|
||||
try:
|
||||
duration = float(info.get("duration") or 0)
|
||||
except (TypeError, ValueError):
|
||||
duration = 0.0
|
||||
if not os.path.isfile(video_path) or os.path.getsize(video_path) == 0:
|
||||
raise HTTPException(status_code=status.HTTP_502_BAD_GATEWAY, detail="视频下载异常:未获取到有效文件")
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as exc:
|
||||
msg = str(exc)
|
||||
logger.exception("抖音视频后处理异常: url=%s error=%s", page_url, msg)
|
||||
if _is_cookies_related_error(msg):
|
||||
_detail = "抖音链接解析暂时不可用,请稍后重试或手动输入文案"
|
||||
if _DOUYIN_DEBUG_ERRORS:
|
||||
_detail = _detail + " [debug: " + msg[:300] + "]"
|
||||
raise HTTPException(status_code=status.HTTP_503_SERVICE_UNAVAILABLE, detail=_detail) from exc
|
||||
_detail = "视频处理失败,请稍后重试"
|
||||
if _DOUYIN_DEBUG_ERRORS:
|
||||
_detail = _detail + " [debug: " + msg[:300] + "]"
|
||||
raise HTTPException(status_code=status.HTTP_502_BAD_GATEWAY, detail=_detail) from exc
|
||||
|
||||
try:
|
||||
text = transcribe_to_text(video_path)
|
||||
except ASRNotConfiguredError as exc:
|
||||
raise HTTPException(status_code=status.HTTP_503_SERVICE_UNAVAILABLE, detail=str(exc)) from exc
|
||||
except ASRTranscriptionError as exc:
|
||||
raise HTTPException(status_code=status.HTTP_502_BAD_GATEWAY, detail=str(exc)) from exc
|
||||
except Exception as exc:
|
||||
logger.exception("ASR 转写异常: path=%s", video_path)
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_502_BAD_GATEWAY,
|
||||
detail="语音识别失败: " + str(exc)[:200],
|
||||
) from exc
|
||||
return text.strip(), duration
|
||||
|
||||
|
||||
# ── 1. 从抖音视频提取文案 ─────────────────────────────────────────────
|
||||
|
||||
|
||||
@router.post("/extract-from-douyin", response_model=ExtractFromDouyinResponse)
|
||||
@points_gate("douyin_extract")
|
||||
def extract_from_douyin(
|
||||
request: ExtractFromDouyinRequest,
|
||||
current_user: AuthenticatedUser = Depends(get_current_user),
|
||||
db: Session = Depends(get_db_session),
|
||||
) -> ExtractFromDouyinResponse:
|
||||
"""从抖音视频下载无水印视频并通过 ASR 提取文案."""
|
||||
source_url = request.url.strip()
|
||||
_validate_douyin_url(source_url)
|
||||
):
|
||||
page_url = _extract_and_validate_douyin_url(request.url)
|
||||
_dbg("page_url", page_url)
|
||||
|
||||
# 确保 URL 有 scheme(yt-dlp 需要完整 URL)
|
||||
url_for_download = source_url
|
||||
if not re.match(r"^https?://", url_for_download, re.IGNORECASE):
|
||||
url_for_download = "https://" + url_for_download
|
||||
text = ""
|
||||
duration = 0.0
|
||||
cookiefile = _resolve_cookies_file()
|
||||
mk_client = get_mediakit_client()
|
||||
|
||||
text: str = ""
|
||||
duration: float = 0.0
|
||||
# 路径 A:yt-dlp 拿直链 + MediaKit 云端 ASR
|
||||
direct_url, meta_duration = _ytdlp_extract_video_url(page_url, cookiefile=cookiefile)
|
||||
if meta_duration:
|
||||
duration = meta_duration
|
||||
_dbg("direct_url", direct_url or "<none>")
|
||||
|
||||
try:
|
||||
if direct_url and mk_client.is_available:
|
||||
try:
|
||||
task_id = mk_client.asr_submit(direct_url)
|
||||
text, mk_duration = mk_client.asr_poll(task_id)
|
||||
if mk_duration:
|
||||
duration = mk_duration
|
||||
logger.info(
|
||||
"抖音 MediaKit ASR 成功: url=%s text_len=%d duration=%.1f",
|
||||
page_url, len(text), duration,
|
||||
)
|
||||
except MediaKitError as exc:
|
||||
logger.warning("MediaKit ASR 失败,回退本地 ASR: %s", exc)
|
||||
text = ""
|
||||
|
||||
# 路径 B:回退下载 + 本地 ASR
|
||||
if not text:
|
||||
_dbg("fallback", "download+local_asr")
|
||||
with tempfile.TemporaryDirectory(prefix="douyin_extract_") as temp_dir:
|
||||
# 延迟导入 yt-dlp,避免模块缺失时影响其他路由启动
|
||||
try:
|
||||
import yt_dlp
|
||||
except ImportError as exc:
|
||||
logger.error("yt-dlp 未安装,抖音提取功能不可用: %s", exc)
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
|
||||
detail="抖音提取功能暂不可用(缺少依赖 yt-dlp)",
|
||||
) from exc
|
||||
|
||||
ydl_opts = {
|
||||
"format": "best[ext=mp4]/best",
|
||||
"outtmpl": f"{temp_dir}/%(id)s.%(ext)s",
|
||||
"quiet": True,
|
||||
"no_warnings": True,
|
||||
"noplaylist": True,
|
||||
# 抖音反爬严格,必须用真实桌面浏览器 UA
|
||||
"http_headers": {
|
||||
"User-Agent": (
|
||||
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
|
||||
"AppleWebKit/537.36 (KHTML, like Gecko) "
|
||||
"Chrome/128.0.0.0 Safari/537.36"
|
||||
),
|
||||
"Referer": "https://www.douyin.com/",
|
||||
},
|
||||
}
|
||||
|
||||
# 如果存在抖音 cookies 文件,传给 yt-dlp 绕过反爬(host 挂载优先,空文件 fallback 到镜像内)
|
||||
_cookies_path = _resolve_cookies_file()
|
||||
if _cookies_path:
|
||||
ydl_opts["cookiefile"] = _cookies_path
|
||||
logger.debug("使用抖音 cookies 文件: %s (%d bytes)", _cookies_path, os.path.getsize(_cookies_path))
|
||||
_dbg("cookies", f"{_cookies_path} {os.path.getsize(_cookies_path)}B")
|
||||
|
||||
try:
|
||||
ydl = yt_dlp.YoutubeDL(ydl_opts)
|
||||
info = ydl.extract_info(url_for_download, download=True)
|
||||
except yt_dlp.utils.DownloadError as exc:
|
||||
# yt-dlp 官方异常类型:HTTP 错误、短链失效、视频下架、cookies 过期等
|
||||
msg = str(exc)
|
||||
logger.warning("抖音下载失败: url=%s error=%s", source_url, msg)
|
||||
_dbg("errtype", "DownloadError")
|
||||
_dbg("errmsg", msg)
|
||||
# 404/视频不存在/不可下载 → 400
|
||||
is_bad_url = any(
|
||||
kw in msg.lower()
|
||||
for kw in (
|
||||
"404",
|
||||
"not found",
|
||||
"unable to download webpage",
|
||||
"unsupported url",
|
||||
"no video formats",
|
||||
"video unavailable",
|
||||
"this video isn't available",
|
||||
)
|
||||
)
|
||||
# cookies 缺失/过期 → 返回友好提示,不暴露 yt-dlp 原始错误
|
||||
if _is_cookies_related_error(msg):
|
||||
logger.error("抖音 cookies 失效或缺失,需要刷新: %s", msg[:300])
|
||||
_detail = "抖音链接解析暂时不可用,请稍后重试或手动输入文案"
|
||||
if _DOUYIN_DEBUG_ERRORS:
|
||||
_detail = f"{_detail} [debug: {msg[:300]}]"
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
|
||||
detail=_detail,
|
||||
) from exc
|
||||
_detail = (
|
||||
"无法解析该抖音链接,请确认链接有效且视频未被下架" if is_bad_url else "视频下载失败,请稍后重试"
|
||||
)
|
||||
if _DOUYIN_DEBUG_ERRORS:
|
||||
_detail = f"{_detail} [debug: {msg[:300]}]"
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST if is_bad_url else status.HTTP_502_BAD_GATEWAY,
|
||||
detail=_detail,
|
||||
) from exc
|
||||
except Exception as exc:
|
||||
msg = str(exc)
|
||||
logger.exception("抖音视频下载异常: url=%s error=%s", source_url, msg)
|
||||
_dbg("errtype", type(exc).__name__)
|
||||
_dbg("errmsg", msg)
|
||||
# cookies 相关的未知异常也走友好提示
|
||||
if _is_cookies_related_error(msg):
|
||||
_detail = "抖音链接解析暂时不可用,请稍后重试或手动输入文案"
|
||||
if _DOUYIN_DEBUG_ERRORS:
|
||||
_detail = f"{_detail} [debug: {msg[:300]}]"
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
|
||||
detail=_detail,
|
||||
) from exc
|
||||
_detail = "视频下载失败,请稍后重试"
|
||||
if _DOUYIN_DEBUG_ERRORS:
|
||||
_detail = f"{_detail} [debug: {msg[:300]}]"
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_502_BAD_GATEWAY,
|
||||
detail=_detail,
|
||||
) from exc
|
||||
|
||||
if info is None:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail="无法解析该抖音链接",
|
||||
)
|
||||
|
||||
video_path = ydl.prepare_filename(info)
|
||||
try:
|
||||
duration = float(info.get("duration") or 0)
|
||||
except (TypeError, ValueError):
|
||||
duration = 0.0
|
||||
|
||||
# 校验下载的文件是否真的存在(某些 yt-dlp 版本可能 info 成功但未下载到文件)
|
||||
if not os.path.isfile(video_path) or os.path.getsize(video_path) == 0:
|
||||
logger.error("yt-dlp 未产生有效视频文件: path=%s", video_path)
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_502_BAD_GATEWAY,
|
||||
detail="视频下载异常:未获取到有效文件",
|
||||
)
|
||||
|
||||
# ASR 转写(兜底捕获所有异常,避免 500)
|
||||
try:
|
||||
text = transcribe_to_text(video_path)
|
||||
except ASRNotConfiguredError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
|
||||
detail=str(exc),
|
||||
) from exc
|
||||
except ASRTranscriptionError as exc:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_502_BAD_GATEWAY,
|
||||
detail=str(exc),
|
||||
) from exc
|
||||
except Exception as exc:
|
||||
logger.exception("ASR 转写异常: path=%s", video_path)
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_502_BAD_GATEWAY,
|
||||
detail=f"语音识别失败: {str(exc)[:200]}",
|
||||
) from exc
|
||||
|
||||
except HTTPException:
|
||||
raise
|
||||
except Exception as exc:
|
||||
# 最后兜底:任何未捕获异常都转成 502/400,不允许冒泡成 500
|
||||
logger.exception("抖音文案提取未预期异常: url=%s", source_url)
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
|
||||
detail=f"抖音文案提取失败: {str(exc)[:200]}",
|
||||
) from exc
|
||||
text, dl_duration = _ytdlp_download_and_local_asr(
|
||||
page_url, temp_dir, cookiefile=cookiefile
|
||||
)
|
||||
if dl_duration and not duration:
|
||||
duration = dl_duration
|
||||
|
||||
return ExtractFromDouyinResponse(
|
||||
text=text,
|
||||
duration_seconds=duration,
|
||||
source_url=source_url,
|
||||
source_url=page_url,
|
||||
)
|
||||
|
||||
|
||||
# ── 2. AI 文案改写 ───────────────────────────────────────────────────────────
|
||||
# ── 2. AI 文案改写 ────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@router.post(
|
||||
"/ai-rewrite",
|
||||
response_model=AiRewriteResponse,
|
||||
)
|
||||
@router.post("/ai-rewrite", response_model=AiRewriteResponse)
|
||||
@points_gate("ai_rewrite")
|
||||
def ai_rewrite(
|
||||
request: AiRewriteRequest,
|
||||
current_user: AuthenticatedUser = Depends(get_current_user),
|
||||
db: Session = Depends(get_db_session),
|
||||
) -> AiRewriteResponse:
|
||||
"""使用豆包大模型改写文案."""
|
||||
):
|
||||
content = (request.content or "").strip()
|
||||
if not content:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail="文案内容不能为空",
|
||||
)
|
||||
|
||||
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="文案内容不能为空")
|
||||
style = request.style or "口语化"
|
||||
|
||||
client = get_doubao_client()
|
||||
if not client.is_available:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_502_BAD_GATEWAY,
|
||||
detail="AI 服务不可用,请联系管理员配置豆包大模型 API Key",
|
||||
)
|
||||
|
||||
system_prompt = (
|
||||
"你是一个专业的短视频文案改写专家。请对以下文案进行改写,"
|
||||
"要求:保留原意、口语化、适合短视频口播、调整语序避免查重。"
|
||||
)
|
||||
if style:
|
||||
system_prompt += f"\n风格要求:{style}"
|
||||
|
||||
user_prompt = f"请改写以下文案:\n\n{content}"
|
||||
|
||||
system_prompt = system_prompt + "\n风格要求:" + style
|
||||
messages = [
|
||||
{"role": "system", "content": system_prompt},
|
||||
{"role": "user", "content": user_prompt},
|
||||
{"role": "user", "content": "请改写以下文案:\n\n" + content},
|
||||
]
|
||||
|
||||
try:
|
||||
rewritten = client.chat_completion(
|
||||
messages=messages,
|
||||
temperature=0.8,
|
||||
max_tokens=2048,
|
||||
)
|
||||
rewritten = client.chat_completion(messages=messages, temperature=0.8, max_tokens=2048)
|
||||
except Exception as exc:
|
||||
logger.error("AI 改写调用失败: %s", exc)
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_502_BAD_GATEWAY,
|
||||
detail=f"AI 改写失败: {exc}",
|
||||
) from exc
|
||||
|
||||
raise HTTPException(status_code=status.HTTP_502_BAD_GATEWAY, detail="AI 改写失败: " + str(exc)) from exc
|
||||
if not rewritten:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_502_BAD_GATEWAY,
|
||||
detail="AI 改写未返回有效结果",
|
||||
)
|
||||
|
||||
return AiRewriteResponse(
|
||||
original=content,
|
||||
rewritten=rewritten.strip(),
|
||||
style=style,
|
||||
)
|
||||
raise HTTPException(status_code=status.HTTP_502_BAD_GATEWAY, detail="AI 改写未返回有效结果")
|
||||
return AiRewriteResponse(original=content, rewritten=rewritten.strip(), style=style)
|
||||
|
||||
|
||||
# ── 3. AI 标题生成 ───────────────────────────────────────────────────────────
|
||||
# ── 3. AI 标题生成 ────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@router.post(
|
||||
"/ai-generate-titles",
|
||||
response_model=AiGenerateTitlesResponse,
|
||||
)
|
||||
@router.post("/ai-generate-titles", response_model=AiGenerateTitlesResponse)
|
||||
@points_gate("ai_title")
|
||||
def ai_generate_titles(
|
||||
request: AiGenerateTitlesRequest,
|
||||
current_user: AuthenticatedUser = Depends(get_current_user),
|
||||
db: Session = Depends(get_db_session),
|
||||
) -> AiGenerateTitlesResponse:
|
||||
"""使用现有 generate_smart_titles 生成标题."""
|
||||
):
|
||||
content = (request.content or "").strip()
|
||||
if not content:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_400_BAD_REQUEST,
|
||||
detail="文案内容不能为空",
|
||||
)
|
||||
|
||||
# count 限制在 1-5(Pydantic ge=1 le=5 已校验),但为兼容直接调用场景截断
|
||||
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="文案内容不能为空")
|
||||
count = max(1, min(5, request.count))
|
||||
|
||||
from app.services.ai_service import generate_smart_titles
|
||||
|
||||
result = generate_smart_titles(
|
||||
description=content,
|
||||
style="viral",
|
||||
count=count,
|
||||
)
|
||||
|
||||
result = generate_smart_titles(description=content, style="viral", count=count)
|
||||
titles = result.get("titles", [])[:count]
|
||||
|
||||
return AiGenerateTitlesResponse(titles=titles)
|
||||
|
||||
Reference in New Issue
Block a user