Files
xiaoxia-saas/apps/worker/viral_video/video_analyzer.py
T
xiaoxia 9814fcdc22
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m7s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 12m46s
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 3s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
AI Code Review / AI Code Review (pull_request) Successful in 53s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m26s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 30s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 43s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Successful in 1m25s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 1m32s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 1m5s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 5m14s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 6m57s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 8m3s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Validate - Security (pull_request) Successful in 13m36s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 23m20s
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 0s
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
ACR Cleanup / ACR Image Cleanup (pull_request_target) Has been cancelled
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 31s
feat(worker): 参考爆款视频风格分析模块 video_analyzer(#2051)
- 新建 apps/worker/viral_video/video_analyzer.py:6 步管线
  ① FFmpeg 抽关键帧(每 2s + 场景切换帧)
  ② PySceneDetect ContentDetector 镜头分割
  ③ OpenCV Farneback 光流(径向分量法避免平移误判为zoom)
  ④ librosa BPM 分析 → pace 档位
  ⑤ OSS 上传关键帧 + 豆包 VLM 色调/构图/光线分析
  ⑥ 豆包 LLM 整合成 style_guide JSON
- style_guide schema:pace/camera_movements/transitions/color_palette/
  lighting/mood/ken_burns_params(per-shot)/transition_map/
  video_filter_eq_params
- 映射函数:运镜→ken_burns / 转场→xfade / 色调→eq+colorchannelmixer /
  BPM→BGM,并提供 build_render_params_for_clip 按 clip_index 聚合
- 全链路降级(ffmpeg/opencv/librosa/VLM/LLM 任一失败均 best-effort 填充)
- 资源约束:≤60s ≤100MB 超时≤60s try/finally 清理临时帧
- 新增 3 个 worker 依赖:scenedetect / librosa / soundfile
- 48 个单元测试覆盖映射/光流/BPM/降级/清理
- bandit B404/B603 加 nosec,B310 用 httpx 替换 urllib
2026-09-30 00:05:54 +08:00

962 lines
36 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
"""参考爆款视频风格分析模块(#2051,v1.3)。
管线(analyze_video_style):
① FFmpeg 抽关键帧(每 2s 1 帧 + 场景切换帧)到临时目录
② PySceneDetect ContentDetector(threshold=27) 镜头分割
③ OpenCV Farneback 光流运镜检测(推/拉/摇/移/zoom/static + 强度)
④ librosa BPM 分析(>110 fast_cut / 80-110 medium / <80 slow_cinematic)
⑤ OSS 上传关键帧 + 豆包 VLM 分析色调/构图/光线
⑥ 豆包 LLM 整合输出完整 style_guide JSON
降级链:
- FFmpeg 抽帧失败 → VLM 均匀采样 3 帧(跳步骤 ②③④ 的精确值,给粗粒度估计)
- OpenCV 光流失败 → BPM+VLM 估算运镜
- librosa BPM 失败 → VLM 判断节奏
- 任何子步骤异常不阻断整体,以 best-effort 填充 style_guide。
资源约束:
- 参考视频 ≤60s 且 ≤100MB;分析总超时 ≤60s;临时帧 try/finally 清理。
"""
from __future__ import annotations
import json
import logging
import math
import shutil
import subprocess # nosec B404
import tempfile
import uuid
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, Optional
logger = logging.getLogger(__name__)
# ── 资源约束 ─────────────────────────────────────────────────────────────
MAX_REFERENCE_DURATION_SEC = 60
MAX_REFERENCE_SIZE_MB = 100
DEFAULT_ANALYSIS_TIMEOUT = 60 # 秒
KEYFRAME_INTERVAL_SEC = 2
SCENEDETECT_THRESHOLD = 27
VLM_SAMPLE_FRAMES = 5 # 上传给 VLM 的关键帧上限
FARNEBACK_PARAMS = dict(pyr_scale=0.5, levels=3, winsize=15, iterations=3, poly_n=5, poly_sigma=1.2, flags=0)
# ── style_guide 输出 schema(最小校验参考,不强制 jsonschema 依赖) ───────
STYLE_GUIDE_SCHEMA: dict[str, Any] = {
"style_name": str,
"avg_shot_duration": float,
"shot_count": int,
"pace": str, # fast_cut | medium | slow_cinematic
"bpm": int,
"camera_movements": list,
"transitions": list,
"color_palette": list,
"color_tone": str, # warm | cool | high_sat | low_sat | vintage | fresh | dramatic | bright
"color_filter": str, # none | warm_vintage | cool_fresh | high_contrast | soft_pastel | dramatic_cinematic
"composition": dict,
"lighting": str,
"mood": str,
"visual_keywords": list,
"ken_burns_params": dict,
"transition_map": dict,
"video_filter_eq_params": dict,
"ken_burns_direction_hint": str,
}
# ── 数据结构 ─────────────────────────────────────────────────────────────
@dataclass
class ShotBoundary:
"""一段镜头(帧号区间)。"""
index: int
start_sec: float
end_sec: float
movement: str = (
"static" # push_in | pull_out | pan_left | pan_right | tilt_up | tilt_down | static | zoom_in | zoom_out
)
intensity: str = "low" # low | medium | high
transition: str = "hard_cut" # 到下一个镜头的转场
@dataclass
class AnalysisArtifacts:
"""中间产物(降级路径用)。"""
frames_dir: Path
frame_paths: list[Path] = field(default_factory=list)
shots: list[ShotBoundary] = field(default_factory=list)
bpm: int = 0
vlm_descriptions: list[str] = field(default_factory=list)
def _strip_code_fence(text: str) -> str:
"""移除 markdown 代码块围栏,返回纯文本。"""
t = text.strip()
for fence in ("```json", "```JSON", "```"):
if t.startswith(fence):
t = t[len(fence) :].lstrip()
if t.endswith("```"):
t = t[:-3].rstrip()
return t
# ── FFmpeg / ffprobe ─────────────────────────────────────────────────────
def _ffmpeg_bin() -> str:
return shutil.which("ffmpeg") or "ffmpeg"
def _ffprobe_bin() -> str:
return shutil.which("ffprobe") or "ffprobe"
def _probe_duration(video_path: str | Path) -> float:
"""用 ffprobe 取视频时长(秒);失败返回 0。"""
try:
out = subprocess.check_output(
[
_ffprobe_bin(),
"-v",
"error",
"-show_entries",
"format=duration",
"-of",
"default=noprint_wrappers=1:nokey=1",
str(video_path),
],
stderr=subprocess.DEVNULL,
timeout=10,
text=True,
) # nosec B603
return float(out.strip() or 0)
except Exception as exc: # noqa: BLE001
logger.warning("ffprobe 时长探测失败 %s: %s", video_path, exc)
return 0.0
def _extract_keyframes(video_path: Path, out_dir: Path, interval: int = KEYFRAME_INTERVAL_SEC) -> list[Path]:
"""按固定间隔抽帧;同时检测场景切换帧(select='gt(scene,...)')。"""
out_dir.mkdir(parents=True, exist_ok=True)
# 固定间隔
fixed_tpl = str(out_dir / "f_%04d.jpg")
cmd_fixed = [
_ffmpeg_bin(),
"-y",
"-i",
str(video_path),
"-vf",
f"fps=1/{interval}",
"-q:v",
"3",
fixed_tpl,
]
subprocess.run(
cmd_fixed, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, timeout=DEFAULT_ANALYSIS_TIMEOUT, check=False
) # nosec B603
# 场景切换帧(独立命名,scene_ 前缀)
scene_tpl = str(out_dir / "scene_%04d.jpg")
cmd_scene = [
_ffmpeg_bin(),
"-y",
"-i",
str(video_path),
"-vf",
"select='gt(scene,0.35)',showinfo",
"-vsync",
"vfr",
"-q:v",
"3",
scene_tpl,
]
subprocess.run(
cmd_scene, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, timeout=DEFAULT_ANALYSIS_TIMEOUT, check=False
) # nosec B603
frames = sorted(out_dir.glob("f_*.jpg")) + sorted(out_dir.glob("scene_*.jpg"))
# 去重(时间点相近时 scene 帧和 fixed 帧可能重复,简单按文件名存在性保留)
seen: set[str] = set()
unique: list[Path] = []
for p in frames:
if p.name not in seen:
seen.add(p.name)
unique.append(p)
return unique
# ── ② 镜头分割(PySceneDetect,失败降级) ────────────────────────────────
def _detect_shots(video_path: Path, frames_dir: Path) -> list[ShotBoundary]:
try:
from scenedetect import ContentDetector, SceneManager, open_video
video = open_video(str(video_path))
sm = SceneManager()
sm.add_detector(ContentDetector(threshold=SCENEDETECT_THRESHOLD))
sm.detect_scenes(video)
scenes = sm.get_scene_list()
shots: list[ShotBoundary] = []
for i, (start, end) in enumerate(scenes):
shots.append(
ShotBoundary(
index=i,
start_sec=start.get_seconds(),
end_sec=end.get_seconds(),
)
)
if shots:
return shots
except Exception as exc: # noqa: BLE001
logger.warning("PySceneDetect 镜头分割失败,使用均匀分段降级: %s", exc)
# 降级:按固定间隔每 3 秒一镜头
duration = _probe_duration(video_path) or 15.0
dur = max(3.0, min(duration, float(MAX_REFERENCE_DURATION_SEC)))
shots = []
seg = 3.0
i = 0
t = 0.0
while t < dur - 0.1:
shots.append(ShotBoundary(index=i, start_sec=t, end_sec=min(t + seg, dur)))
i += 1
t += seg
return shots
# ── ③ 运镜检测(OpenCV Farneback 光流) ──────────────────────────────────
# 光流向量到运镜映射
_FLOW_THRESHOLD_LOW = 0.3
_FLOW_THRESHOLD_HIGH = 1.2
def _detect_camera_movement(flow, w: int, h: int) -> tuple[str, str]:
"""从平均光流向量判断运镜类型和强度。"""
import numpy as np # noqa: PLC0415 - numpy 已在 requirements 中
fx = float(np.median(flow[..., 0]))
fy = float(np.median(flow[..., 1]))
trans_mag = math.hypot(fx, fy)
# 发散/收敛判断 zoom:比较边缘流沿径向外指的平均分量(稳健版)
cx, cy = w / 2.0, h / 2.0
ys, xs = np.mgrid[0:h, 0:w].astype(np.float32)
rx, ry = (xs - cx) / max(cx, 1.0), (ys - cy) / max(cy, 1.0)
rmag = np.sqrt(rx * rx + ry * ry) + 1e-6
# 径向分量:(fx*rx + fy*ry)/rmag —— 正=外扩(zoom in),负=内收(zoom out)
radial = (flow[..., 0] * rx + flow[..., 1] * ry) / rmag
# 只看边缘带(|r|>0.5),且减去平移贡献:径向减去平均平移投影
edge_mask = (rmag > 0.5).astype(np.float32)
if edge_mask.sum() > 10:
trans_radial = (fx * rx + fy * ry) / rmag
zoom_signal = float(np.mean((radial - trans_radial)[edge_mask > 0]))
else:
zoom_signal = 0.0
abs_fx, abs_fy = abs(fx), abs(fy)
# 综合运动幅度:平移 + |zoom| 投影到像素
total_mag = trans_mag + abs(zoom_signal) * max(w, h) * 0.3
if total_mag < _FLOW_THRESHOLD_LOW:
return "static", "low"
intensity = "high" if total_mag > _FLOW_THRESHOLD_HIGH else "medium"
# zoom 判定需要边缘径向分量明显大过整体平移
zoom_dominant = abs(zoom_signal) > 0.6 and abs(zoom_signal) * max(w, h) * 0.3 > trans_mag * 1.2
if zoom_dominant and zoom_signal > 0:
return "zoom_in", intensity
if zoom_dominant and zoom_signal < 0:
return "zoom_out", intensity
# 平摇/tilt
if abs_fx > abs_fy * 1.5:
return "pan_right" if fx > 0 else "pan_left", intensity
if abs_fy > abs_fx * 1.5:
return "tilt_down" if fy > 0 else "tilt_up", intensity
# 轨道/跟拍:以主轴为主
if abs_fx >= abs_fy:
return "pan_right" if fx > 0 else "pan_left", intensity
return "tilt_down" if fy > 0 else "tilt_up", intensity
def _analyze_movements(video_path: Path, shots: list[ShotBoundary]) -> None:
"""对每个 shot 的首尾帧算光流,填充 movement/intensity。失败时静默降级为 static/low。"""
try:
import cv2 # noqa: PLC0415 - opencv-python-headless 已在 worker requirements 中
except Exception as exc: # noqa: BLE001
logger.warning("OpenCV 不可用,运镜检测降级为 static/low: %s", exc)
return
try:
cap = cv2.VideoCapture(str(video_path))
for shot in shots:
mid_t = (shot.start_sec + shot.end_sec) / 2.0
dt = max(0.2, min(0.5, (shot.end_sec - shot.start_sec) / 4.0))
cap.set(cv2.CAP_PROP_POS_MSEC, max(0.0, (mid_t - dt)) * 1000)
ok1, f1 = cap.read()
cap.set(cv2.CAP_PROP_POS_MSEC, min(mid_t + dt, shot.end_sec - 0.05) * 1000)
ok2, f2 = cap.read()
if not (ok1 and ok2):
continue
g1 = cv2.cvtColor(f1, cv2.COLOR_BGR2GRAY)
g2 = cv2.cvtColor(f2, cv2.COLOR_BGR2GRAY)
h, w = g1.shape
# 降采样加速
scale = 360.0 / h if h > 360 else 1.0
if scale < 1.0:
g1 = cv2.resize(g1, (int(w * scale), int(h * scale)))
g2 = cv2.resize(g2, (int(w * scale), int(h * scale)))
flow = cv2.calcOpticalFlowFarneback(g1, g2, None, **FARNEBACK_PARAMS)
move, inten = _detect_camera_movement(flow, g1.shape[1], g1.shape[0])
shot.movement = move
shot.intensity = inten
cap.release()
except Exception as exc: # noqa: BLE001
logger.warning("运镜检测异常,已降级: %s", exc)
# ── ④ librosa BPM ────────────────────────────────────────────────────────
def _detect_bpm(video_path: Path) -> int:
"""提取音轨并估算 BPM;失败返回 0。"""
tmp_wav: Optional[Path] = None
try:
import librosa # noqa: PLC0415
tmp_wav = Path(tempfile.mkstemp(suffix=".wav")[1])
# ffmpeg 抽 22050Hz 单声道 wav
subprocess.run(
[_ffmpeg_bin(), "-y", "-i", str(video_path), "-vn", "-ac", "1", "-ar", "22050", "-f", "wav", str(tmp_wav)],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
timeout=20,
check=False,
) # nosec B603
if not tmp_wav.exists() or tmp_wav.stat().st_size < 1024:
return 0
y, sr = librosa.load(str(tmp_wav), sr=22050, mono=True)
if len(y) < sr * 2:
return 0
tempo, _ = librosa.beat.beat_track(y=y, sr=sr)
try:
bpm = int(round(float(tempo)))
except Exception: # noqa: BLE001
bpm = int(round(float(tempo[0]))) if len(tempo) else 0
return max(40, min(bpm, 220))
except Exception as exc: # noqa: BLE001
logger.warning("librosa BPM 分析失败: %s", exc)
return 0
finally:
if tmp_wav and tmp_wav.exists():
try:
tmp_wav.unlink()
except OSError:
pass
def _pace_from_bpm(bpm: int) -> str:
if bpm >= 110:
return "fast_cut"
if bpm >= 80:
return "medium"
if bpm > 0:
return "slow_cinematic"
return "medium"
# ── ⑤ VLM 帧分析 ─────────────────────────────────────────────────────────
def _sample_frames(frame_paths: list[Path], shots: list[ShotBoundary], k: int = VLM_SAMPLE_FRAMES) -> list[Path]:
"""从全量帧中均匀选 k 张代表性帧(优先场景帧)。"""
if not frame_paths:
return []
scene_frames = sorted(p for p in frame_paths if p.name.startswith("scene_"))
fixed_frames = sorted(p for p in frame_paths if p.name.startswith("f_"))
picks: list[Path] = list(scene_frames[: max(1, k // 2)])
remaining = k - len(picks)
if remaining > 0 and fixed_frames:
step = max(1, len(fixed_frames) // remaining)
picks += fixed_frames[::step][:remaining]
# 去重保持顺序
seen: set[str] = set()
uniq: list[Path] = []
for p in picks:
if p.name not in seen and p.exists():
seen.add(p.name)
uniq.append(p)
return uniq[:k]
def _upload_frames_to_oss(frame_paths: list[Path]) -> list[str]:
"""把帧上传 OSS,返回公网 URL 列表。失败时降级为 data URI。"""
urls: list[str] = []
try:
from video_processing.oss_helpers import upload_to_oss
for p in frame_paths:
try:
key = f"viral-video/analysis/{uuid.uuid4().hex}/{p.name}"
url = upload_to_oss(p, key)
if url:
urls.append(url)
except Exception as exc: # noqa: BLE001
logger.warning("单帧 OSS 上传失败 %s: %s", p.name, exc)
except Exception as exc: # noqa: BLE001
logger.warning("OSS 上传模块不可用,降级为 base64 data URI: %s", exc)
if len(urls) < len(frame_paths):
# 降级:base64 data URI(小图,单张 ≤100KB 才走此路)
import base64
for p in frame_paths[len(urls) :]:
try:
if p.stat().st_size > 120_000:
continue
b64 = base64.b64encode(p.read_bytes()).decode("ascii")
urls.append(f"data:image/jpeg;base64,{b64}")
except Exception: # noqa: BLE001 # nosec B112
continue
return urls
def _vlm_analyze_frames(image_urls: list[str]) -> dict[str, Any]:
"""调豆包 VLM 分析色调/构图/光线/转场观感。"""
if not image_urls:
return {}
try:
from packages.shared.ai_client import get_doubao_client
client = get_doubao_client()
if not client.is_available:
raise RuntimeError("豆包客户端未配置")
sys_prompt = (
"你是资深短视频导演和调色师。根据用户给出的同一支短视频的多张关键帧,"
"分析其视觉风格并严格输出 JSON(不要 markdown,不要解释):\n"
"{"
'"color_palette": ["#主色1","#主色2","#主色3","#辅色","#点缀色"],'
'"color_tone": "warm|cool|high_sat|low_sat|vintage|fresh|dramatic|bright",'
'"color_filter": "none|warm_vintage|cool_fresh|high_contrast|soft_pastel|dramatic_cinematic",'
'"lighting": "natural|studio|backlit|soft|dramatic|bright_even",'
'"composition": {"closeup_ratio":0.0,"medium_ratio":0.0,"wide_ratio":0.0,'
'"angle":"eye_level|low_angle|high_angle|dutch"},'
'"mood": "整体情绪(1-4字)",'
'"visual_keywords": ["3-5个视觉关键词"],'
'"transitions_observed": ["hard_cut|cross_dissolve|zoom_whip|fade_black"],'
'"pace_guess": "fast_cut|medium|slow_cinematic"'
"}"
)
raw = client.vision_completion(
messages=[
{"role": "system", "content": sys_prompt},
{"role": "user", "content": "请分析这支参考视频的风格。"},
],
images=image_urls,
temperature=0.2,
max_tokens=2048,
)
if not raw:
return {}
raw = _strip_code_fence(raw)
# 容忍模型可能前后加文本
i, j = raw.find("{"), raw.rfind("}")
if i >= 0 and j > i:
return json.loads(raw[i : j + 1])
return {}
except Exception as exc: # noqa: BLE001
logger.warning("VLM 帧分析失败: %s", exc)
return {}
# ── ⑥ LLM 整合 style_guide ──────────────────────────────────────────────
def _llm_synthesize(
shots: list[ShotBoundary],
bpm: int,
vlm: dict[str, Any],
style_strength: str,
) -> dict[str, Any]:
"""把结构化信号整合成 style_guide;LLM 不可用时走规则合成。"""
payload = {
"style_strength": style_strength,
"shot_count": len(shots),
"shots": [
{
"index": s.index,
"start_sec": round(s.start_sec, 2),
"end_sec": round(s.end_sec, 2),
"movement": s.movement,
"intensity": s.intensity,
"transition": s.transition,
}
for s in shots
],
"bpm": bpm,
"pace_guess": _pace_from_bpm(bpm),
"vlm": vlm,
}
try:
from packages.shared.ai_client import get_doubao_client
client = get_doubao_client()
if not client.is_available:
raise RuntimeError("豆包客户端未配置")
sys_prompt = (
"你是资深短视频导演。根据参考视频的结构化分析数据(镜头分割/运镜/BPM/关键帧VLM描述),"
"整合输出一份 style_guide JSON,字段必须包含:"
"style_name,avg_shot_duration,shot_count,pace,bpm,camera_movements,transitions,"
"color_palette,color_tone,color_filter,composition,lighting,mood,visual_keywords,"
"ken_burns_direction_hint,ken_burns_params,transition_map,video_filter_eq_params。"
"严格输出一个合法 JSON 对象,不要 markdown/解释。"
)
user_text = "分析数据:\n" + json.dumps(payload, ensure_ascii=False)
raw = client.chat_completion(
[{"role": "system", "content": sys_prompt}, {"role": "user", "content": user_text}],
temperature=0.3,
max_tokens=4096,
)
if raw:
raw = _strip_code_fence(raw)
i, j = raw.find("{"), raw.rfind("}")
if i >= 0 and j > i:
result = json.loads(raw[i : j + 1])
if isinstance(result, dict) and result.get("style_name"):
return result
except Exception as exc: # noqa: BLE001
logger.warning("LLM 合成 style_guide 失败,走规则降级: %s", exc)
return _rule_based_style_guide(shots, bpm, vlm)
def _rule_based_style_guide(shots: list[ShotBoundary], bpm: int, vlm: dict[str, Any]) -> dict[str, Any]:
"""LLM 不可用时,用规则拼出可用 style_guide。"""
durations = [s.end_sec - s.start_sec for s in shots] or [3.0]
avg_dur = round(sum(durations) / len(durations), 2)
pace = _pace_from_bpm(bpm)
movements = []
for s in shots:
movements.append(
{
"shot_index": s.index + 1,
"movement": s.movement,
"intensity": s.intensity,
"duration": round(s.end_sec - s.start_sec, 2),
"subject_hint": _default_subject_hint(s.movement),
}
)
transitions = []
for i in range(len(shots) - 1):
transitions.append({"between_shot": [i + 1, i + 2], "type": shots[i].transition})
color_palette = vlm.get("color_palette") or ["#E0E0E0", "#333333", "#F5F5F5", "#888888", "#FF6B35"]
color_tone = vlm.get("color_tone") or "bright"
color_filter = vlm.get("color_filter") or "none"
lighting = vlm.get("lighting") or "bright_even"
composition = vlm.get("composition") or {
"closeup_ratio": 0.4,
"medium_ratio": 0.4,
"wide_ratio": 0.2,
"angle": "eye_level",
}
mood = vlm.get("mood") or "明快"
vk = vlm.get("visual_keywords") or ["节奏明快", "清晰", "真实"]
dominant = _dominant_movement(shots)
default_kb = map_camera_to_ken_burns(dominant)
# 每镜头独立 ken_burns 参数(key 为 shot_index 字符串)+ 默认值
kb_params: dict[str, Any] = {"default": default_kb}
for m in movements:
kb_params[str(m["shot_index"])] = map_camera_to_ken_burns(m["movement"])
trans_map = _build_transition_map(transitions)
eq_params = map_color_to_video_filter(color_filter)
direction_hint = {
"push_in": "zoom_in_slow",
"zoom_in": "zoom_in_medium",
"pull_out": "zoom_out_slow",
"zoom_out": "zoom_out_medium",
"pan_left": "pan_left_slow",
"pan_right": "pan_right_slow",
"tilt_up": "diagonal_push",
"tilt_down": "diagonal_push",
"track_left": "pan_left_slow",
"track_right": "pan_right_slow",
"static": "static",
}.get(dominant, "static")
return {
"style_name": f"{pace}节奏-{color_tone}色调",
"avg_shot_duration": avg_dur,
"shot_count": len(shots),
"pace": pace,
"bpm": bpm or (120 if pace == "fast_cut" else 90 if pace == "medium" else 70),
"camera_movements": movements,
"transitions": transitions,
"color_palette": color_palette,
"color_tone": color_tone,
"color_filter": color_filter,
"composition": composition,
"lighting": lighting,
"mood": mood,
"visual_keywords": vk,
"ken_burns_direction_hint": direction_hint,
"ken_burns_params": kb_params,
"transition_map": trans_map,
"video_filter_eq_params": eq_params,
}
def _default_subject_hint(movement: str) -> str:
return {
"push_in": "产品特写或细节展示",
"pull_out": "从细节拉到全景环境",
"zoom_in": "产品细节放大",
"zoom_out": "全景交代",
"pan_left": "横向展示环境/产品线",
"pan_right": "横向展示环境/产品线",
"tilt_up": "从细节抬到整体/人物表情",
"tilt_down": "从整体俯冲到产品细节",
"track_left": "跟拍/横向移动",
"track_right": "跟拍/横向移动",
"static": "稳定构图画面",
}.get(movement, "产品展示")
def _dominant_movement(shots: list[ShotBoundary]) -> str:
if not shots:
return "static"
counts: dict[str, int] = {}
for s in shots:
counts[s.movement] = counts.get(s.movement, 0) + 1
return max(counts, key=counts.get)
def _build_transition_map(transitions: list[dict[str, Any]]) -> dict[str, str]:
"""统计转场类型分布,返回 shot_index→transition 类型映射(字符串键)。"""
m: dict[str, str] = {}
for t in transitions:
pair = t.get("between_shot") or [0, 0]
if len(pair) >= 2:
m[f"{pair[0]}-{pair[1]}"] = t.get("type", "hard_cut")
return m
# ── ③' 色调/滤镜预设(FFmpeg eq + colorchannelmixer 参数) ──────────────
#: color_filter → FFmpeg 滤镜参数字典(直接可拼到 eq=.../colorchannelmixer=...)
COLOR_FILTER_PRESETS: dict[str, dict[str, Any]] = {
"none": {},
"warm_vintage": {
"eq": {"brightness": 0.02, "contrast": 1.05, "saturation": 0.9, "gamma": 1.05},
"colorchannelmixer": {"rr": 1.1, "gg": 0.98, "bb": 0.82, "ra": 0, "ga": 0, "ba": 0, "aa": 1},
},
"cool_fresh": {
"eq": {"brightness": 0.03, "contrast": 1.08, "saturation": 1.05},
"colorchannelmixer": {"rr": 0.9, "gg": 1.0, "bb": 1.12, "ra": 0, "ga": 0, "ba": 0, "aa": 1},
},
"high_contrast": {
"eq": {"brightness": 0.0, "contrast": 1.3, "saturation": 1.2},
"colorchannelmixer": {},
},
"soft_pastel": {
"eq": {"brightness": 0.05, "contrast": 0.92, "saturation": 0.85},
"colorchannelmixer": {"rr": 1.05, "gg": 1.03, "bb": 1.05, "ra": 0, "ga": 0, "ba": 0, "aa": 1},
},
"dramatic_cinematic": {
"eq": {"brightness": -0.03, "contrast": 1.2, "saturation": 0.85},
"colorchannelmixer": {"rr": 1.05, "gg": 0.98, "bb": 0.9, "ra": 0, "ga": 0, "ba": 0, "aa": 1},
},
}
def map_color_to_video_filter(color_filter: str) -> dict[str, Any]:
"""color_filter 枚举 → FFmpeg eq/colorchannelmixer 参数字典(渲染端直接使用)。"""
preset = COLOR_FILTER_PRESETS.get(color_filter) or COLOR_FILTER_PRESETS["none"]
# 返回深拷贝防污染
return json.loads(json.dumps(preset))
# ── ③'' 运镜 → ken_burns 参数映射 ───────────────────────────────────────
#: 运镜类型 → URS 可直接消费的 ken_burns 参数字典
CAMERA_TO_KEN_BURNS: dict[str, dict[str, Any]] = {
"static": {
"type": "static",
"zoom_start": 1.0,
"zoom_end": 1.0,
"pan_x": 0.0,
"pan_y": 0.0,
"duration_factor": 1.0,
},
"push_in": {
"type": "zoom",
"zoom_start": 1.0,
"zoom_end": 1.12,
"pan_x": 0.0,
"pan_y": 0.0,
"duration_factor": 1.0,
},
"zoom_in": {
"type": "zoom",
"zoom_start": 1.0,
"zoom_end": 1.18,
"pan_x": 0.0,
"pan_y": 0.0,
"duration_factor": 1.0,
},
"pull_out": {
"type": "zoom",
"zoom_start": 1.12,
"zoom_end": 1.0,
"pan_x": 0.0,
"pan_y": 0.0,
"duration_factor": 1.0,
},
"zoom_out": {
"type": "zoom",
"zoom_start": 1.18,
"zoom_end": 1.0,
"pan_x": 0.0,
"pan_y": 0.0,
"duration_factor": 1.0,
},
"pan_left": {
"type": "pan",
"zoom_start": 1.05,
"zoom_end": 1.05,
"pan_x": -0.08,
"pan_y": 0.0,
"duration_factor": 1.0,
},
"pan_right": {
"type": "pan",
"zoom_start": 1.05,
"zoom_end": 1.05,
"pan_x": 0.08,
"pan_y": 0.0,
"duration_factor": 1.0,
},
"tilt_up": {
"type": "pan+zoom",
"zoom_start": 1.08,
"zoom_end": 1.14,
"pan_x": 0.0,
"pan_y": -0.05,
"duration_factor": 1.0,
},
"tilt_down": {
"type": "pan+zoom",
"zoom_start": 1.14,
"zoom_end": 1.08,
"pan_x": 0.0,
"pan_y": 0.05,
"duration_factor": 1.0,
},
"track_left": {
"type": "pan",
"zoom_start": 1.05,
"zoom_end": 1.05,
"pan_x": -0.10,
"pan_y": 0.0,
"duration_factor": 1.0,
},
"track_right": {
"type": "pan",
"zoom_start": 1.05,
"zoom_end": 1.05,
"pan_x": 0.10,
"pan_y": 0.0,
"duration_factor": 1.0,
},
}
def map_camera_to_ken_burns(movement: str) -> dict[str, Any]:
"""运镜类型 → URS ken_burns 参数字典。未知类型回退 static。"""
preset = CAMERA_TO_KEN_BURNS.get(movement) or CAMERA_TO_KEN_BURNS["static"]
return json.loads(json.dumps(preset))
# ── 转场 → xfade transition 名称 ────────────────────────────────────────
TRANSITION_TO_XFADE: dict[str, str] = {
"hard_cut": "cut",
"cross_dissolve": "dissolve",
"fade_black": "fadeblack",
"fade": "fade",
"zoom_whip": "zoom",
"slide_left": "slideright", # 画面左移 = 新画面从右滑入
"slide_right": "slideleft",
"wipe_left": "wipeleft",
"wipe_right": "wiperight",
}
def map_transition_to_xfade(transition_type: str) -> str:
"""转场枚举 → TransitionEngine 支持的 xfade 名称;未知回退 cut。"""
return TRANSITION_TO_XFADE.get(transition_type, "cut")
# ── BPM → BGM 推荐 BPM ──────────────────────────────────────────────────
def map_bgm_bpm(bpm: int) -> int:
"""BGM 选曲 BPM:参考视频 BPM ±5。bpm=0 返回 90(默认 medium)。"""
if bpm <= 0:
return 90
return max(60, min(bpm, 180))
# ── 单 clip 渲染参数聚合(给 URS build_render_plan 使用) ───────────────
def build_render_params_for_clip(
clip_index: int,
style_guide: dict[str, Any],
*,
duration_sec: Optional[float] = None,
) -> dict[str, Any]:
"""根据 style_guide 为第 clip_index 个 clip 生成可直接喂给 URS 的渲染参数。"""
shot_idx = clip_index + 1
movements = style_guide.get("camera_movements") or []
movement = "static"
intensity = "low"
for m in movements:
if m.get("shot_index") == shot_idx:
movement = m.get("movement", "static")
intensity = m.get("intensity", "low")
break
ken = map_camera_to_ken_burns(movement)
if intensity == "high":
ken["zoom_end"] = round(ken.get("zoom_end", 1.0) * 1.08, 3)
for k in ("pan_x", "pan_y"):
ken[k] = round(ken.get(k, 0.0) * 1.3, 3)
elif intensity == "low":
for k in ("pan_x", "pan_y"):
ken[k] = round(ken.get(k, 0.0) * 0.6, 3)
transitions = style_guide.get("transitions") or []
trans_type = "hard_cut"
for t in transitions:
pair = t.get("between_shot") or []
if len(pair) >= 2 and pair[0] == shot_idx:
trans_type = t.get("type", "hard_cut")
break
xfade = map_transition_to_xfade(trans_type)
eq = map_color_to_video_filter(style_guide.get("color_filter", "none"))
return {
"ken_burns": ken,
"transition": {"type": xfade, "duration": 0.3 if xfade != "cut" else 0.0},
"video_filter": eq,
"bgm_bpm_hint": map_bgm_bpm(int(style_guide.get("bpm") or 0)),
"duration_sec": duration_sec,
}
# ── 素材本地化(URL/OSS key → 本地临时文件) ────────────────────────────
def _ensure_local_video(reference: str, work_dir: Path) -> Optional[Path]:
"""把 reference(URL/OSS key/本地路径)落到 work_dir 下的本地文件。"""
p = Path(reference)
if p.exists() and p.is_file():
return p
try:
from video_processing.oss_helpers import download_asset
target = work_dir / f"ref_{uuid.uuid4().hex}.mp4"
ok = download_asset(reference, target)
if ok and target.exists() and target.stat().st_size > 0:
return target
except Exception as exc: # noqa: BLE001
logger.warning("download_asset 失败,尝试 http 直连: %s", exc)
if reference.startswith(("http://", "https://")):
try:
import httpx # noqa: PLC0415 - 项目依赖,延迟导入
target = work_dir / f"ref_{uuid.uuid4().hex}.mp4"
with httpx.Client(timeout=20.0, follow_redirects=True) as client:
with client.stream("GET", reference) as resp:
resp.raise_for_status()
with open(target, "wb") as f:
for chunk in resp.iter_bytes(chunk_size=64 * 1024):
f.write(chunk)
if target.exists() and target.stat().st_size > 0:
return target
except Exception as exc: # noqa: BLE001
logger.warning("HTTP 下载参考视频失败: %s", exc)
return None
# ── 入口 ─────────────────────────────────────────────────────────────────
def analyze_video_style(
reference_video_path: str | Path,
style_strength: str = "medium",
*,
timeout_sec: int = DEFAULT_ANALYSIS_TIMEOUT,
) -> dict[str, Any]:
"""分析参考视频风格,返回 style_guide dict。
Args:
reference_video_path: 本地路径、HTTP(S) URL 或 OSS storage key。
style_strength: light | medium | strict。
timeout_sec: 单步超时(秒),默认 60。
Returns:
style_guide dict,详见 STYLE_GUIDE_SCHEMA。任何子步骤失败都会降级,不抛异常。
"""
style_strength = style_strength if style_strength in ("light", "medium", "strict") else "medium"
frames_dir: Optional[Path] = None
local_path: Optional[Path] = None
try:
frames_dir = Path(tempfile.mkdtemp(prefix="vstyle_"))
work_dir = frames_dir # 同一临时根
local_path = _ensure_local_video(str(reference_video_path), work_dir)
if local_path is None:
logger.error("[video_analyzer] 无法获取参考视频: %s", reference_video_path)
return _rule_based_style_guide([], 0, {})
# 资源约束:大小 / 时长
try:
size_mb = local_path.stat().st_size / (1024 * 1024)
if size_mb > MAX_REFERENCE_SIZE_MB:
logger.warning(
"[video_analyzer] 参考视频 %.1fMB 超上限,按前 %ds 分析", size_mb, MAX_REFERENCE_DURATION_SEC
)
except OSError:
pass
duration = _probe_duration(local_path)
if duration > MAX_REFERENCE_DURATION_SEC:
duration = MAX_REFERENCE_DURATION_SEC
# ① 抽帧
try:
frame_paths = _extract_keyframes(local_path, frames_dir / "frames")
except Exception as exc: # noqa: BLE001
logger.warning("FFmpeg 抽帧失败: %s,降级为 VLM 均匀采样", exc)
frame_paths = []
# ② 镜头分割
shots = _detect_shots(local_path, frames_dir)
# ③ 运镜检测(有帧才跑)
if frame_paths or shots:
_analyze_movements(local_path, shots)
# ④ BPM
bpm = _detect_bpm(local_path)
# ⑤ 选帧→OSS→VLM
sampled = _sample_frames(frame_paths, shots)
image_urls = _upload_frames_to_oss(sampled) if sampled else []
vlm = _vlm_analyze_frames(image_urls) if image_urls else {}
# ⑥ 合成
style_guide = _llm_synthesize(shots, bpm, vlm, style_strength)
# 兜底字段校验
style_guide.setdefault("style_strength", style_strength)
style_guide.setdefault("pace", _pace_from_bpm(bpm))
style_guide.setdefault("bpm", bpm)
style_guide.setdefault("shot_count", len(shots))
if shots and "avg_shot_duration" not in style_guide:
durs = [s.end_sec - s.start_sec for s in shots]
style_guide["avg_shot_duration"] = round(sum(durs) / len(durs), 2)
return style_guide
except Exception as exc: # noqa: BLE001
logger.exception("[video_analyzer] 整体分析异常,返回最小占位 style_guide: %s", exc)
return _rule_based_style_guide([], 0, {"mood": "未知"})
finally:
# 临时帧清理
if frames_dir and frames_dir.exists():
shutil.rmtree(frames_dir, ignore_errors=True)