21d7a6f45a
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 3s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 1m31s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 1m49s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 33s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 2m8s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m32s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 1m32s
AI Code Review / AI Code Review (pull_request) Failing after 3m15s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 2m12s
CI/CD Pipeline / Integration Tests (pull_request) Failing after 2m57s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 4m37s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 6m32s
CI/CD Pipeline / CI Gate (pull_request) Failing after 3s
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 1m57s
CI/CD Pipeline / Canary Release to Production (pull_request) Failing after 237h24m9s
CI/CD Pipeline / Build Production Worker Image (pull_request) Failing after 237h24m12s
CI/CD Pipeline / Build Production Web Image (pull_request) Failing after 237h24m12s
CI/CD Pipeline / PR Build Web Image (pull_request) Failing after 237h29m13s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Failing after 237h29m14s
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 237h30m36s
CI/CD Pipeline / Frontend Lint (pull_request) Failing after 237h29m15s
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 237h30m36s
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Failing after 237h30m42s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Failing after 237h30m38s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Failing after 237h30m43s
CI/CD Pipeline / Check push changed paths (pull_request) Failing after 237h30m50s
CI/CD Pipeline / Build Staging Worker Image (pull_request) Failing after 237h30m45s
CI/CD Pipeline / Build Staging Web Image (pull_request) Failing after 237h30m46s
CI/CD Pipeline / Deploy Production (pull_request) Failing after 237h58m45s
CI/CD Pipeline / Build Production API Image (pull_request) Failing after 237h58m48s
CI/CD Pipeline / ACR Image Cleanup (pull_request) Failing after 238h5m11s
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Failing after 238h5m18s
CI/CD Pipeline / Build Staging API Image (pull_request) Failing after 238h5m22s
根因:批量生成的变体 1..N 走 clone_plan_for_variant 只重算起点,
素材/顺序/速度与源 plan 完全相同,导致 N 条成片片段同源(批次查重率 60.1%/33.3%)。
修复(多视频 = 单视频逻辑 × N):
- 新增 packages/domain/variant_plan_selector.py 纯核心:
素材池 shuffle + main 片段顺序洗牌 + _resolve_start_time 起点重选
(镜头洗牌/随机起点/历史区间避让,与单视频同一入口)+ 批次内同素材
区间重叠 >20% 自动重选起点(最多 6 次)
- EditPlanService.reselect_plan_for_variant:源 plan 骨架(clip_type/order/
duration/文案/转场/速度)保留,素材池=源素材∪批次素材,新片段区间
record_used_segments 写回(跨变体/跨任务避让),replace 事务落库
- 正式生成(generation_tasks):创建任务前预生成 N-1 个独立 plan,
变体0保留用户预览 plan;count>1 无源 plan 直接 400(严禁 N 任务兜底
共用同一 plan);任务循环兜底块加批量守卫(缺 plan 500 中断)
- 批量预览(generation_preview):count>1 变体0用源 plan、变体1..N-1
reselect 真实选片(所见即所得);count==1 保持 clone 零回归
- worker 渲染:上传 OSS 与落库查重拆分(_upload_rendered_video/
_record_video_and_dedup);批次任务 batch_similarity>20% 重选 plan
重渲一次(最多1次,重选失败保留首版,首版_v1后缀不覆盖);
封面按 task_id md5 稳定哈希分散选候选帧(非批次仍取首帧)
- dedup_helpers.create_video_record_and_dedup 返回 dict:
{video_count,is_duplicate,batch_similarity,duplicate_of}
测试:新增 24 个单测(纯核心选片差异化/重叠避让/骨架保留、服务层
reselect、重渲阈值判定、封面哈希、批次查重 dict 契约),适配 6 个旧测试,
全量 14469 passed;diff coverage 63%。
81 lines
3.8 KiB
Python
81 lines
3.8 KiB
Python
"""#1743 worker 批量重渲判定 + 封面哈希选帧纯函数测试。
|
||
|
||
- should_rerender_for_batch_dedup:批次首版查重率 >20% 才重渲(非批次/重渲版/无查重率不重渲)
|
||
- pick_batch_cover_index:task_id md5 稳定哈希分散候选帧(同任务稳定、跨任务分散)
|
||
"""
|
||
|
||
from __future__ import annotations
|
||
|
||
import hashlib
|
||
import sys
|
||
from pathlib import Path
|
||
|
||
REPO_ROOT = Path(__file__).resolve().parents[2]
|
||
for sub in ("apps/worker", "apps/api", "packages", ""):
|
||
p = str(REPO_ROOT / sub) if sub else str(REPO_ROOT)
|
||
if p not in sys.path:
|
||
sys.path.insert(0, p)
|
||
|
||
from worker_app.tasks.generation import ( # noqa: E402
|
||
BATCH_RENDER_SIMILARITY_LIMIT,
|
||
pick_batch_cover_index,
|
||
should_rerender_for_batch_dedup,
|
||
)
|
||
|
||
|
||
class TestShouldRerenderForBatchDedup:
|
||
def test_non_batch_never_rerenders(self):
|
||
"""非批次任务(batch_id 为空)任何查重率都不重渲。"""
|
||
assert should_rerender_for_batch_dedup(batch_id="", render_attempt=0, batch_similarity=0.99) is False
|
||
assert should_rerender_for_batch_dedup(batch_id="", render_attempt=0, batch_similarity=None) is False
|
||
|
||
def test_batch_first_version_over_threshold_rerenders(self):
|
||
"""批次首版(attempt=0)查重率 >20% → 重渲。"""
|
||
assert should_rerender_for_batch_dedup(batch_id="batch-1", render_attempt=0, batch_similarity=0.601) is True
|
||
assert should_rerender_for_batch_dedup(batch_id="batch-1", render_attempt=0, batch_similarity=0.21) is True
|
||
|
||
def test_batch_first_version_under_threshold_no_rerender(self):
|
||
"""批次首版查重率 ≤20% → 不重渲。"""
|
||
assert should_rerender_for_batch_dedup(batch_id="batch-1", render_attempt=0, batch_similarity=0.20) is False
|
||
assert should_rerender_for_batch_dedup(batch_id="batch-1", render_attempt=0, batch_similarity=0.0) is False
|
||
assert should_rerender_for_batch_dedup(batch_id="batch-1", render_attempt=0, batch_similarity=0.05) is False
|
||
|
||
def test_rerendered_version_never_rerenders_again(self):
|
||
"""重渲版(attempt=1)即使仍超阈值也不再重渲(最多重渲一次)。"""
|
||
assert should_rerender_for_batch_dedup(batch_id="batch-1", render_attempt=1, batch_similarity=0.99) is False
|
||
|
||
def test_missing_similarity_no_rerender(self):
|
||
"""查重率缺失(None,非批次查重路径)→ 不重渲。"""
|
||
assert should_rerender_for_batch_dedup(batch_id="batch-1", render_attempt=0, batch_similarity=None) is False
|
||
|
||
def test_threshold_constant_is_20_percent(self):
|
||
assert BATCH_RENDER_SIMILARITY_LIMIT == 0.20
|
||
|
||
|
||
class TestPickBatchCoverIndex:
|
||
def test_stable_for_same_task(self):
|
||
"""同一 task_id 多次调用结果稳定(重试封面不变)。"""
|
||
first = pick_batch_cover_index("task-abc", 3)
|
||
for _ in range(5):
|
||
assert pick_batch_cover_index("task-abc", 3) == first
|
||
|
||
def test_zero_or_one_candidate_returns_zero(self):
|
||
assert pick_batch_cover_index("task-x", 0) == 0
|
||
assert pick_batch_cover_index("task-x", 1) == 0
|
||
|
||
def test_index_within_range(self):
|
||
for i in range(20):
|
||
idx = pick_batch_cover_index(f"task-{i}", 3)
|
||
assert 0 <= idx < 3
|
||
|
||
def test_matches_md5_formula(self):
|
||
"""与主流程公式一致:md5(task_id) % 候选数。"""
|
||
task_id = "task-formula-check"
|
||
expected = int(hashlib.md5(task_id.encode()).hexdigest(), 16) % 3
|
||
assert pick_batch_cover_index(task_id, 3) == expected
|
||
|
||
def test_batch_tasks_spread_across_frames(self):
|
||
"""30 个批次任务在 3 个候选帧上分散(不能全部落在 frame_0——旧 bug 回归守卫)。"""
|
||
indexes = {pick_batch_cover_index(f"batch-task-{i}", 3) for i in range(30)}
|
||
assert len(indexes) >= 2, f"封面帧应分散到多个帧位,实际全部落在: {indexes}"
|