fix: 渲染产物临时目录不在render_plan中提前清理,改由调用方上传后清理
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 32s
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 33s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 1m52s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m55s
CI/CD Pipeline / Validate - Migration (alembic) (pull_request) Successful in 2m15s
CI/CD Pipeline / Validate - Type Check (mypy) (pull_request) Successful in 2m20s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 2m7s
CI/CD Pipeline / Validate - Code Quality (pull_request) Has been cancelled
CI/CD Pipeline / Unit Tests (pull_request) Has been cancelled
CI/CD Pipeline / Integration Tests (pull_request) Has been cancelled
CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / CI Gate (pull_request) Has been cancelled
AI Code Review / AI Code Review (pull_request) Has been cancelled
PR Automation / Auto Approve on CI Green (pull_request) Has been cancelled

根因:render_plan的finally块在返回前shutil.rmtree清理了临时目录,
但generation.py还需要访问其中的文件进行OSS上传,导致FileNotFoundError。

修复:
1. RenderAdapterResult增加temp_dir字段
2. render_plan成功时将temp_dir传给result并置None阻止finally清理
3. render_plan失败时finally仍正常清理(异常路径不受影响)
4. generation.py在_upload_and_record完成后清理临时目录

新增4个回归测试验证清理时序正确。
This commit is contained in:
CI Bot
2026-08-24 17:33:01 +08:00
parent 7fdea8301c
commit 548eabbd2c
3 changed files with 82 additions and 3 deletions
@@ -84,6 +84,7 @@ class RenderAdapterResult:
cover_candidates: list[dict] | None = (
None # 封面候选帧 [{"image_url": "...", "frame_time": 5.0, "storage_key": "..."}]
)
temp_dir: str | None = None # 调用方负责清理的临时目录路径
def __post_init__(self):
if self.rendered_clip_ids is None:
@@ -200,7 +201,7 @@ class RenderAdapter:
self._report_progress(progress_cb, 35.0, "准备 BGM 音频")
# 3~6. 统一渲染核心流程(BGM + ASR + 渲染 + 缩略图 + 上传)
return self._do_render(
result = self._do_render(
plan=plan,
clips=ready_clips,
asset_path_map=asset_path_map,
@@ -212,6 +213,10 @@ class RenderAdapter:
failed_clip_ids=failed_clip_ids,
voiceover_audio_path=voiceover_audio_path,
)
# 将临时目录路径传递给调用方,由调用方负责清理
result.temp_dir = temp_dir
temp_dir = None # 成功时不清理,由调用方清理
return result
except subprocess.CalledProcessError as exc:
stderr_text = (exc.stderr or "").strip()
+12 -2
View File
@@ -578,8 +578,9 @@ def _render_from_edit_plan(
output_path = result.output_path
cover_candidates = getattr(result, "cover_candidates", None)
temp_dir = getattr(result, "temp_dir", None)
return output_path, result.duration, cover_candidates, voiceover_path
return output_path, result.duration, cover_candidates, voiceover_path, temp_dir
finally:
db.close()
@@ -690,7 +691,7 @@ def generate_video(self, task_id: str) -> dict:
gen_task.append_log("渲染模式", "从草稿数据渲染(与预览一致)")
_flush_logs(task_id, gen_task)
output_path, render_duration, cover_candidates, voiceover_tmp_path = _render_from_edit_plan(
output_path, render_duration, cover_candidates, voiceover_tmp_path, render_temp_dir = _render_from_edit_plan(
task_id=task_id,
source_edit_plan_id=source_edit_plan_id,
task_info=task_info,
@@ -725,6 +726,15 @@ def generate_video(self, task_id: str) -> dict:
_update_task_progress(task_id, 95, "上传完成")
# ── 4.1 清理渲染临时目录 ────────────────────────────────────────────
if render_temp_dir:
import shutil as _shutil
try:
_shutil.rmtree(render_temp_dir, ignore_errors=True)
logger.info("[task_id=%s] 渲染临时目录已清理: %s", task_id, render_temp_dir)
except Exception:
logger.warning("[task_id=%s] 渲染临时目录清理失败: %s", task_id, render_temp_dir, exc_info=True)
# ── 4.5 封面帧持久化 ────────────────────────────────────────────
try:
if cover_candidates:
+64
View File
@@ -0,0 +1,64 @@
"""
回归测试渲染产物临时目录不在 render_plan 中提前清理
根因render_plan finally 块在返回前清理了临时目录
generation.py 还需要访问其中的文件进行 OSS 上传
修复将清理责任交给调用方generation.pyrender_plan 只在失败时清理
"""
import ast
def test_render_adapter_result_has_temp_dir_field():
"""RenderAdapterResult 包含 temp_dir 字段"""
with open("apps/worker/video_processing/render_adapter.py") as f:
source = f.read()
tree = ast.parse(source)
for node in ast.walk(tree):
if isinstance(node, ast.ClassDef) and node.name == "RenderAdapterResult":
# 检查类体中是否有 temp_dir 的赋值(dataclass field
for item in node.body:
if isinstance(item, ast.AnnAssign) and isinstance(item.target, ast.Name):
if item.target.id == "temp_dir":
return
raise AssertionError("RenderAdapterResult 缺少 temp_dir 字段")
def test_render_plan_does_not_cleanup_on_success():
"""render_plan 成功时不在 finally 中清理临时目录"""
with open("apps/worker/video_processing/render_adapter.py") as f:
source = f.read()
# 验证成功路径中 temp_dir 被置为 None(阻止 finally 清理)
assert "temp_dir = None # 成功时不清理" in source or "temp_dir = None" in source, (
"render_plan 成功时应将 temp_dir 置为 None 以阻止 finally 清理"
)
def test_render_plan_passes_temp_dir_to_result():
"""render_plan 将 temp_dir 传递给返回结果"""
with open("apps/worker/video_processing/render_adapter.py") as f:
source = f.read()
assert "result.temp_dir = temp_dir" in source, (
"render_plan 应将 temp_dir 设置到 result 上"
)
def test_generation_cleans_up_temp_dir():
"""generation.py 在上传完成后清理临时目录"""
with open("apps/worker/worker_app/tasks/generation.py") as f:
source = f.read()
# 验证 _render_from_edit_plan 返回 temp_dir
assert "render_temp_dir" in source, "generation.py 应接收 render_temp_dir"
# 验证有清理逻辑
assert "rmtree(render_temp_dir" in source, "generation.py 应清理 render_temp_dir"
# 验证清理发生在上传之后(通过查找顺序)
upload_pos = source.find("_upload_and_record")
cleanup_pos = source.find("rmtree(render_temp_dir")
assert upload_pos > 0 and cleanup_pos > upload_pos, (
"清理临时目录应在 _upload_and_record 之后执行"
)