feat(generation): #2024 视频渲染后延迟到封面确认才入库(新增 AWAITING_COVER 状态 + finalize 接口) #2026

Merged
auto-approve-bot merged 2 commits from feat/defer-video-finalization-until-cover-confirmed into develop 2026-09-24 12:34:44 +08:00
Owner

背景

需求来自 #2024:Worker 渲染视频并上传 OSS 后,不要立刻创建 GeneratedVideo 成品记录,而是停留在「待确认封面」中间态;用户在 Step5 选完封面/自动封面、点「完成」时才调用 finalize 接口正式入库。AI 数字人流程独立不受影响。

核心改动

领域模型

  • GenerationTaskStatus 新增 AWAITING_COVER(非终态)
  • 新增 mark_awaiting_cover():progress=100,清空 error_message,不设 completed_at
  • 状态转换:running → {awaiting_cover, completed, failed, cancelled};awaiting_cover → {completed, failed, cancelled}
  • _missing_ 兼容别名:waiting_cover/video_ready/rendered/pending_cover
  • 保留 running→completed 兜底兼容(历史代码/测试)

Worker(apps/worker/worker_app/tasks/generation.py)

  • _record_video_and_dedup → _precompute_render_metadata:调用新的 compute_render_fingerprint_and_dedup,返回可序列化 dict(指纹+查重结果),不写任何 DB 记录
  • 删除运行时「批次内相似度>20% 重选 plan 重渲」分支:未 finalize 的任务互相看不到 GeneratedVideo,查重统一在 finalize 阶段对全量完成记录做
  • 预计算结果序列化写入 task.extra_meta["rendered_output"]
  • 最终状态:mark_awaiting_cover 代替 mark_completed

查重工具(apps/worker/video_processing/dedup_helpers.py)

  • 新函数 compute_render_fingerprint_and_dedup(video_path, ...) → dict:本地指纹计算 + 历史查重 + 批次查重,返回可序列化结果(含 fingerprint_dict、fingerprint_chunks 列表、is_duplicate/duplicate_of/duplicate_rate/match_count/visual_similarity/video_fingerprint_md5)
  • 旧函数 create_video_record_and_dedup() 保留兼容:支持直接传 pre_dedup_result 复用预计算结果,避免重复读本地视频
  • VideoFingerprint.from_dict() 新增类方法,用于 finalize 阶段从 dict 还原指纹对象

应用层(新增 packages/application/generated_video_finalize.py)

  • RenderedOutput dataclass + from_dict():承载 worker 预计算结果
  • finalize_generated_video(task, session, effective_cover_url) → dict:读 meta → 构造 GeneratedVideo → 批量写入 VideoFingerprintChunkModel → video_repo.create() → commit,不依赖任何 worker 模块导入

API Service(新增 apps/api/app/services/generation_finalize_service.py)

  • GenerationFinalizeService.finalize_task(task_id, user_id, cover_url=None)
    • 权限/存在性校验
    • 幂等:若该 task 已存在 GeneratedVideo,直接补 mark_completed 返回(前端重复点击安全)
    • 状态门:只接受 awaiting_cover(兼容历史 completed 数据)
    • 封面决策:显式传参 > task.cover_url
    • 调 finalize_generated_video → mark_completed(result_count=1) → 清理 extra_meta.rendered_output → commit

API 端点(apps/api/app/api/routes/generation_tasks.py)

  • 新增 POST /api/v1/generation/tasks/{task_id}/finalize
    • Body: { "cover_url": "..." }(可选;不传则用 task.cover_url)
    • 响应:{ video_id, cover_url, file_url, status="success", is_duplicate }(file_url 是签名下载链接)
    • 404 task 不存在、403 无权限、400 状态非法/封面缺失
  • 调整 confirm_generation:confirm 预览后 mark_confirmed 不再隐式 mark_completed,任务停留在 awaiting_cover;历史已 completed 的预览任务回退到 awaiting_cover
  • 预览复用条件从 source_task.is_completed 放宽到 (completed, awaiting_cover)

预览/任务中心兼容

  • GET /preview/{task_id}:awaiting_cover 状态下从 extra_meta.rendered_output 构造轻量 _PreviewVideo(file_url/duration/file_size),finalize 前前端即可预览播放
  • 任务中心 step map:awaiting_cover → 等待确认封面;状态筛选加入 awaiting_cover

不需要 DB 迁移

GenerationTaskModel.status 字段为 String(20) 无 CHECK 约束,新增枚举值可直接写入。

未触碰范围(明确保证)

  • AI 数字人流程:AiAvatarRenderJob 表和 ai_avatar_render_service.finalize_job 完全独立,未做任何修改
  • PR #2023 字号缩放相关文件(ass_subtitle_builder/video_filter_builder/subtitle_generator)未触碰,不回退修复
  • 临时文件清理:交给现有定时清理任务,本次不做
  • 不做生产发版

测试

  • 新增 tests/unit/test_finalize_generation.py:13 个用例覆盖状态枚举/属性/转换、mark_awaiting_cover、RenderedOutput 解析、finalize 用例(成功/缺 meta/封面兜底)
  • 更新 tests/unit/domain/test_generation_task.py:6 个状态断言
  • 全量单测:15978 passed, 28 skipped(与 PR #2023 基线一致,无回归)
  • black + isort + ruff 全部通过

前端联调提示

  1. Step5 「完成」按钮调用 POST /api/v1/generation/tasks/{task_id}/finalize
  2. 自动生成封面完成后,封面 URL 已有(沿用现有 cover_url 机制),finalize 请求可省略 cover_url 参数
  3. 手动选封面时,把选定封面 URL 作为 cover_url 传过去
  4. finalize 返回的 file_url 是签名链接,可直接用于播放/下载;is_duplicate=true 时前端可按需提示
  5. 渲染完成(轮询到 status=awaiting_cover)后即可调 preview 接口播放视频,不用等 finalize
## 背景 需求来自 #2024:Worker 渲染视频并上传 OSS 后,不要立刻创建 GeneratedVideo 成品记录,而是停留在「待确认封面」中间态;用户在 Step5 选完封面/自动封面、点「完成」时才调用 finalize 接口正式入库。AI 数字人流程独立不受影响。 ## 核心改动 ### 领域模型 - `GenerationTaskStatus` 新增 `AWAITING_COVER`(非终态) - 新增 `mark_awaiting_cover()`:progress=100,清空 error_message,**不设** `completed_at` - 状态转换:`running → {awaiting_cover, completed, failed, cancelled}`;`awaiting_cover → {completed, failed, cancelled}` - `_missing_` 兼容别名:`waiting_cover/video_ready/rendered/pending_cover` - 保留 `running→completed` 兜底兼容(历史代码/测试) ### Worker(`apps/worker/worker_app/tasks/generation.py`) - `_record_video_and_dedup` → `_precompute_render_metadata`:调用新的 `compute_render_fingerprint_and_dedup`,返回可序列化 dict(指纹+查重结果),**不写任何 DB 记录** - 删除运行时「批次内相似度>20% 重选 plan 重渲」分支:未 finalize 的任务互相看不到 GeneratedVideo,查重统一在 finalize 阶段对全量完成记录做 - 预计算结果序列化写入 `task.extra_meta["rendered_output"]` - 最终状态:`mark_awaiting_cover` 代替 `mark_completed` ### 查重工具(`apps/worker/video_processing/dedup_helpers.py`) - 新函数 `compute_render_fingerprint_and_dedup(video_path, ...) → dict`:本地指纹计算 + 历史查重 + 批次查重,返回可序列化结果(含 `fingerprint_dict`、`fingerprint_chunks` 列表、`is_duplicate/duplicate_of/duplicate_rate/match_count/visual_similarity/video_fingerprint_md5`) - 旧函数 `create_video_record_and_dedup()` 保留兼容:支持直接传 `pre_dedup_result` 复用预计算结果,避免重复读本地视频 - `VideoFingerprint.from_dict()` 新增类方法,用于 finalize 阶段从 dict 还原指纹对象 ### 应用层(新增 `packages/application/generated_video_finalize.py`) - `RenderedOutput` dataclass + `from_dict()`:承载 worker 预计算结果 - `finalize_generated_video(task, session, effective_cover_url) → dict`:读 meta → 构造 `GeneratedVideo` → 批量写入 `VideoFingerprintChunkModel` → `video_repo.create()` → commit,不依赖任何 worker 模块导入 ### API Service(新增 `apps/api/app/services/generation_finalize_service.py`) - `GenerationFinalizeService.finalize_task(task_id, user_id, cover_url=None)` - 权限/存在性校验 - 幂等:若该 task 已存在 GeneratedVideo,直接补 mark_completed 返回(前端重复点击安全) - 状态门:只接受 `awaiting_cover`(兼容历史 `completed` 数据) - 封面决策:显式传参 > task.cover_url - 调 finalize_generated_video → mark_completed(result_count=1) → 清理 extra_meta.rendered_output → commit ### API 端点(`apps/api/app/api/routes/generation_tasks.py`) - **新增** `POST /api/v1/generation/tasks/{task_id}/finalize` - Body: `{ "cover_url": "..." }`(可选;不传则用 task.cover_url) - 响应:`{ video_id, cover_url, file_url, status="success", is_duplicate }`(file_url 是签名下载链接) - 404 task 不存在、403 无权限、400 状态非法/封面缺失 - 调整 `confirm_generation`:confirm 预览后 `mark_confirmed` 不再隐式 `mark_completed`,任务停留在 `awaiting_cover`;历史已 completed 的预览任务回退到 awaiting_cover - 预览复用条件从 `source_task.is_completed` 放宽到 `(completed, awaiting_cover)` ### 预览/任务中心兼容 - `GET /preview/{task_id}`:awaiting_cover 状态下从 `extra_meta.rendered_output` 构造轻量 _PreviewVideo(file_url/duration/file_size),finalize 前前端即可预览播放 - 任务中心 step map:`awaiting_cover → 等待确认封面`;状态筛选加入 awaiting_cover ## 不需要 DB 迁移 `GenerationTaskModel.status` 字段为 `String(20)` 无 CHECK 约束,新增枚举值可直接写入。 ## 未触碰范围(明确保证) - AI 数字人流程:`AiAvatarRenderJob` 表和 `ai_avatar_render_service.finalize_job` 完全独立,未做任何修改 - PR #2023 字号缩放相关文件(`ass_subtitle_builder`/`video_filter_builder`/`subtitle_generator`)未触碰,不回退修复 - 临时文件清理:交给现有定时清理任务,本次不做 - 不做生产发版 ## 测试 - 新增 `tests/unit/test_finalize_generation.py`:13 个用例覆盖状态枚举/属性/转换、mark_awaiting_cover、RenderedOutput 解析、finalize 用例(成功/缺 meta/封面兜底) - 更新 `tests/unit/domain/test_generation_task.py`:6 个状态断言 - 全量单测:**15978 passed, 28 skipped**(与 PR #2023 基线一致,无回归) - black + isort + ruff 全部通过 ## 前端联调提示 1. Step5 「完成」按钮调用 `POST /api/v1/generation/tasks/{task_id}/finalize` 2. 自动生成封面完成后,封面 URL 已有(沿用现有 cover_url 机制),finalize 请求可省略 cover_url 参数 3. 手动选封面时,把选定封面 URL 作为 cover_url 传过去 4. finalize 返回的 `file_url` 是签名链接,可直接用于播放/下载;`is_duplicate=true` 时前端可按需提示 5. 渲染完成(轮询到 status=awaiting_cover)后即可调 preview 接口播放视频,不用等 finalize
xiaoxia added 1 commit 2026-09-24 11:30:21 +08:00
feat(generation): #2024 defer video finalization until cover confirmed
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m22s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m32s
AI Code Review / AI Code Review (pull_request) Successful in 6m51s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 10m53s
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 0s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 30s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 29s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 3m46s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 4m59s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 5m33s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 10m57s
CI/CD Pipeline / Unit Tests (pull_request) Failing after 11m4s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Failing after 1s
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
48f6f49f7b
Two-phase pipeline: worker renders and precomputes dedup metadata, but does
not create GeneratedVideo until user confirms cover in Step 5.

Domain
- Add GenerationTaskStatus.AWAITING_COVER (non-terminal)
- Add mark_awaiting_cover(): progress=100, clears error, leaves completed_at unset
- Transitions: running → {awaiting_cover, completed, failed, cancelled};
  awaiting_cover → {completed, failed, cancelled}
- _missing_ aliases: waiting_cover/video_ready/rendered/pending_cover
- Keep running→completed for backward compat / legacy paths

Worker (apps/worker/worker_app/tasks/generation.py)
- Replace _record_video_and_dedup with _precompute_render_metadata:
  calls compute_render_fingerprint_and_dedup, returns dict without DB writes
- Remove runtime batch-rerender decision (should_rerender_for_batch_dedup path);
  dedup now happens at finalize time against all finished records
- Persist precomputed metadata into task.extra_meta['rendered_output']
- Final status: mark_awaiting_cover instead of mark_completed

Dedup helpers (apps/worker/video_processing/dedup_helpers.py)
- New compute_render_fingerprint_and_dedup(video_path,...): local fingerprint
  + historical dedup + batch dedup, returns fully serializable dict
  (fingerprint_dict, fingerprint_chunks list, is_duplicate, duplicate_of, etc.)
- create_video_record_and_dedup() retained for compat/tests; now supports
  pre_dedup_result to reuse worker-precomputed data without re-reading video
- VideoFingerprint.from_dict() added to reconstruct from serialized form

Application layer (packages/application/generated_video_finalize.py)
- RenderedOutput dataclass + from_dict() for deserializing worker output
- finalize_generated_video(): creates GeneratedVideo, bulk-inserts
  VideoFingerprintChunk rows, commits; API-layer free

API service (apps/api/app/services/generation_finalize_service.py)
- GenerationFinalizeService.finalize_task(task_id, user_id, cover_url):
  * permission / existence check
  * idempotent: if GeneratedVideo already exists for this task, just ensure
    task is marked completed and return (safe for double-click)
  * status gate: only awaiting_cover (or legacy completed) accepted
  * cover resolution: explicit cover_url arg > task.cover_url
  * delegates to finalize_generated_video, marks task completed,
    clears extra_meta['rendered_output']

API endpoint (apps/api/app/api/routes/generation_tasks.py)
- POST /api/v1/generation/tasks/{task_id}/finalize
- Body: { cover_url?: string }; returns video_id/cover_url/file_url/is_duplicate
- Adjust confirm_generation fast path: mark_confirmed keeps task in
  awaiting_cover (don't auto-complete); historical 'completed' previews
  migrated back to awaiting_cover
- Preview-reuse accepts awaiting_cover tasks

Preview / task center
- GET /preview/{task_id} constructs lightweight _PreviewVideo from
  extra_meta.rendered_output when task is awaiting_cover
- Task center step map: awaiting_cover → 等待确认封面; status filter includes it

Tests
- tests/unit/test_finalize_generation.py: 13 new tests covering status
  transitions, mark_awaiting_cover, RenderedOutput parsing, finalize use case
  (success / missing metadata / cover fallback)
- test_generation_task.py updated for 6th status value
- Full suite: 15978 passed, 28 skipped (matches #2023 baseline, no regressions)
- black/isort/ruff clean

Out of scope (intentionally untouched)
- AI avatar pipeline uses separate AiAvatarRenderJob; finalize_job and
  /{job_id}/finalize are unchanged
- PR #2023 subtitle font scaling files (ass_subtitle_builder,
  video_filter_builder, subtitle_generator) not modified
- No DB migration: GenerationTaskModel.status is String(20) without CHECK
- Temp file cleanup: leave to existing periodic job

🚀 预览环境已部署

项目 详情
PR号 #2026
预览链接 https://pr-2026.preview.xiaoxiajianji.com
API环境 staging

💡 预览环境使用 staging API 数据,请勿在预览环境中操作重要数据。

🔄 每次提交新代码后预览环境会自动更新。

🗑️ PR 关闭或合并后,预览环境会自动清理。

🚀 **预览环境已部署** | 项目 | 详情 | |------|------| | PR号 | #2026 | | 预览链接 | [https://pr-2026.preview.xiaoxiajianji.com](https://pr-2026.preview.xiaoxiajianji.com) | | API环境 | staging | > 💡 预览环境使用 staging API 数据,请勿在预览环境中操作重要数据。 > > 🔄 每次提交新代码后预览环境会自动更新。 > > 🗑️ PR 关闭或合并后,预览环境会自动清理。
xiaoxia added 1 commit 2026-09-24 12:20:57 +08:00
test(generation): #2024 add service-layer + use-case coverage for finalize flow
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 3s
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 36s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 38s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m58s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 3m59s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 4m42s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 4m40s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 6m4s
AI Code Review / AI Code Review (pull_request) Successful in 6m43s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 11m28s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 12m58s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 2s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 9m0s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 28s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 50s
4996ddc7d0
Adds tests/unit/test_generation_finalize_service.py covering:
- GenerationFinalizeService: TaskNotFound (404), InvalidTaskStatus (400),
  idempotent path (existing GeneratedVideo → cover update + mark_completed),
  already-completed idempotent skip, RenderedOutputMissing error,
  success path (video created, rendered_output cleared, task completed),
  cover fallback to task.cover_url, explicit cover_url override (with strip)
- generated_video_finalize use case: RenderedOutput.from_dict non-dict
  rejection, _safe_float/_safe_int edge cases, non-dict fingerprint chunk
  skip + bulk_save, missing file_url raises, None fingerprint_chunks
  skips bulk_save, name fallback when empty

Bumps unit test count by 15; full suite 15993 passed. Also addresses
CI diff-coverage gate (60%) that was failing because the new service
file had 0% coverage.
auto-approve-bot merged commit 23fe5f9822 into develop 2026-09-24 12:34:44 +08:00
auto-approve-bot deleted branch feat/defer-video-finalization-until-cover-confirmed 2026-09-24 12:34:47 +08:00

🗑️ 预览环境已清理

PR #2026 已关闭或合并,对应的预览环境已被清理。

如有需要,可以重新打开 PR 来重新生成预览环境。

🗑️ **预览环境已清理** PR #2026 已关闭或合并,对应的预览环境已被清理。 > 如有需要,可以重新打开 PR 来重新生成预览环境。
Sign in to join this conversation.