fix(worker+api): batch title vfb alignment, race-free config, auto-queue (3 bugs) #2097

Merged

xiaoxia merged 4 commits from fix/gpu-direct-title-vfb-alignment into develop 2026-09-29 19:51:40 +08:00

Conversation 2 Commits 4 Files Changed 13

+467 -541

xiaoxia commented 2026-09-29 18:08:56 +08:00

Owner

Copy Link

Copy Source

背景

staging 17:30-17:43 用户提交 6 个批量视频测试，反馈：

1. 渲染视频标题与模板配置视觉不一致（PR #2095 已修 baseline/缩放，但默认常量未对齐 CPU vfb 路径）
2. 批量提交超 3 个任务直接 429 拒绝，无法一次提交一批
3. 日志排查发现同 plan 多任务并发时 title_config 可能被覆盖（潜在 P1 竞态）

本 PR 一次性修复以下 3 个问题：

---

Fix 1: GPU 直连标题默认值对齐 CPU vfb 路径（视觉一致性）

参数	PR #2095 取值 (GPU)	CPU vfb 取值	schema 默认	本 PR
默认字号	36@720p	48@720p	48	48
top/bottom 边距	40@720p	50@720p	—	50
字幕底部边距	60@720p	50@720p	—	50
仿粗描边	同色 1px（白字+白边=隐形，赤字+红边=重影）	黑 2px（#2001 有意选择）	—	黑 2px
margin_top 语义	绝对 y 坐标	base+叠加	—	base+叠加
subtitle stroke	s_borderw 未初始化 F821	—	—	正确初始化+解析

---

Fix 2 (Bug A, P1): 同 plan 多任务并发渲染时 title_config 竞态覆盖

根因：_sync_task_config_to_plan() 把每个任务的 title_config/bgm/分辨率写入共享的 plan.config["title"] 行，RenderAdapter.render_plan() 再从 plan.config 读取渲染；并发任务 B 写入会覆盖任务 A 已写入的值，任务 A 可能读到任务 B 的标题文字/样式。

修复：

• 新增 task_config_override 参数贯穿 render_plan → _do_render → _prepare_bgm → UnifiedRenderService
• URS 新增 _effective_config() 方法：deep-copy plan.config 后 merge 任务级 override（title/bgm/export）
• 渲染路径所有读 self.plan.config 的位置（title/subtitle/bgm/export/TTS）改为 self._effective_config()
• Worker 侧替换 _sync_task_config_to_plan 为 _build_task_config_override（深拷贝 task_info，含字段归一化）+ _download_voice_for_task（独立临时文件）；不再写 plan.config
• plan.config 保持 API writeback 写入的状态，渲染用任务级 override，彻底消除竞态

---

Fix 3 (Bug B, P0): USER_PENDING_LIMIT=3 429 硬拒 → 自动排队

根因：批量预检查 + safe_enqueue 双重用 USER_PENDING_LIMIT=3 硬拒绝，用户一次提交 6 个任务时第 4 个起 429。

修复：

• USER_PENDING_LIMIT: 3 → 20（软上限，反滥用兜底，支持 6-10 批量）
• 批量创建/重试/单视频确认三个入口：移除用户级 429 预检查，只保留全局 GLOBAL_PENDING_LIMIT=20 的 503 系统繁忙保护
• safe_enqueue_generation_task：用户级超限改为 warning 日志，仍入队；全局超限仍回滚失败
• 入队后兜底校验：用户超限仅日志不回滚
• Worker GENERATION_CONCURRENCY=2 保持不变，Celery 队列自然消费

---

批量场景排查结论

• 配置透传：每个变体独立 plan（variant 0=clone，1..N-1=reselect），config=dict(source.config or {}) 深拷贝；title text/voice/cover 通过 titles[]/voice_library_ids[]/cover_urls[] per-variant 覆盖；title 样式/BGM 共享（设计如此，schema 无 title_configs[]）。链路正确。
• Worker 并发：generation 队列 concurrency=2，6 个任务分 3 批自然消费（每批 2 个并行）。
• 配音独立性：variant_voice_resolver 严格守卫（#1749），每个任务独立下载 voiceover 到临时文件，per-video 独立。

测试

• tests/unit/test_gpu_direct_pipeline.py: 30 passed
• tests/unit/test_render_adapter_pure.py: 18 passed
• ruff check/format clean
• py_compile clean

## 背景 staging 17:30-17:43 用户提交 6 个批量视频测试，反馈： 1. 渲染视频标题与模板配置视觉不一致（PR #2095 已修 baseline/缩放，但默认常量未对齐 CPU vfb 路径） 2. 批量提交超 3 个任务直接 429 拒绝，无法一次提交一批 3. 日志排查发现同 plan 多任务并发时 title_config 可能被覆盖（潜在 P1 竞态） 本 PR 一次性修复以下 3 个问题： --- ## Fix 1: GPU 直连标题默认值对齐 CPU vfb 路径（视觉一致性） | 参数 | PR #2095 取值 (GPU) | CPU vfb 取值 | schema 默认 | 本 PR | |---|---|---|---|---| | 默认字号 | 36@720p | 48@720p | 48 | 48 | | top/bottom 边距 | 40@720p | 50@720p | — | 50 | | 字幕底部边距 | 60@720p | 50@720p | — | 50 | | 仿粗描边 | 同色 1px（白字+白边=隐形，赤字+红边=重影） | 黑 2px（#2001 有意选择） | — | 黑 2px | | margin_top 语义 | 绝对 y 坐标 | base+叠加 | — | base+叠加 | | subtitle stroke | s_borderw 未初始化 F821 | — | — | 正确初始化+解析 | --- ## Fix 2 (Bug A, P1): 同 plan 多任务并发渲染时 title_config 竞态覆盖 根因：`_sync_task_config_to_plan()` 把每个任务的 title_config/bgm/分辨率写入共享的 `plan.config["title"]` 行，`RenderAdapter.render_plan()` 再从 `plan.config` 读取渲染；并发任务 B 写入会覆盖任务 A 已写入的值，任务 A 可能读到任务 B 的标题文字/样式。 修复： - 新增 `task_config_override` 参数贯穿 `render_plan → _do_render → _prepare_bgm → UnifiedRenderService` - URS 新增 `_effective_config()` 方法：deep-copy `plan.config` 后 merge 任务级 override（title/bgm/export） - 渲染路径所有读 `self.plan.config` 的位置（title/subtitle/bgm/export/TTS）改为 `self._effective_config()` - Worker 侧替换 `_sync_task_config_to_plan` 为 `_build_task_config_override`（深拷贝 task_info，含字段归一化）+ `_download_voice_for_task`（独立临时文件）；不再写 `plan.config` - plan.config 保持 API writeback 写入的状态，渲染用任务级 override，彻底消除竞态 --- ## Fix 3 (Bug B, P0): USER_PENDING_LIMIT=3 429 硬拒 → 自动排队 根因：批量预检查 + safe_enqueue 双重用 `USER_PENDING_LIMIT=3` 硬拒绝，用户一次提交 6 个任务时第 4 个起 429。 修复： - `USER_PENDING_LIMIT: 3 → 20`（软上限，反滥用兜底，支持 6-10 批量） - 批量创建/重试/单视频确认三个入口：移除用户级 429 预检查，只保留全局 `GLOBAL_PENDING_LIMIT=20` 的 503 系统繁忙保护 - `safe_enqueue_generation_task`：用户级超限改为 warning 日志，仍入队；全局超限仍回滚失败 - 入队后兜底校验：用户超限仅日志不回滚 - Worker GENERATION_CONCURRENCY=2 保持不变，Celery 队列自然消费 --- ## 批量场景排查结论 - **配置透传**：每个变体独立 plan（variant 0=clone，1..N-1=reselect），`config=dict(source.config or {})` 深拷贝；title text/voice/cover 通过 `titles[]/voice_library_ids[]/cover_urls[]` per-variant 覆盖；title 样式/BGM 共享（设计如此，schema 无 title_configs[]）。链路正确。 - **Worker 并发**：generation 队列 concurrency=2，6 个任务分 3 批自然消费（每批 2 个并行）。 - **配音独立性**：variant_voice_resolver 严格守卫（#1749），每个任务独立下载 voiceover 到临时文件，per-video 独立。 ## 测试 - `tests/unit/test_gpu_direct_pipeline.py`: 30 passed - `tests/unit/test_render_adapter_pure.py`: 18 passed - ruff check/format clean - py_compile clean

xiaoxia added 1 commit 2026-09-29 18:08:56 +08:00

fix(worker): align gpu-direct title defaults (size/margin/bold) with CPU vfb path ... dfc5e5a5b6

PR #2095 fixed title baseline positioning and width-scaling but didn't fully
align default constants with the CPU video_filter_builder path, causing GPU
rendered titles to appear slightly smaller / higher / bolder-differently than
the CPU/ASS preview the template was authored against. This shows up in both
single and batch GPU renders since every variant shares the same gpu_direct
pipeline.

Root causes in PR #2095 defaults:
- Default title size 36@720p vs config_schemas DEFAULT 48 / vfb default 48
- Top/bottom margin 40@720p (PAD16+margin24) vs vfb _scale_title_len(50)
- Subtitle bottom margin 60@720p vs vfb 50
- Faux bold used same-color 1px border (white-on-white invisible for default
  white text; red-on-red for colored) vs vfb black 2px (intentional choice
  per #2001 to avoid double-print/halo artifact)
- margin_top was treated as absolute y offset, now correctly added on top of
  base margin (matches vfb additive semantics)
- subtitle stroke/bold block referenced uninitialized s_borderw (ruff F821)
  — added proper init + stroke parsing consistent with title block

Changes:
- TITLE_DEFAULT_MARGIN_TOP/BOTTOM = 50 (was 24+PAD=40)
- SUBTITLE_DEFAULT_MARGIN_BOTTOM = 50 (was 60)
- TITLE_FAUX_BOLD_WIDTH = 2, border color = #000000 (was 1, same as text)
- default title size 48@720p (was 36)
- margin_top from cfg added on top of base 50 (additive, same as vfb)
- subtitle: init s_borderw/s_border_color, parse stroke dict before bold check
- bottom position: y = h - th - margin_bottom (correct baseline; previously
  used margin_top variable which was misleading but numerically equivalent
  before margin split; now uses explicit margin_bottom)

Tests updated to new expected scaled values (1280x720 scale=1.778: fontsize 85,
y=89, bold borderw=4 black; 1080x1920 vertical: fontsize 72, y=75;
margin_top=100 user offset => y=267). Added test_title_bold_false_disables_faux_bold.

Batch investigation notes (separate findings, not bugs in this PR):
- per-variant plan config is correctly deep-copied via clone_plan_for_variant
  (config=dict(source.config or {})); title text per-variant via titles[]
  override; voice per-variant independent download with #1749 strict guards;
  bgm merged via merge_bgm_config — batch passthrough chain is correct.
- Worker generation concurrency = 2 (compose.yml default); USER_PENDING_LIMIT=3,
  6 tasks will enqueue in two waves (first 3 → then next 3 as workers free up);
  GLOBAL_PENDING_LIMIT allows it. This is expected behavior, not a bug.
- Legacy key mismatch: subtitle_render_engine.py reads plan_config['subtitle_config']
  (snake_case) but normalize_plan_config writes 'subtitle' short key. This file
  is not imported by unified_render_service (which uses render_subtitles.py with
  explicit subtitle_config= parameter), so no runtime impact; left for separate
  cleanup.
- Frontend usePlanConfigLoader.ts reads config.title_config snake_case while
  backend writes 'title' short key — frontend-only issue for subsequent edit
  sessions, out of backend scope.

gitea-actions bot commented 2026-09-29 18:11:10 +08:00

Copy Link

Copy Source

🚀 预览环境已部署

项目	详情
PR号	#2097
预览链接	https://pr-2097.preview.xiaoxiajianji.com
API环境	staging

💡 预览环境使用 staging API 数据，请勿在预览环境中操作重要数据。

🔄 每次提交新代码后预览环境会自动更新。

🗑️ PR 关闭或合并后，预览环境会自动清理。

🚀 **预览环境已部署** | 项目 | 详情 | |------|------| | PR号 | #2097 | | 预览链接 | [https://pr-2097.preview.xiaoxiajianji.com](https://pr-2097.preview.xiaoxiajianji.com) | | API环境 | staging | > 💡 预览环境使用 staging API 数据，请勿在预览环境中操作重要数据。 > > 🔄 每次提交新代码后预览环境会自动更新。 > > 🗑️ PR 关闭或合并后，预览环境会自动清理。

xiaoxia added 1 commit 2026-09-29 18:31:16 +08:00

fix(worker+api): resolve batch title race + auto-queue pending tasks (#2098) ... 96bcec5fdd

Two batch-rendering bugs found from staging logs (6-video batch 17:30-17:43):

Bug A (P1) — title_config race when same plan renders concurrently:
  _sync_task_config_to_plan wrote each task's title_config/bgm/resolution
  to the shared plan.config row, then RenderAdapter.render_plan read back
  from plan.config during rendering. If two tasks sharing the same plan
  (e.g. batch retry, preview+render, retry+new) ran concurrently, Task B's
  write could overwrite Task A's title before Task A's ffmpeg read it,
  producing videos with the wrong title text/style.

  Fix: stop mutating plan.config in the worker render path. Introduce
  task_config_override threading through render_plan / _do_render /
  UnifiedRenderService; URS reads config via _effective_config() which
  does a deep-copy plan.config merged with per-task override (title/bgm/
  export). _build_task_config_override builds the override dict (with
  field-name normalization: font_size→size, font_color→color) from
  task_info, and _download_voice_for_task handles voiceover download
  independently. plan.config is left as-authored by the API writeback;
  concurrent tasks no longer race on it.

  Files:
  - apps/worker/video_processing/unified_render_service.py: add
    override_config param + _cfg_section/_effective_config helpers;
    replace all render-time self.plan.config reads (title/subtitle/bgm/
    export/TTS) with _effective_config().
  - apps/worker/video_processing/render_adapter.py: render_plan/_do_render/
    _prepare_bgm accept task_config_override, merge into bgm/export reads,
    forward to URS.
  - apps/worker/worker_app/tasks/generation.py: replace
    _sync_task_config_to_plan (which wrote plan.config) with
    _build_task_config_override + _download_voice_for_task; pass
    task_config_override to render_plan.

Bug B (P0) — USER_PENDING_LIMIT=3 returns HTTP 429, blocks batch submit:
  Pre-check in create_generation_task rejected the whole batch with 429
  USER_QUEUE_FULL once user had ≥3 pending tasks; UI showed 'wait ~4 min'
  and prevented any more submissions. Users expect to submit a batch of
  6 and have them queue naturally (worker concurrency=2).

  Fix:
  - USER_PENDING_LIMIT 3→20 (soft cap for abuse protection, supports
    typical batch sizes of 6-10 with headroom).
  - Remove user-level 429 rejection from pre-check, retry endpoint,
    single-task confirm endpoint, and safe_enqueue (user path now logs
    a warning and continues to enqueue). Global GLOBAL_PENDING_LIMIT=20
    is retained as a hard 503 system-busy guard.
  - safe_enqueue post-enqueue check: user-over only logs, does not
    fail the task or raise.
  - Batch loop UserPendingLimitExceeded except branch now treats it as
    a successful enqueue (should not trigger in practice).

  Files:
  - apps/api/app/core/task_enqueue.py: limit 3→20; user checks log-only.
  - apps/api/app/api/routes/generation_tasks.py: remove user pre-check
    429; soften retry/single/batch except branches.

Stacked on PR #2095/#2097 which already fixed title baseline/width-scaling
and default-constant alignment with CPU video_filter_builder.

Tests:
- tests/unit/test_gpu_direct_pipeline.py: 30 passed
- tests/unit/test_render_adapter_pure.py: 18 passed
- ruff check/format clean; py_compile clean

xiaoxia changed title from fix(worker): align gpu-direct title defaults (size/margin/bold) with CPU vfb path to fix(worker+api): batch title vfb alignment, race-free config, auto-queue (3 bugs) 2026-09-29 18:31:42 +08:00

xiaoxia added 1 commit 2026-09-29 19:06:19 +08:00

test(api+worker): align unit tests with #2098 soft user limit + override refactor ... 5e1520230f

- Bug B: check_queue_limits no longer raises UserPendingLimitExceeded (user
  limit is soft); only global GLOBAL_PENDING_LIMIT hard-rejects with 503.
- safe_enqueue_generation_task user over-limit is warning-only (pre/post
  enqueue); GlobalQueueFull still hard-rejects/rolls-back.
- Preview route (/generation/preview) user-pending pre-check removed to
  match batch/generation/retry routes; only global 503 pre-check kept.
- Route enqueue-loop: UserPendingLimitExceeded except branch removed;
  all-failed rate-limit response only triggers on GlobalQueueFull (503).
- Tests rewritten to reflect new semantics (197 queue/preview/batch/pipeline
  tests pass).
- test_worker_generate_video_task_binding: replaced legacy
  _sync_task_config_to_plan helper test with _build_task_config_override
  + _download_voice_for_task (Bug A refactor).
- USER_PENDING_LIMIT constant verified at 20 (soft cap).

auto-approve-bot added 1 commit 2026-09-29 19:17:26 +08:00

style: auto-format with black + isort + ruff + prettier [skip ci-format-check] efb7fa5729

xiaoxia merged commit ff1d878c62 into develop 2026-09-29 19:51:40 +08:00

gitea-actions bot commented 2026-09-29 19:56:40 +08:00

Copy Link

Copy Source

🗑️ 预览环境已清理

PR #2097 已关闭或合并，对应的预览环境已被清理。

如有需要，可以重新打开 PR 来重新生成预览环境。

🗑️ **预览环境已清理** PR #2097 已关闭或合并，对应的预览环境已被清理。 > 如有需要，可以重新打开 PR 来重新生成预览环境。

Sign in to join this conversation.

Reviewers

No Reviewers

Labels

Clear labels

architecture

架构设计

backend

后端相关

blocked

阻塞中

bug

Bug修复

ci-bug

CI本身的bug

ci-optimization

CI优化类任务

ci-pipeline

CI流水线/workflow相关

cleanup

代码清理

devops

运维/部署/CI

documentation

文档相关

feature

新功能

frontend

前端相关

P0

阻塞性，必须立即处理

P1

高优先级

P2

中优先级

P3

低优先级/后考虑

question

需要讨论确认

ready

PR CI通过，可合并

refactor

代码重构

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

auto-approve-bot frontend-dev xiaoxia

No Assignees

3 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: xiaoxia/xiaoxia-saas#2097