7 Commits

Author	SHA1	Message	Date
saas-backend-agent	96bcec5fdd	fix(worker+api): resolve batch title race + auto-queue pending tasks (#2098) ... Two batch-rendering bugs found from staging logs (6-video batch 17:30-17:43): Bug A (P1) — title_config race when same plan renders concurrently: _sync_task_config_to_plan wrote each task's title_config/bgm/resolution to the shared plan.config row, then RenderAdapter.render_plan read back from plan.config during rendering. If two tasks sharing the same plan (e.g. batch retry, preview+render, retry+new) ran concurrently, Task B's write could overwrite Task A's title before Task A's ffmpeg read it, producing videos with the wrong title text/style. Fix: stop mutating plan.config in the worker render path. Introduce task_config_override threading through render_plan / _do_render / UnifiedRenderService; URS reads config via _effective_config() which does a deep-copy plan.config merged with per-task override (title/bgm/ export). _build_task_config_override builds the override dict (with field-name normalization: font_size→size, font_color→color) from task_info, and _download_voice_for_task handles voiceover download independently. plan.config is left as-authored by the API writeback; concurrent tasks no longer race on it. Files: - apps/worker/video_processing/unified_render_service.py: add override_config param + _cfg_section/_effective_config helpers; replace all render-time self.plan.config reads (title/subtitle/bgm/ export/TTS) with _effective_config(). - apps/worker/video_processing/render_adapter.py: render_plan/_do_render/ _prepare_bgm accept task_config_override, merge into bgm/export reads, forward to URS. - apps/worker/worker_app/tasks/generation.py: replace _sync_task_config_to_plan (which wrote plan.config) with _build_task_config_override + _download_voice_for_task; pass task_config_override to render_plan. Bug B (P0) — USER_PENDING_LIMIT=3 returns HTTP 429, blocks batch submit: Pre-check in create_generation_task rejected the whole batch with 429 USER_QUEUE_FULL once user had ≥3 pending tasks; UI showed 'wait ~4 min' and prevented any more submissions. Users expect to submit a batch of 6 and have them queue naturally (worker concurrency=2). Fix: - USER_PENDING_LIMIT 3→20 (soft cap for abuse protection, supports typical batch sizes of 6-10 with headroom). - Remove user-level 429 rejection from pre-check, retry endpoint, single-task confirm endpoint, and safe_enqueue (user path now logs a warning and continues to enqueue). Global GLOBAL_PENDING_LIMIT=20 is retained as a hard 503 system-busy guard. - safe_enqueue post-enqueue check: user-over only logs, does not fail the task or raise. - Batch loop UserPendingLimitExceeded except branch now treats it as a successful enqueue (should not trigger in practice). Files: - apps/api/app/core/task_enqueue.py: limit 3→20; user checks log-only. - apps/api/app/api/routes/generation_tasks.py: remove user pre-check 429; soften retry/single/batch except branches. Stacked on PR #2095/#2097 which already fixed title baseline/width-scaling and default-constant alignment with CPU video_filter_builder. Tests: - tests/unit/test_gpu_direct_pipeline.py: 30 passed - tests/unit/test_render_adapter_pure.py: 18 passed - ruff check/format clean; py_compile clean	2026-09-29 18:31:05 +08:00
saas-backend-bot	df99305dd6	feat(worker): celery 队列隔离 + 孤儿任务消息作废 (#1714) ... 问题：素材转码与视频生成共用 celery 默认队列、worker 单进程消费， 20+ 转码积压会把用户生成任务堵 40 分钟以上；孤儿清理把任务标 failed 后 Redis 队列消息未作废，消息被重投导致 failed→running 非法转换， worker 打印 ERROR 后继续产出半成品。 队列隔离： - 新增 packages/shared/celery_queues.py：generation/transcode/celery 三队列与 task_routes（generate_video→generation；ingest_asset/ classify_asset/duplication→transcode），apply_queue_settings() - worker 入口改双进程：generation worker 独占队列并内嵌 beat （prefetch=1, GENERATION_CONCURRENCY 默认 2），transcode worker 消费 transcode,celery（并发=总-2，最小 1），任一退出则整体终止 - compose/部署脚本/ps1 同步新增 GENERATION_CONCURRENCY 与健康检查 消息作废： - 新增 packages/shared/celery_orphan_guard.py：终态守卫 ensure_task_claimable、Redis 队列消息物理清理（JSON 信封解析， 按业务 id + celery headers.id 双匹配，未命中 rpush 保序）、 revoke_and_purge（control.revoke + 物理清队列双保险） - 入队点（生成/上传/分片/重试）send_task 后持久化 celery_task_id 到 generation_tasks/ingest_jobs（新列，067 迁移，失败仅 warning） - generate_video/ingest_asset 执行前校验 DB 状态：终态直接 discarded 不进业务逻辑；mark_processing 返回 False（非法转换）安全中止 - 孤儿/超时清理标 failed 时同时 revoke + 清队列消息 - pending 超时阈值 15→45 分钟，与 running 孤儿(20min)区分 测试：新增 22 个单测（路由表/真实 Redis 消息清理/终态守卫/ 非法转换中止/标 failed 后消息不重投/入队持久化），全量 14301 passed；067 迁移隔离 DDL 验证 upgrade/downgrade 通过。	2026-09-05 19:07:47 +08:00
xiaoxia	21c26b5b26	feat(generation): worker孤儿任务自动恢复 + 429限流结构化提示 (#1677) (#1710) ... Co-authored-by: xiaoxia <dev@xiaoxiajianji.com> Co-committed-by: xiaoxia <dev@xiaoxiajianji.com>	2026-09-05 11:37:49 +08:00
CI Bot	e9d2831850	feat(ci): mypy upgrade to hard gate + fix type errors	2026-07-16 07:59:07 +08:00
xiaoxia	ad671d94c5	fix(voice-clone): source_audio_url 不做预签名转换，原样返回用户输入 ... fix(voice-clone): source_audio_url 不做预签名转换，原样返回用户输入	2026-07-11 19:50:30 +08:00
xiaoxia	efe7f6b52a	feat: 任务队列限流防护 ... feat: 任务队列限流防护 - 用户级 pending 限流（3 个上限，返回 429） - 全局队列保护（20 个上限，返回 503） - 4 处入口全覆盖：批量创建、generation 重试、任务中心两级重试 - 入队前预检查 + 入队后最终校验双层兜底，处理并发竞态 - 阈值统一常量管理，边界语义一致 - 350+ 行单元测试覆盖所有边界场景	2026-07-11 16:29:07 +08:00
用户CI Test	9aa450bb8b	refactor: 安全入队函数抽为公共工具，消除重复定义 ... - 新增 app/core/task_enqueue.py，统一 safe_enqueue_generation_task 实现 - generation_tasks.py 和 task_center.py 改为 import 公共函数 - 修复 task_center.py 缺失 logging 导入的问题 - 日志前缀通过参数传入，保持各模块语义不变	2026-07-11 11:31:17 +08:00