37 Commits

Author	SHA1	Message	Date
saas-backend-agent	5e1520230f	test(api+worker): align unit tests with #2098 soft user limit + override refactor ... - Bug B: check_queue_limits no longer raises UserPendingLimitExceeded (user limit is soft); only global GLOBAL_PENDING_LIMIT hard-rejects with 503. - safe_enqueue_generation_task user over-limit is warning-only (pre/post enqueue); GlobalQueueFull still hard-rejects/rolls-back. - Preview route (/generation/preview) user-pending pre-check removed to match batch/generation/retry routes; only global 503 pre-check kept. - Route enqueue-loop: UserPendingLimitExceeded except branch removed; all-failed rate-limit response only triggers on GlobalQueueFull (503). - Tests rewritten to reflect new semantics (197 queue/preview/batch/pipeline tests pass). - test_worker_generate_video_task_binding: replaced legacy _sync_task_config_to_plan helper test with _build_task_config_override + _download_voice_for_task (Bug A refactor). - USER_PENDING_LIMIT constant verified at 20 (soft cap).	2026-09-29 19:06:12 +08:00
saas-backend-agent	96bcec5fdd	fix(worker+api): resolve batch title race + auto-queue pending tasks (#2098) ... Two batch-rendering bugs found from staging logs (6-video batch 17:30-17:43): Bug A (P1) — title_config race when same plan renders concurrently: _sync_task_config_to_plan wrote each task's title_config/bgm/resolution to the shared plan.config row, then RenderAdapter.render_plan read back from plan.config during rendering. If two tasks sharing the same plan (e.g. batch retry, preview+render, retry+new) ran concurrently, Task B's write could overwrite Task A's title before Task A's ffmpeg read it, producing videos with the wrong title text/style. Fix: stop mutating plan.config in the worker render path. Introduce task_config_override threading through render_plan / _do_render / UnifiedRenderService; URS reads config via _effective_config() which does a deep-copy plan.config merged with per-task override (title/bgm/ export). _build_task_config_override builds the override dict (with field-name normalization: font_size→size, font_color→color) from task_info, and _download_voice_for_task handles voiceover download independently. plan.config is left as-authored by the API writeback; concurrent tasks no longer race on it. Files: - apps/worker/video_processing/unified_render_service.py: add override_config param + _cfg_section/_effective_config helpers; replace all render-time self.plan.config reads (title/subtitle/bgm/ export/TTS) with _effective_config(). - apps/worker/video_processing/render_adapter.py: render_plan/_do_render/ _prepare_bgm accept task_config_override, merge into bgm/export reads, forward to URS. - apps/worker/worker_app/tasks/generation.py: replace _sync_task_config_to_plan (which wrote plan.config) with _build_task_config_override + _download_voice_for_task; pass task_config_override to render_plan. Bug B (P0) — USER_PENDING_LIMIT=3 returns HTTP 429, blocks batch submit: Pre-check in create_generation_task rejected the whole batch with 429 USER_QUEUE_FULL once user had ≥3 pending tasks; UI showed 'wait ~4 min' and prevented any more submissions. Users expect to submit a batch of 6 and have them queue naturally (worker concurrency=2). Fix: - USER_PENDING_LIMIT 3→20 (soft cap for abuse protection, supports typical batch sizes of 6-10 with headroom). - Remove user-level 429 rejection from pre-check, retry endpoint, single-task confirm endpoint, and safe_enqueue (user path now logs a warning and continues to enqueue). Global GLOBAL_PENDING_LIMIT=20 is retained as a hard 503 system-busy guard. - safe_enqueue post-enqueue check: user-over only logs, does not fail the task or raise. - Batch loop UserPendingLimitExceeded except branch now treats it as a successful enqueue (should not trigger in practice). Files: - apps/api/app/core/task_enqueue.py: limit 3→20; user checks log-only. - apps/api/app/api/routes/generation_tasks.py: remove user pre-check 429; soften retry/single/batch except branches. Stacked on PR #2095/#2097 which already fixed title baseline/width-scaling and default-constant alignment with CPU video_filter_builder. Tests: - tests/unit/test_gpu_direct_pipeline.py: 30 passed - tests/unit/test_render_adapter_pure.py: 18 passed - ruff check/format clean; py_compile clean	2026-09-29 18:31:05 +08:00
xiaoxia	a582d3b4dc	fix(lipsync, P0): Worker imports路径修复，任务卡在tts_processing (#1852) ... Co-authored-by: xiaoxia <dev@xiaoxiajianji.com> Co-committed-by: xiaoxia <dev@xiaoxiajianji.com>	2026-09-11 08:05:11 +08:00
saas-backend-bot	df99305dd6	feat(worker): celery 队列隔离 + 孤儿任务消息作废 (#1714) ... 问题：素材转码与视频生成共用 celery 默认队列、worker 单进程消费， 20+ 转码积压会把用户生成任务堵 40 分钟以上；孤儿清理把任务标 failed 后 Redis 队列消息未作废，消息被重投导致 failed→running 非法转换， worker 打印 ERROR 后继续产出半成品。 队列隔离： - 新增 packages/shared/celery_queues.py：generation/transcode/celery 三队列与 task_routes（generate_video→generation；ingest_asset/ classify_asset/duplication→transcode），apply_queue_settings() - worker 入口改双进程：generation worker 独占队列并内嵌 beat （prefetch=1, GENERATION_CONCURRENCY 默认 2），transcode worker 消费 transcode,celery（并发=总-2，最小 1），任一退出则整体终止 - compose/部署脚本/ps1 同步新增 GENERATION_CONCURRENCY 与健康检查 消息作废： - 新增 packages/shared/celery_orphan_guard.py：终态守卫 ensure_task_claimable、Redis 队列消息物理清理（JSON 信封解析， 按业务 id + celery headers.id 双匹配，未命中 rpush 保序）、 revoke_and_purge（control.revoke + 物理清队列双保险） - 入队点（生成/上传/分片/重试）send_task 后持久化 celery_task_id 到 generation_tasks/ingest_jobs（新列，067 迁移，失败仅 warning） - generate_video/ingest_asset 执行前校验 DB 状态：终态直接 discarded 不进业务逻辑；mark_processing 返回 False（非法转换）安全中止 - 孤儿/超时清理标 failed 时同时 revoke + 清队列消息 - pending 超时阈值 15→45 分钟，与 running 孤儿(20min)区分 测试：新增 22 个单测（路由表/真实 Redis 消息清理/终态守卫/ 非法转换中止/标 failed 后消息不重投/入队持久化），全量 14301 passed；067 迁移隔离 DDL 验证 upgrade/downgrade 通过。	2026-09-05 19:07:47 +08:00
xiaoxia	21c26b5b26	feat(generation): worker孤儿任务自动恢复 + 429限流结构化提示 (#1677) (#1710) ... Co-authored-by: xiaoxia <dev@xiaoxiajianji.com> Co-committed-by: xiaoxia <dev@xiaoxiajianji.com>	2026-09-05 11:37:49 +08:00
xiaoxia	ed21ee11d2	refactor(#776): 统一配置入口，建立packages/config消除重复定义 (#791)	2026-07-23 23:55:01 +08:00
CI Bot	e9d2831850	feat(ci): mypy upgrade to hard gate + fix type errors	2026-07-16 07:59:07 +08:00
xiaoxia	881eea9195	fix(api+ci): DELETE 204响应体全修复 + isort/black整理 + CI环境兼容 + 测试修复 (#286) ... fix(api+ci): DELETE 204响应体全修复 + isort/black整理 + CI环境兼容 + 测试修复	2026-07-14 07:07:51 +08:00
CI Bot	a3967c6829	Phase 4 Backend P0 cleanup ... 1. Remove unused aiosmtplib dependency from requirements.txt 2. Remove 5 unused config items (API_PREFIX, REDIS_MAX_CONNECTION, JWT_SECRET_KEY_OLD, SECRET_ROTATION_DAYS, LOG_LEVEL) from config.py, .env.production.example, and init_production_env.sh. Kept ENABLE_REDIS_SESSIONS (actively used in dependencies.py). 3. Fix project permission check TODOs in test_error_scenarios.py — permissions are implemented, updated assertions to expect 403. 4. Merge two Storage implementations into shared package: - packages/shared/storage.py: merged API features (HTTPS endpoint fix, diagnose(), detailed logging) into SharedStorageService - apps/api/app/core/storage.py: thin re-export wrapper for backward compatibility - Updated test mocks to target packages.shared.storage	2026-07-13 17:45:37 +08:00
CI Bot	9c6c477f55	fix(backend): Phase 1 后端代码清理与修复 ... P0 关键修复: - P0-1: 注册接口添加 RateLimitMiddleware 限流保护 - P0-3: /metrics 端点添加 JWT 认证（移除匿名访问） - P0-4: 修复 Celery 任务名冲突（generation_task vs generate_video） - P1-5: JWT logout token 黑名单机制 P1 修复: - P1-1: forgot_password 硬编码 localhost → 使用 settings.APP_BASE_URL - P1-2: generation.py 直接创建 DB 连接 → 使用依赖注入 - P1-6: Image.open() 未关闭 → 统一使用 with 语句 - P1-7: 订阅续费事务修复 P2 代码质量: - P2-1: 修复 EditingMode 枚举重复定义 → 统一引用 shared 包 - P2-2: 修复 SMTP_FRON_NAME → SMTP_FROM_NAME 拼写 - P2-3: UserModel subscription_quota 类型统一为 float - P2-4: .env.production DATABASE_MAX_OVERFLOW 30 → 10 - 清理 15 处 except:pass（保留 2 处有注释说明的） - 禁用 SVG 上传（XSS 风险） - 删除 decode_token_unsafe() 不安全函数 - 简化 /ready 端点 - 删除 8 处死代码、10 个空文件/模块 - 合并 3 对 100% 重复函数 - 对齐 6 个废弃环境变量 v2 修复（代码审查后）: - 修复密码重置路由路径: /password/forgot → /forgot-password, /password/reset → /reset-password（与前端 API 对齐） - 合并 _check_project_access: asset_libraries.py 和 edit_plans.py 中的重复函数统一到 _helpers.py（含空字符串守卫 + 中文错误信息） - 顺手修复: HTTPException 统一从 fastapi 导入（替换 starlette 导入） - OSS_ENDPOINT 拼写修复拆分为单独 PR，本 PR 不包含	2026-07-13 13:50:52 +08:00
xiaoxia	ad671d94c5	fix(voice-clone): source_audio_url 不做预签名转换，原样返回用户输入 ... fix(voice-clone): source_audio_url 不做预签名转换，原样返回用户输入	2026-07-11 19:50:30 +08:00
xiaoxia	efe7f6b52a	feat: 任务队列限流防护 ... feat: 任务队列限流防护 - 用户级 pending 限流（3 个上限，返回 429） - 全局队列保护（20 个上限，返回 503） - 4 处入口全覆盖：批量创建、generation 重试、任务中心两级重试 - 入队前预检查 + 入队后最终校验双层兜底，处理并发竞态 - 阈值统一常量管理，边界语义一致 - 350+ 行单元测试覆盖所有边界场景	2026-07-11 16:29:07 +08:00
用户CI Test	9aa450bb8b	refactor: 安全入队函数抽为公共工具，消除重复定义 ... - 新增 app/core/task_enqueue.py，统一 safe_enqueue_generation_task 实现 - generation_tasks.py 和 task_center.py 改为 import 公共函数 - 修复 task_center.py 缺失 logging 导入的问题 - 日志前缀通过参数传入，保持各模块语义不变	2026-07-11 11:31:17 +08:00
xiaoxia	f5d3482fa9	fix: P0-2 OSS 凭证启动验证 + 诊断日志 (#209)	2026-07-10 18:15:42 +08:00
用户CI Test	81984ae938	fix: P0-2 预签名URL诊断日志 + P0-3 xfade exit 234 三重修复 ... P0-2 (预签名URL staging未生效): - storage.py get_download_url() 增加诊断日志(bucket=None / sign_url失败) - 代码逻辑已验证正确,8个单元测试覆盖全路径 - staging问题最可能是未重新部署或OSS凭证未配置 P0-3 (FFmpeg xfade exit 234): - Fix 1: _build_single_clip_filter 始终 trim 到 effective_duration - Fix 2: _build_filter_complex 用 effective_duration 计算 xfade offset - Fix 3: build_xfade_filter_chain 逐步安全钳制 td/offset - 动态 first_input_dur 跟踪 - per-step td 钳制到 available 和 remaining - 保证 offset + td ≤ first_input_duration 新增 20 个单元测试全部通过,全量 1213 测试无回归 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>	2026-07-10 14:47:09 +08:00
CI Test	abfc598d3f	refactor: 代码精简优化 - 移除无用代码和重复定义 ... 1. 删除无用文件: - apps/api/app/core/database.py (未被引用的死代码) - requirements-quality.txt (与 requirements-dev.txt 完全重复) - fix_script.py, fix_tracker_encoding.py (一次性修复脚本) - init_tracker_phase4.py, init_tracker_simple.py, update_tracker.py (SQLite迁移脚本) 2. 清理死代码: - apps/api/app/db.py: 移除未使用的 get_db() 函数和 sqlalchemy.orm 导入 - 11个路由/中间件文件: 移除未使用的 import (os, datetime, BaseModel, Session, List 等) 3. 消除重复定义: - apps/worker/video_processing/video_compose_service.py: 移除重复的 EditingMode 枚举 - 改为从 packages.domain.editing_mode 导入统一的 EditingMode 影响: 无功能变更，仅移除未使用的代码	2026-07-03 00:44:29 +08:00
xiaoxia	1217d8cef0	style: apply isort formatting to pass CI validation (#129)	2026-06-30 18:17:44 +08:00
xiaoxia	52ff2f80ad	style: apply black formatting to pass CI validation (#126)	2026-06-30 17:23:08 +08:00
Audit Bot	05c9137ece	fix: 上传端点异常处理 + OSS 诊断能力 ... 问题：PR#119 修复变量名后上传端点仍然 500，说明根因不仅是变量名错误 修复： 1. upload.py — 所有 3 个上传端点添加完整异常捕获和日志 - prepare_direct_upload: 捕获 RuntimeError→503 + 其他异常→500+日志 - complete_direct_upload: file_exists() 异常捕获→503（之前未捕获→500） - upload_asset: upload_file() 异常捕获→503/500+日志（之前未捕获→500） 2. storage.py — OSSStorageService.__init__ 添加启动诊断日志 - 凭证缺失时记录 missing env vars - oss2 SDK 未安装时记录错误 - bucket 初始化成功/失败时记录 3. health.py — /ready 端点增加 OSS 检查 - 检查凭证是否配置 - 检查 oss2 SDK 是否可用 - 调用 get_bucket_info() 验证连通性 部署后请调用 GET /api/v1/ready 查看 OSS 状态	2026-06-30 09:56:20 +08:00
API文档维护Agent	e2c68888e9	refactor: 统一 OSS 配置命名（MINIO_* → OSS_*） ... - .env.production: MINIO_* 环境变量全部改为 OSS_* 命名 - MINIO_ENDPOINT → OSS_ENDPOINT - MINIO_ACCESS_KEY → OSS_ACCESS_KEY_ID - MINIO_SECRET_KEY → OSS_ACCESS_KEY_SECRET - MINIO_BUCKET → OSS_BUCKET_NAME - MINIO_SECURE → OSS_SECURE - MINIO_PUBLIC_URL → OSS_PUBLIC_URL - .env.staging: 修正 section header 为 OSS 对象存储配置 - storage.py: 移除 MinIOService 和 get_minio_service() 向后兼容别名 - test_architecture_boundaries.py: 移除 storage.py 的 allowed 例外 - 更新文档中的 MINIO_* 示例为 OSS_*: - docs/CI-CD.md - docs/基础设施安装指南.md - docs/本地验证检查清单.md - docs/静态验证报告-2026-06-19.md	2026-06-28 08:33:48 +08:00
xiaoxia	7651a5f3f2	fix: P3-1 unify database config with app.config	2026-06-26 17:56:33 +08:00
Xiaoxia AI	6e42f944c0	feat(upload): add OSS direct upload flow	2026-06-23 15:47:03 +08:00
Xiaoxia AI	bfbaddbd9a	style: normalize python formatting gates	2026-06-21 06:52:19 +08:00
Xiaoxia AI	a3fe72a65f	refactor(config): centralize api settings source	2026-06-21 00:36:13 +08:00
Xiaoxia AI	57310f33d4	fix(auth): harden simple auth and router imports	2026-06-21 00:27:58 +08:00
Xiaoxia AI	1e3a52fa17	fix(worker): persist generated mp4 without oss	2026-06-20 23:17:52 +08:00
Xiaoxia AI	d31c62d4e2	fix(deploy): align deployment with split infrastructure	2026-06-20 19:02:48 +08:00
Xiaoxia AI	94dd60108c	refactor: 彻底迁移到阿里云 OSS 并重构基础设施 ... - 删除 MinIO 依赖，改用阿里云 OSS (oss2) - 重写存储服务 storage.py（完全兼容原接口） - 更新配置：config.py 改为 OSS_* 配置项 - 重构 Docker Compose：分离基础设施（infra.yml）和应用（compose.yml） - 添加健康检查和自动重启配置 - 创建标准化部署脚本（init-server.sh + deploy.sh） - 统一 Nginx 配置（主域名 api.xiaoxiajianji.com） - 修复 Web Dockerfile 端口映射（80 not 3000） 服务器清理完成： - 删除旧云控制台、监控系统、桌面版代码 - 删除 2.6GB 旧安装包 - 清理 Docker 缓存 2.5GB - 磁盘使用率从 48% 降至 26%	2026-06-20 15:24:24 +08:00
Xiaoxia AI	8493471c19	feat(project-d): implement real video generation pipeline with FFmpeg	2026-06-19 10:54:20 +08:00
Xiaoxia AI	642d01eee4	feat(phase7): prefer presigned generated video downloads	2026-06-18 20:49:31 +08:00
Xiaoxia AI	8d78798a7d	fix(phase7): align api runtime wiring for integration	2026-06-17 18:48:08 +08:00
Xiaoxia AI	c6e6213209	fix: decouple api from worker package imports	2026-06-15 18:59:58 +08:00
Xiaoxia AI	4475677763	chore: wire deployment to environment-based configuration	2026-06-15 18:40:56 +08:00
Xiaoxia AI	36126b47d7	chore: add production environment configuration system	2026-06-15 18:32:05 +08:00
Xiaoxia AI	ba159549e2	feat: integrate MinIO for real file storage	2026-06-15 18:19:12 +08:00
Xiaoxia AI	e4e2595e70	feat: add PostgreSQL persistence layer ... - adapters: SQLAlchemy implementations for all 4 repositories (Project, AssetLibrary, Asset, IngestJob) - models: SQLAlchemy ORM models with proper schema - database: connection config and session management - tests: SQLAlchemy repository integration test (in-memory SQLite) - all 7 integration tests passing	2026-06-15 15:29:13 +08:00
Xiaoxia AI	b5a62ee9a3	feat: initial SaaS scaffold ... - domain: User, Workspace, Project, AssetLibrary, Asset, IngestJob - ports: repository interfaces - application: use cases - adapters: in-memory - API: FastAPI 5 routes - worker: Celery ingest_asset - tests: 4 passing	2026-06-15 15:17:40 +08:00