Compare commits

..

35 Commits

Author SHA1 Message Date
xiaoxia f5ad1b2b31 fix(ci): 修复CI Gate失败时返回exit 0而非exit 1的P0 Bug
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 18s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 46s
CI/CD Pipeline / Validate - Migration (alembic) (pull_request) Successful in 1m14s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 1m17s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 32s
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Validate - Type Check (mypy) (pull_request) Successful in 1m23s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been skipped
Preview Deploy / Deploy Preview Environment (pull_request) Failing after 48s
AI Code Review / AI Code Review (pull_request) Successful in 1m19s
CI/CD Pipeline / Validate - Code Quality (pull_request) Successful in 3m26s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 4m21s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 8m56s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Unit Tests (pull_request) Successful in 3m11s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m56s
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 2s
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 18s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 34s
CI Gate判断失败时脚本错误地写了exit 0,导致所有PR不管CI失败成什么样,
Gate永远显示success,分支保护完全失效。
修复:将失败分支的exit 0改为exit 1
2026-07-28 10:12:35 +08:00
xiaoxia 02e3246f5a fix(ci): 格式修复防循环索引 + AI审查fail-open(2个bug修复) (#1045)
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Frontend Lint (push) Successful in 55s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m3s
CI/CD Pipeline / Build Production API Image (push) Failing after 27s
CI/CD Pipeline / Build Production Web Image (push) Failing after 20s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m25s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 1m18s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 23s
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Validate - Code Quality (push) Failing after 2m7s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 2m4s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 2m3s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m6s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Unit Tests (push) Successful in 3m26s
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (push) Has been skipped
CI/CD Pipeline / Integration Tests (push) Successful in 2m46s
CI/CD Pipeline / Canary Release to Production (push) Has been cancelled
CI/CD Pipeline / CI Gate (push) Has been skipped
fix(ci): 修复auto_fix_formatting防循环索引错误 + AI审查fail-open未生效

1. 防循环索引bug:Gitea API返回commits倒序,commits[-1]取到最旧commit,改为commits[0]
2. fail-open bug:LLM调用失败和未捕获异常都是exit 1,改为exit 0不阻塞合并
2026-07-28 09:52:23 +08:00
xiaoxia 5cdafd2559 feat: AI代码审查添加commit status输出和阻塞级问题判定 (#1035)
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m49s
CI/CD Pipeline / Frontend Lint (push) Successful in 1m53s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m58s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 2m2s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m31s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 2m33s
CI/CD Pipeline / Validate - Code Quality (push) Failing after 2m55s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 34s
CI/CD Pipeline / Build Production API Image (push) Failing after 18s
CI/CD Pipeline / Build Production Web Image (push) Failing after 17s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 18s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Unit Tests (push) Successful in 4m38s
CI/CD Pipeline / Integration Tests (push) Successful in 4m21s
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been cancelled
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Code Quality (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Type Check (mypy) (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Migration (alembic) (pull_request) Has been cancelled
CI/CD Pipeline / Unit Tests (pull_request) Has been cancelled
CI/CD Pipeline / Integration Tests (pull_request) Has been cancelled
CI/CD Pipeline / Frontend Lint (pull_request) Has been cancelled
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been cancelled
CI/CD Pipeline / PR Build API Image (pull_request) Has been cancelled
CI/CD Pipeline / PR Build Web Image (pull_request) Has been cancelled
CI/CD Pipeline / PR Build Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Staging API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been cancelled
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been cancelled
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been cancelled
CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been cancelled
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / CI Gate (pull_request) Has been cancelled
AI Code Review / AI Code Review (pull_request) Has been cancelled
Preview Deploy / Deploy Preview Environment (pull_request) Has been cancelled
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been skipped
PR Automation / Auto Approve on CI Green (pull_request) Successful in 1m2s
- 为AI代码审查添加commit status输出,PR页面可直接看到审查结果
- 添加阻塞级问题判定逻辑,严重问题标记为failure状态
- 优化审查报告格式和输出精度
2026-07-27 21:40:07 +08:00
xiaoxia a6afb344ba feat: 格式自动修复对所有PR开放,添加防循环机制 (#1037)
CI/CD Pipeline / Validate - Type Check (mypy) (push) Failing after 0s
CI/CD Pipeline / Validate - Code Quality (push) Failing after 0s
CI/CD Pipeline / Validate - Migration (alembic) (push) Failing after 0s
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Frontend Lint (push) Failing after 0s
CI/CD Pipeline / Integration Tests (push) Failing after 0s
CI/CD Pipeline / Unit Tests (push) Failing after 0s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 0s
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Failing after 28s
CI/CD Pipeline / Build Production Web Image (push) Failing after 30s
CI/CD Pipeline / Build Production API Image (push) Failing after 34s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m32s
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 2m12s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m13s
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
2026-07-27 20:29:56 +08:00
xiaoxia 504e2e71c9 fix(ci): auto_merge.sh改用CI Gate统一门禁
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Frontend Lint (push) Successful in 30s
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m3s
CI/CD Pipeline / Validate - Code Quality (push) Failing after 1m18s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 55s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m8s
CI/CD Pipeline / Build Staging API Image (push) Failing after 1m38s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 1m29s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 16s
CI/CD Pipeline / Build Production API Image (push) Failing after 5s
CI/CD Pipeline / Build Production Web Image (push) Failing after 7s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 6s
CI/CD Pipeline / Unit Tests (push) Successful in 3m27s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Integration Tests (push) Successful in 2m16s
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been cancelled
CI/CD Pipeline / Canary Release to Production (push) Has been cancelled
2026-07-27 16:21:19 +08:00
xiaoxia fb2884b03c fix(ci): 添加ci-gate汇总job,解决自动合并405问题
CI/CD Pipeline / Check if frontend-only change (push) Has been cancelled
CI/CD Pipeline / Validate - Code Quality (push) Has been cancelled
CI/CD Pipeline / Validate - Type Check (mypy) (push) Has been cancelled
CI/CD Pipeline / Validate - Migration (alembic) (push) Has been cancelled
CI/CD Pipeline / Unit Tests (push) Has been cancelled
CI/CD Pipeline / Integration Tests (push) Has been cancelled
CI/CD Pipeline / Frontend Lint (push) Has been cancelled
CI/CD Pipeline / Frontend Unit Tests (push) Has been cancelled
CI/CD Pipeline / PR Build API Image (push) Has been cancelled
CI/CD Pipeline / PR Build Web Image (push) Has been cancelled
CI/CD Pipeline / PR Build Worker Image (push) Has been cancelled
CI/CD Pipeline / Build Staging API Image (push) Has been cancelled
CI/CD Pipeline / Build Staging Web Image (push) Has been cancelled
CI/CD Pipeline / Build Staging Worker Image (push) Has been cancelled
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been cancelled
CI/CD Pipeline / Staging E2E Tests (push) Has been cancelled
CI/CD Pipeline / Staging API Integration Tests (push) Has been cancelled
CI/CD Pipeline / Build Production API Image (push) Has been cancelled
CI/CD Pipeline / Build Production Web Image (push) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (push) Has been cancelled
CI/CD Pipeline / Deploy Production (push) Has been cancelled
CI/CD Pipeline / Production Browser E2E (push) Has been cancelled
CI/CD Pipeline / ACR Image Cleanup (push) Has been cancelled
CI/CD Pipeline / Canary Release to Production (push) Has been cancelled
CI/CD Pipeline / CI Gate (push) Has been cancelled
2026-07-27 16:17:50 +08:00
xiaoxia 7fab42c3d0 fix(ci): daily-check DooD挂载路径修复 + shell bash修复 (#1024)
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m33s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m33s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m38s
CI/CD Pipeline / Build Production API Image (push) Failing after 9s
CI/CD Pipeline / Frontend Lint (push) Successful in 1m45s
CI/CD Pipeline / Validate - Code Quality (push) Failing after 1m47s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 16s
CI/CD Pipeline / Build Production Web Image (push) Failing after 11s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 13s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 2m3s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m19s
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Unit Tests (push) Successful in 3m57s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been cancelled
CI/CD Pipeline / Integration Tests (push) Successful in 2m2s
2026-07-27 12:04:41 +08:00
xiaoxia 561548c84c fix(ci): daily-check shell从sh改为bash,修复PIPESTATUS Bad substitution (#1023)
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Frontend Lint (push) Successful in 44s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 49s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m39s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m36s
CI/CD Pipeline / Build Production API Image (push) Failing after 7s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m48s
CI/CD Pipeline / Build Production Web Image (push) Failing after 12s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 16s
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Validate - Code Quality (push) Failing after 2m10s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m17s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 2m2s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
CI/CD Pipeline / Integration Tests (push) Successful in 2m52s
CI/CD Pipeline / Unit Tests (push) Successful in 5m5s
2026-07-27 11:39:29 +08:00
xiaoxia 77704e7ec6 fix(ci): 同步daily-check和acr-cleanup的docker兼容性修复到main (#1012)
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 48s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 46s
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Validate - Code Quality (push) Failing after 1m20s
CI/CD Pipeline / Frontend Lint (push) Successful in 36s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m25s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 2m1s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 2m22s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 25s
CI/CD Pipeline / Build Production API Image (push) Failing after 11s
CI/CD Pipeline / Build Production Web Image (push) Failing after 19s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 16s
CI/CD Pipeline / Unit Tests (push) Successful in 2m23s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Integration Tests (push) Successful in 1m56s
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
cherry-pick #981的核心修复到main分支:
- daily-check.yml: checkout步骤改为curl step_checkout.sh方式
- acr-cleanup.yml: 同上,修复Setup Python秒败问题

相关PR: #981
相关工单: #980
2026-07-27 08:46:05 +08:00
xiaoxia 5ae6c33bf6 sync(ci): 同步缺失的CI监控脚本到main分支
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 53s
CI/CD Pipeline / Frontend Lint (push) Successful in 38s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 54s
CI/CD Pipeline / Validate - Code Quality (push) Failing after 2m0s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m21s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m8s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 2m25s
CI/CD Pipeline / Build Production Web Image (push) Failing after 25s
CI/CD Pipeline / Build Production API Image (push) Failing after 27s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 37s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Failing after 35s
CI/CD Pipeline / Unit Tests (push) Successful in 2m56s
CI/CD Pipeline / Integration Tests (push) Successful in 2m21s
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been cancelled
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
同步2个CI监控脚本到main分支:scripts/ci_trigger_monitor.py、scripts/ci_code_review.py
2026-07-26 18:13:27 +08:00
xiaoxia db07738178 sync(ci): 同步schedule类workflow修复到main分支 (#933)
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Frontend Lint (push) Successful in 1m1s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m13s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m10s
CI/CD Pipeline / Validate - Code Quality (push) Failing after 2m15s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m11s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 1m48s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m13s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 24s
CI/CD Pipeline / Build Production Web Image (push) Failing after 18s
CI/CD Pipeline / Build Production API Image (push) Failing after 21s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 18s
CI/CD Pipeline / Unit Tests (push) Successful in 3m31s
CI/CD Pipeline / Integration Tests (push) Successful in 2m28s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been cancelled
2026-07-26 16:18:24 +08:00
xiaoxia 53c09e7d3c chore(ci): 同步金丝雀发布流水线到main分支(#450)
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Frontend Lint (push) Successful in 25s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 58s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 54s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m20s
CI/CD Pipeline / Validate - Code Quality (push) Failing after 2m25s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 16s
CI/CD Pipeline / Build Production API Image (push) Failing after 13s
CI/CD Pipeline / Build Production Worker Image (push) Failing after 11s
CI/CD Pipeline / Build Production Web Image (push) Failing after 14s
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m14s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 1m57s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
CI/CD Pipeline / Integration Tests (push) Successful in 1m52s
CI/CD Pipeline / Unit Tests (push) Successful in 3m16s
2026-07-25 15:14:33 +08:00
xiaoxia e602439769 chore(ci): 同步renovate.json5到main分支
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m6s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m1s
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Validate - Code Quality (push) Failing after 1m47s
CI/CD Pipeline / Frontend Lint (push) Successful in 42s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m3s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 1m47s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m11s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 21s
CI/CD Pipeline / ACR Image Cleanup (push) Has been skipped
CI/CD Pipeline / Integration Tests (push) Successful in 2m12s
CI/CD Pipeline / Unit Tests (push) Successful in 3m23s
同步#876的renovate依赖自动更新配置到main,保持main和develop CI配置一致。
2026-07-25 14:44:52 +08:00
xiaoxia a26fda1597 feat(ci): ACR镜像自动清理增强版 - 同步到main(#524)
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 46s
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / Validate - Code Quality (push) Failing after 1m2s
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 44s
CI/CD Pipeline / Frontend Lint (push) Successful in 22s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 2m21s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m22s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 16s
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 2m53s
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Integration Tests (push) Successful in 3m21s
CI/CD Pipeline / Unit Tests (push) Successful in 5m18s
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (push) Has been skipped
ACR镜像自动清理增强版同步到main:3种模式+白名单保护+cron/PR关闭/手动触发
2026-07-25 12:21:41 +08:00
xiaoxia 99a8ffa97b chore(ci): 升级migration验证,新增4项检查 (#451) 同步到main
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 51s
CI/CD Pipeline / Frontend Lint (push) Successful in 34s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 52s
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Validate - Code Quality (push) Failing after 1m13s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 22s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Build Staging API Image (push) Failing after 1m44s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 1m27s
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m43s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (push) Has been skipped
CI/CD Pipeline / Unit Tests (push) Successful in 3m11s
CI/CD Pipeline / Integration Tests (push) Successful in 2m3s
同步#857 migration验证升级到main分支

新增:ci_env.sh共享变量、check_migration_naming.py命名检查
升级:validate_migration.sh 5阶段完整检查
2026-07-25 09:54:58 +08:00
xiaoxia f03c9d5453 chore(ci): P2端口收尾同步到main分支 (#798 #799) (#853)
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Frontend Lint (push) Successful in 42s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m27s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m28s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 54s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m44s
CI/CD Pipeline / Validate - Code Quality (push) Failing after 1m48s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m27s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 2m27s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (push) Has been skipped
CI/CD Pipeline / Integration Tests (push) Successful in 2m17s
CI/CD Pipeline / Unit Tests (push) Successful in 4m22s
chore(ci): P2端口收尾同步到main分支 (#798 #799)

- docs: 新增统一端口分配清单文档 (PORTS.md)
- chore(ci): 端口变量命名统一 (chatops配置)

跳过ci-pipeline.yml的CI_PG_PORT改动:main分支无顶层env块,后续CI配置对齐时再同步。
2026-07-24 23:54:09 +08:00
xiaoxia 8322e2b6e2 fix(ci): staging E2E/API tests shell由sh改为bash
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 36s
CI/CD Pipeline / Frontend Lint (push) Successful in 49s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m39s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m44s
CI/CD Pipeline / Validate - Code Quality (push) Failing after 1m52s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 1m35s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 2m0s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m31s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (push) Has been skipped
CI/CD Pipeline / Integration Tests (push) Successful in 3m0s
CI/CD Pipeline / Unit Tests (push) Successful in 5m20s
2026-07-24 15:27:33 +08:00
CI Bot d2409e16c1 style: auto-format with black + isort + prettier
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Validate - Code Quality (push) Failing after 1m23s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 55s
CI/CD Pipeline / Frontend Lint (push) Successful in 33s
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m1s
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m38s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 2m17s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 2m3s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 14s
CI/CD Pipeline / Integration Tests (push) Successful in 3m4s
CI/CD Pipeline / Unit Tests (push) Successful in 4m45s
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (push) Has been skipped
2026-07-24 02:37:59 +00:00
xiaoxia 6eac0b2cf2 chore(ci): 同步main分支CI配置与scripts/ci脚本 - 与develop对齐
Worker Base Image Build / Build Worker Base Images (worker-base-builder-cache, infra/docker/worker-base-builder.Dockerfile, worker-base-builder, builder) (push) Failing after 1m54s
Worker Base Image Build / Build Worker Base Images (worker-base-runtime-cache, infra/docker/worker-base-runtime.Dockerfile, worker-base-runtime, runtime) (push) Failing after 1m36s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Validate - Code Quality (push) Failing after 1m29s
CI/CD Pipeline / Validate - Type Check (mypy) (push) Successful in 1m4s
CI/CD Pipeline / Validate - Migration (alembic) (push) Successful in 1m2s
CI/CD Pipeline / Unit Tests (push) Successful in 3m38s
CI/CD Pipeline / Integration Tests (push) Successful in 2m0s
CI/CD Pipeline / Frontend Lint (push) Successful in 28s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 44s
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Build Staging API Image (push) Failing after 2m16s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 8m14s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 2m0s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (push) Has been skipped
同步内容:
1. CI流水线配置(ci-pipeline.yml)与develop对齐
2. PR构建脚本docker_build_only.sh增加buildx→docker build回退
3. pre-build步骤worker基础镜像构建增加buildx回退
4. 单元测试脚本全量覆盖率改为仅报告不阻塞
5. diff-cover依赖加入requirements-dev.txt
6. worker base builder/runtime Dockerfile同步
7. test_config_oss.py clear=False→clear=True修复OSS污染
8. Frontend Lint增加prettier依赖
2026-07-24 10:36:29 +08:00
xiaoxia dfb2feef8a fix(ci): main??auto-merge/approve????Tests/test??
CI/CD Pipeline / Validate Code Quality And Tests (push) Successful in 2m0s
CI/CD Pipeline / Frontend Lint (push) Successful in 2m10s
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Failing after 3s
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
- auto-merge.yml: main????Tests/test,?3???????
- auto-approve.yml: main??????????Tests/test
- ??#464???Tests/test???????????
2026-07-17 14:04:23 +08:00
auto-approve-bot b3ef7bb041 Merge pull request 'feat(ci): main分支auto-merge从定时改为即时合并' (#464) from ci/main-auto-merge-instant into main
CI/CD Pipeline / Validate Code Quality And Tests (push) Successful in 2m10s
CI/CD Pipeline / Frontend Lint (push) Successful in 2m12s
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Failing after 4s
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
2026-07-17 13:27:18 +08:00
xiaoxia a1a272b833 feat(ci): 添加auto-approve到main分支并适配main门禁规则
Tests / lint (pull_request) Successful in 6s
Tests / test (pull_request) Failing after 29s
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Successful in 2m23s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 2m46s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
Auto Approve CI PRs / Auto Approve on CI Green (pull_request) Successful in 2m54s
Auto Merge PRs (main) / Auto Merge on CI Green + Approved (main) (pull_request) Successful in 3m0s
2026-07-17 13:24:21 +08:00
xiaoxia 728db0faf8 chore: empty commit to re-trigger CI for auto-merge verification
Tests / lint (pull_request) Successful in 9s
Tests / test (pull_request) Failing after 21s
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Successful in 1m32s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 1m44s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
Auto Merge PRs (main) / Auto Merge on CI Green + Approved (main) (pull_request) Successful in 20m36s
2026-07-17 13:06:40 +08:00
xiaoxia 7e5e412f7f feat(ci): main分支auto-merge从定时改为即时合并
Tests / lint (pull_request) Failing after 9s
Tests / test (pull_request) Failing after 28s
Auto Merge PRs (main) / Auto Merge on CI Green + Approved (main) (pull_request) Failing after 30s
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Successful in 1m53s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 2m29s
CI/CD Pipeline / Build Production Runtime Images (pull_request) Has been skipped
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
- 触发方式:pull_request事件(CI状态变更时)
- 合并条件:2门禁全绿 + 至少1个APPROVED + 无冲突 + 非草稿
- 安全措施:幂等保护、合并失败留评论、只合main
- 新增check_ci_status.py和check_pr_approval.py辅助脚本
2026-07-17 12:37:24 +08:00
xiaoxia df08161630 fix(ci): 修复auto-merge缺少GITEA_API_TOKEN环境变量
CI/CD Pipeline / Frontend Lint (push) Successful in 58s
CI/CD Pipeline / Validate Code Quality And Tests (push) Successful in 1m36s
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Failing after 7s
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
1. 给auto-merge.yml的两个merge step添加GITEA_API_TOKEN环境变量,复用REVIEW_GITEA_TOKEN secret
2. 修复tests.yml的checkout步骤GITHUB_TOKEN和Python命令
3. 给main分支ci-cd.yml的单元测试增加Redis服务,修复Celery相关测试失败
2026-07-17 09:02:33 +08:00
xiaoxia 0c9375ff32 ci: remove temporary debug workflow - test-ssh-secret.yml
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 36s
CI/CD Pipeline / Frontend Lint (push) Failing after 0s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
Fix Flake8 E741 for PR232 / fix-flake8 (push) Failing after 3h4m59s
2026-07-12 10:25:12 +08:00
xiaoxia ef344e9ffc cleanup: remove upgrade test file
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Failing after 12h43m56s
CI/CD Pipeline / Frontend Lint (push) Failing after 12h45m50s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 12h45m50s
2026-07-09 00:08:40 +08:00
xiaoxia 0d4904433e test: trigger workflow after gitea upgrade
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Failing after 12h49m24s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Failing after 12h53m37s
CI/CD Pipeline / Frontend Lint (push) Failing after 12h56m8s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 12h56m8s
2026-07-08 23:58:20 +08:00
CI Test 708662394f Merge develop into main - v0.1.126
Auto Merge PRs / auto-merge (push) Has been cancelled
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 29h56m45s
CI/CD Pipeline / Frontend Lint (push) Failing after 29h56m45s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
2026-07-08 06:57:42 +08:00
CI Test 9b034764ad Merge develop into main - v0.1.125
Auto Merge PRs / auto-merge (push) Failing after 1m29s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 40h59m5s
CI/CD Pipeline / Frontend Lint (push) Failing after 40h58m8s
CI/CD Pipeline / Deploy Staging (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
2026-07-07 19:54:54 +08:00
CI Test 8748b43070 Merge develop into main - v0.1.124
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 45h17m30s
CI/CD Pipeline / Frontend Lint (push) Failing after 45h17m8s
CI/CD Pipeline / Deploy Staging (push) Failing after 45h15m38s
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
2026-07-07 15:36:36 +08:00
CI Test d213a055a1 Merge develop into main - v0.1.123
Auto Merge PRs / auto-merge (push) Failing after 3m27s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 47h9m34s
CI/CD Pipeline / Frontend Lint (push) Failing after 47h8m47s
CI/CD Pipeline / Deploy Staging (push) Failing after 47h6m52s
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
2026-07-07 13:44:46 +08:00
CI Test 2371860f82 Merge develop into main - v0.1.122 (prettier fix)
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 49h37m40s
CI/CD Pipeline / Frontend Lint (push) Failing after 49h37m23s
CI/CD Pipeline / Deploy Staging (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (push) Failing after 49h35m42s
CI/CD Pipeline / Deploy Production (push) Failing after 49h33m12s
CI/CD Pipeline / Production Browser E2E (push) Failing after 49h27m0s
2026-07-07 11:04:39 +08:00
CI Test dbd956fc6e Merge develop into main - v0.1.122
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 49h54m54s
CI/CD Pipeline / Frontend Lint (push) Failing after 49h54m54s
CI/CD Pipeline / Deploy Staging (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
2026-07-07 10:59:31 +08:00
CI Test 1d59ee5336 Merge develop into main - v0.1.121
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 52h48m47s
CI/CD Pipeline / Frontend Lint (push) Failing after 52h48m2s
CI/CD Pipeline / Deploy Staging (push) Failing after 52h46m33s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 52h45m33s
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
2026-07-07 08:05:42 +08:00
686 changed files with 28731 additions and 109010 deletions
-1
View File
@@ -1 +0,0 @@
re-trigger
+1 -1
View File
@@ -1 +1 @@
trigger: 1784009947
# CI trigger Fri Jun 26 09:53:28 PM CST 2026
+4 -12
View File
@@ -3,7 +3,6 @@
# ==================== 应用配置 ====================
APP_NAME=小虾 SaaS
APP_BASE_URL=http://localhost:3000
APP_ENV=development
# ==================== 数据库配置 ====================
DATABASE_URL=postgresql://xiaoxia_user:your_password@localhost:5432/xiaoxia_saas
@@ -36,8 +35,7 @@ ENVIRONMENT=development
DEBUG=true
# ==================== CORS 配置 ====================
# 逗号分隔的域名列表(Settings 读取 CORS_ORIGINS_RAW
CORS_ORIGINS_RAW=http://localhost:3000,http://localhost:5173
CORS_ORIGINS=["http://localhost:3000","http://localhost:5173"]
# ==================== 阿里云 OSS 配置 ====================
OSS_ENDPOINT=oss-cn-hangzhou.aliyuncs.com
@@ -46,15 +44,9 @@ OSS_ACCESS_KEY_SECRET=your-access-key-secret
OSS_BUCKET_NAME=xiaoxia-autocut
# ==================== CosyVoice 语音合成配置 ====================
# 注意:base_url 只需写到 /api/v1,具体路径由代码拼接
# 模型: cosyvoice-v3-flash (推荐,支持系统音色,性价比高)
# cosyvoice-v3-plus (高质量,系统音色少)
# cosyvoice-v3.5-flash / cosyvoice-v3.5-plus (仅支持克隆/设计音色,无系统音色)
# 音色: v3系列系统音色带 _v3 后缀,如 longxiaochun_v3, longxiaoxia_v3, longanyang (无后缀)
# 注意:COSYVOICE_* 变量由 packages/shared/config.py 的 SharedSettings 读取
COSYVOICE_API_KEY=your-cosyvoice-api-key
COSYVOICE_BASE_URL=https://dashscope.aliyuncs.com/api/v1
COSYVOICE_MODEL=cosyvoice-v3-flash
COSYVOICE_VOICE=longxiaochun_v3
COSYVOICE_BASE_URL=https://dashscope.aliyuncs.com/api/v1/services/aigc/text2audio
COSYVOICE_MODEL=cosyvoice-v1
COSYVOICE_VOICE=longxiaochun
COSYVOICE_SAMPLE_RATE=22050
COSYVOICE_FORMAT=mp3
Executable → Regular
+4 -8
View File
@@ -5,6 +5,7 @@ APP_ENV=production
ENVIRONMENT=production
DEBUG=false
USE_IN_MEMORY_DB=false
LOG_LEVEL=WARNING
# ==================== 数据库(必须修改)====================
DATABASE_URL=postgresql://prod_user:CHANGE_THIS_PASSWORD@db-prod:5432/xiaoxia_prod
@@ -41,15 +42,10 @@ OSS_DIRECT_UPLOAD_MAX_MB=2000
OSS_DIRECT_UPLOAD_EXPIRE_SECONDS=900
# ==================== CosyVoice 语音合成(必须配置)====================
# 注意:base_url 只需写到 /api/v1,具体路径由代码拼接
# 模型: cosyvoice-v3-flash (推荐,支持系统音色,性价比高)
# cosyvoice-v3-plus (高质量,系统音色少)
# cosyvoice-v3.5-flash / cosyvoice-v3.5-plus (仅支持克隆/设计音色,无系统音色)
# 音色: v3系列系统音色带 _v3 后缀,如 longxiaochun_v3, longxiaoxia_v3, longanyang (无后缀)
COSYVOICE_API_KEY=CHANGE_ME_COSYVOICE_API_KEY
COSYVOICE_BASE_URL=https://dashscope.aliyuncs.com/api/v1
COSYVOICE_MODEL=cosyvoice-v3-flash
COSYVOICE_VOICE=longxiaochun_v3
COSYVOICE_BASE_URL=https://dashscope.aliyuncs.com/api/v1/services/aigc/text2audio
COSYVOICE_MODEL=cosyvoice-v1
COSYVOICE_VOICE=longxiaochun
COSYVOICE_SAMPLE_RATE=22050
COSYVOICE_FORMAT=mp3
+13
View File
@@ -0,0 +1,13 @@
[flake8]
max-line-length = 120
exclude =
.git,
__pycache__,
.venv,
venv,
node_modules,
alembic
per-file-ignores =
tests/integration/*:F821
tests/unit/*:F821
+161
View File
@@ -0,0 +1,161 @@
name: ACR Cleanup
on:
schedule:
- cron: '0 19 * * *' # UTC 19:00 = 北京时间凌晨3:00
workflow_dispatch:
inputs:
pr_sha:
description: "PR commit SHA(仅清理指定PR镜像,留空则全量清理)"
required: false
default: ""
dry_run:
description: "预览模式(dry-run),不实际删除"
required: false
default: "true"
pull_request_target:
types: [closed]
branches: [develop, main]
concurrency:
group: acr-cleanup-${{ gitea.ref }}
cancel-in-progress: false
jobs:
cleanup:
name: ACR Image Cleanup
runs-on: ci-l2
timeout-minutes: 20
permissions:
contents: read
env:
ACR_REGISTRY: xiaoxia-registry.cn-hangzhou.cr.aliyuncs.com
ACR_NAMESPACE: xiaoxiakeji
ACR_SERVICE: registry.aliyuncs.com:cn-hangzhou:china:cri-fvec8o9q4mmxrkaa
GITEA_URL: https://git.xiaoxiajianji.com
GITEA_REPO: xiaoxia/xiaoxia-saas
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sH "Authorization: token $GITHUB_TOKEN" "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" | bash
# ====== Cron模式:获取staging运行中镜像作为白名单 ======
- name: Get staging running images (whitelist)
id: protected_images
if: gitea.event_name != 'pull_request_target' && !gitea.event.inputs.pr_sha
env:
STAGING_SSH_KEY: ${{ secrets.PREVIEW_SSH_KEY }}
run: |
set +e
echo "获取staging服务器运行中镜像作为白名单..."
mkdir -p ~/.ssh
echo "$STAGING_SSH_KEY" > ~/.ssh/id_rsa
chmod 600 ~/.ssh/id_rsa
staging_host="${STAGING_SSH_HOST:-47.98.113.167}"
staging_port="${STAGING_SSH_PORT:-22222}"
ssh-keyscan -p "$staging_port" -H "$staging_host" >> ~/.ssh/known_hosts 2>/dev/null
# 获取所有运行容器的镜像,提取tag部分
IMAGES=$(ssh -p "$staging_port" -i ~/.ssh/id_rsa -o StrictHostKeyChecking=no \
"root@$staging_host" "docker ps --format '{{.Image}}' 2>/dev/null" 2>/dev/null | grep -v "^$" | sort -u)
PROTECTED_TAGS=""
if [ -n "$IMAGES" ]; then
while IFS= read -r img; do
# 从完整镜像名中提取tag(最后一个冒号后)
tag=$(echo "$img" | rev | cut -d: -f1 | rev)
if [ -n "$tag" ] && [ "$tag" != "latest" ] && [ ${#tag} -gt 5 ]; then
if [ -z "$PROTECTED_TAGS" ]; then
PROTECTED_TAGS="$tag"
else
PROTECTED_TAGS="$PROTECTED_TAGS,$tag"
fi
fi
done <<< "$IMAGES"
fi
echo "staging运行中镜像tag: ${PROTECTED_TAGS:-(无)}"
echo "protected_tags=$PROTECTED_TAGS" >> $GITEA_OUTPUT
# ====== Docker登录 ======
- name: Docker login to ACR
env:
ACR_USERNAME: ${{ secrets.ACR_USERNAME }}
ACR_PASSWORD: ${{ secrets.ACR_PASSWORD }}
run: |
printf '%s' "$ACR_PASSWORD" | docker login "$ACR_REGISTRY" -u "$ACR_USERNAME" --password-stdin
# ====== 模式1PR关闭时清理 ======
- name: Cleanup PR images (PR closed)
if: gitea.event_name == 'pull_request_target'
env:
ACR_USERNAME: ${{ secrets.ACR_USERNAME }}
ACR_PASSWORD: ${{ secrets.ACR_PASSWORD }}
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
PR_SHA: ${{ gitea.event.pull_request.head.sha }}
PR_NUMBER: ${{ gitea.event.pull_request.number }}
run: |
echo "============================================"
echo " PR #$PR_NUMBER 已关闭,清理对应镜像"
echo " Head SHA: ${PR_SHA::12}"
echo "============================================"
echo ""
python3 scripts/ci/acr_cleanup.py \
--pr-sha "$PR_SHA" \
--execute
# ====== 模式2Cron全量清理 ======
- name: Full cleanup (cron / manual)
if: gitea.event_name != 'pull_request_target' && !gitea.event.inputs.pr_sha
env:
ACR_USERNAME: ${{ secrets.ACR_USERNAME }}
ACR_PASSWORD: ${{ secrets.ACR_PASSWORD }}
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
PROTECTED_TAGS: ${{ steps.protected_images.outputs.protected_tags }}
DRY_RUN_INPUT: ${{ gitea.event.inputs.dry_run }}
run: |
echo "============================================"
echo " ACR 全量清理(${{ gitea.event_name }}"
echo "============================================"
echo ""
# 决定是否dry-run
DRY_RUN_FLAG=""
if [ "$DRY_RUN_INPUT" = "true" ]; then
DRY_RUN_FLAG="--dry-run"
echo "模式: 预览模式 (dry-run)"
else
echo "模式: 执行模式"
fi
echo ""
python3 scripts/ci/acr_cleanup.py \
--keep 20 \
--protected-tags "$PROTECTED_TAGS" \
$DRY_RUN_FLAG
# ====== 模式3:手动指定PR SHA清理 ======
- name: Cleanup specific PR image (manual)
if: gitea.event_name == 'workflow_dispatch' && gitea.event.inputs.pr_sha
env:
ACR_USERNAME: ${{ secrets.ACR_USERNAME }}
ACR_PASSWORD: ${{ secrets.ACR_PASSWORD }}
PR_SHA: ${{ gitea.event.inputs.pr_sha }}
DRY_RUN_INPUT: ${{ gitea.event.inputs.dry_run }}
run: |
echo "手动清理PR镜像: ${PR_SHA::12}"
echo ""
DRY_RUN_FLAG=""
if [ "$DRY_RUN_INPUT" = "true" ]; then
DRY_RUN_FLAG="--dry-run"
fi
python3 scripts/ci/acr_cleanup.py \
--pr-sha "$PR_SHA" \
$DRY_RUN_FLAG
-191
View File
@@ -1,191 +0,0 @@
name: Auto Approve CI PRs
on:
pull_request:
types: [synchronize, opened, ready_for_review]
jobs:
auto-approve:
name: Auto Approve on CI Green
runs-on: ci-check
if: github.event_name == 'pull_request' && !github.event.pull_request.draft
timeout-minutes: 20
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: "set -eu\npython3 - <<'PY'\nimport io, os, tarfile, time, urllib.request, urllib.error\nurl = f\"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz\"\nrequest = urllib.request.Request(url, headers={\"Authorization\": f\"token {os.environ['GITHUB_TOKEN']}\"})\nlast_err = None\nfor attempt in range(5):\n try:\n with urllib.request.urlopen(request, timeout=120) as response:\n archive = response.read()\n break\n except urllib.error.HTTPError as e:\n last_err = e\n if e.code >= 500 and attempt < 4:\n wait = 2 ** attempt\n print(f\"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...\")\n time.sleep(wait)\n continue\n raise\n except Exception as e:\n last_err = e\n if attempt < 4:\n wait = 2 ** attempt\n print(f\"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...\")\n time.sleep(wait)\n continue\n raise\nelse:\n raise last_err\nwith tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:\n root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'\n for member in tar.getmembers():\n name = member.name\n if name == root_prefix[:-1]:\n continue\n if name.startswith(root_prefix):\n member.name = name[len(root_prefix):]\n if member.name:\n tar.extract(member, '.')\nPY\n"
- name: Record start time & report trace start
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\necho "CI_TRACE_START_TIME=$(date +%s)" >> $GITHUB_ENV\nTRACE_SERVICE="auto-approve" TRACE_NAME="自动批准 / Auto Approve on CI Green" python3 .gitea/scripts/ci_trace_report.py --status running\n\nexit 0\n"
- name: Auto approve when CI passes
shell: bash
env:
GITHUB_TOKEN: ${{ github.token }}
REVIEW_TOKEN: ${{ secrets.REVIEW_GITEA_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
set -eu
echo "PR #${PR_NUMBER} - 检查CI状态并自动审批"
# 检查是否纯前端改动
API_URL="${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/files?limit=300"
FILES=$(curl -s -H "Authorization: token ${GITHUB_TOKEN}" "$API_URL" | python3 -c "import sys,json; [print(f['filename']) for f in json.load(sys.stdin)]")
FRONTEND_COUNT=$(echo "$FILES" | grep -c '^apps/web/' || true)
BACKEND_COUNT=$(echo "$FILES" | grep -cv '^apps/web/' || true)
TOTAL=$(echo "$FILES" | grep -cv '^$' || true)
echo "变更文件: ${TOTAL} 个 (前端: ${FRONTEND_COUNT}, 后端/公共: ${BACKEND_COUNT})"
if [ "$BACKEND_COUNT" = "0" ] && [ "$FRONTEND_COUNT" -gt "0" ]; then
SKIP_BACKEND=true
echo "✅ 纯前端改动,只检查Frontend Lint"
else
SKIP_BACKEND=false
echo "🔧 包含后端/公共变更,检查全部CI"
fi
# 定义需要检查的context
if [ "$SKIP_BACKEND" = "true" ]; then
CONTEXTS=("CI/CD Pipeline / Frontend Lint (pull_request)")
else
CONTEXTS=(
"CI/CD Pipeline / Validate Code Quality And Tests (pull_request)"
"CI/CD Pipeline / Unit Tests (pull_request)"
"CI/CD Pipeline / Frontend Lint (pull_request)"
)
fi
echo "需要通过的CI检查: ${#CONTEXTS[@]} 项"
for ctx in "${CONTEXTS[@]}"; do
echo " - $ctx"
done
echo
# 初始等待30秒,给CI启动写status的时间,避免checkout太快导致全找不到context误判
echo "等待30秒让CI启动..."
sleep 30
# 轮询等待,最多20分钟(120次x10秒)
for attempt in $(seq 1 120); do
ALL_SUCCESS=true
ANY_FAILED=false
ANY_PENDING=false
echo "--- 第${attempt}次检查 ($(date '+%H:%M:%S')) ---"
# 调用辅助脚本检查每个context状态
for ctx in "${CONTEXTS[@]}"; do
STATE=$(python3 scripts/check_ci_status.py "$GITHUB_TOKEN" "$GITHUB_REPOSITORY" "$PR_HEAD_SHA" "$ctx")
echo " $ctx: $STATE"
if [ "$STATE" != "success" ]; then
ALL_SUCCESS=false
fi
if [ "$STATE" = "failure" ] || [ "$STATE" = "error" ]; then
ANY_FAILED=true
fi
if [ "$STATE" = "pending" ] || [ "$STATE" = "null" ]; then
ANY_PENDING=true
fi
done
if [ "$ALL_SUCCESS" = "true" ]; then
echo
echo "✅ 所有CI检查通过,自动审批 PR #${PR_NUMBER}"
# 检查是否已有审批(任何用户的APPROVED都算,避免重复审批)
EXISTING=$(curl -s -H "Authorization: token ${REVIEW_TOKEN}" \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/reviews" \
| python3 -c "import sys,json; reviews=json.load(sys.stdin); print('yes' if any(r.get('state')=='APPROVED' for r in reviews) else 'no')")
if [ "$EXISTING" = "yes" ]; then
echo "️ PR #${PR_NUMBER} 已有审批,跳过"
exit 0
fi
# 第一步:创建PENDING reviewGitea API需要先创建再提交)
echo "创建review..."
REVIEW_CREATE=$(curl -s -X POST \
-H "Authorization: token ${REVIEW_TOKEN}" \
-H "Content-Type: application/json" \
-d '{"event": "PENDING", "body": "CI全绿,自动审批通过。"}' \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/reviews")
REVIEW_ID=$(echo "$REVIEW_CREATE" | python3 -c "import sys,json; print(json.load(sys.stdin).get('id',''))")
REVIEW_STATE=$(echo "$REVIEW_CREATE" | python3 -c "import sys,json; print(json.load(sys.stdin).get('state',''))")
echo "创建结果: id=$REVIEW_ID state=$REVIEW_STATE"
if [ -z "$REVIEW_ID" ]; then
echo "❌ 创建review失败"
echo "$REVIEW_CREATE"
exit 1
fi
# 如果已经是APPROVED就不用再submit了(兼容不同Gitea版本)
if [ "$REVIEW_STATE" = "APPROVED" ]; then
echo "✅ 自动审批成功(直接创建为APPROVED"
exit 0
fi
# 第二步:submit review为APPROVED
echo "提交review审批..."
SUBMIT_CODE=$(curl -s -o /tmp/submit_resp.json -w "%{http_code}" \
-X POST \
-H "Authorization: token ${REVIEW_TOKEN}" \
-H "Content-Type: application/json" \
-d '{"event": "APPROVED", "body": "CI全绿,自动审批通过。"}' \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/reviews/${REVIEW_ID}")
echo "提交API HTTP状态: $SUBMIT_CODE"
cat /tmp/submit_resp.json 2>/dev/null || true
echo
if [ "$SUBMIT_CODE" = "200" ] || [ "$SUBMIT_CODE" = "201" ]; then
FINAL_STATE=$(python3 -c "import json; print(json.load(open('/tmp/submit_resp.json')).get('state',''))" 2>/dev/null || echo "?")
echo "✅ 自动审批成功 (state: $FINAL_STATE)"
exit 0
else
echo "❌ 提交审批失败"
exit 1
fi
fi
# 还有CI在跑(pending状态)→ 继续等
if [ "$ANY_PENDING" = "true" ]; then
echo "⏳ CI仍在运行中,继续等待(第${attempt}/120次轮询)..."
sleep 10
continue
fi
# 所有CI都跑完了但有失败 → 退出
if [ "$ANY_FAILED" = "true" ]; then
echo
echo "❌ CI检查有失败项,不自动审批"
exit 0
fi
# 其他情况继续等
sleep 10
done
echo
echo "⏰ 等待超时(20分钟),CI尚未全部完成"
exit 0
- name: Report trace end (always)
if: always()
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\nif [ "${{ job.status }}" = "success" ]; then\n TRACE_STATUS="ok"\nelse\n TRACE_STATUS="error"\nfi\n\nTRACE_SERVICE="auto-approve" TRACE_NAME="自动批准 / Auto Approve on CI Green" TRACE_STATUS="$TRACE_STATUS" python3 .gitea/scripts/ci_trace_report.py --start-time "$CI_TRACE_START_TIME"\n\nexit 0\n"
-191
View File
@@ -1,191 +0,0 @@
name: Auto Merge CI PRs
on:
pull_request:
types: [synchronize, opened, ready_for_review, review_requested]
jobs:
auto-merge:
name: Auto Merge on CI Green + Approved
runs-on: ci-check
if: github.event_name == 'pull_request' && !github.event.pull_request.draft && github.event.pull_request.base.ref == 'develop'
timeout-minutes: 30
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: "set -eu\npython3 - <<'PY'\nimport io, os, tarfile, time, urllib.request, urllib.error\nurl = f\"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz\"\nrequest = urllib.request.Request(url, headers={\"Authorization\": f\"token {os.environ['GITHUB_TOKEN']}\"})\nlast_err = None\nfor attempt in range(5):\n try:\n with urllib.request.urlopen(request, timeout=120) as response:\n archive = response.read()\n break\n except urllib.error.HTTPError as e:\n last_err = e\n if e.code >= 500 and attempt < 4:\n wait = 2 ** attempt\n print(f\"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...\")\n time.sleep(wait)\n continue\n raise\n except Exception as e:\n last_err = e\n if attempt < 4:\n wait = 2 ** attempt\n print(f\"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...\")\n time.sleep(wait)\n continue\n raise\nelse:\n raise last_err\nwith tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:\n root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'\n for member in tar.getmembers():\n name = member.name\n if name == root_prefix[:-1]:\n continue\n if name.startswith(root_prefix):\n member.name = name[len(root_prefix):]\n if member.name:\n tar.extract(member, '.')\nPY\n"
- name: Record start time & report trace start
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\necho "CI_TRACE_START_TIME=$(date +%s)" >> $GITHUB_ENV\nTRACE_SERVICE="auto-merge" TRACE_NAME="自动合并 / Auto Merge on CI Green + Approved" python3 .gitea/scripts/ci_trace_report.py --status running\n\nexit 0\n"
- name: Auto merge when CI passes and approved
shell: bash
env:
GITHUB_TOKEN: ${{ github.token }}
MERGE_TOKEN: ${{ secrets.REVIEW_GITEA_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
BASE_REF: ${{ github.event.pull_request.base.ref }}
run: |
set -eu
echo "PR #${PR_NUMBER} - 检查CI状态+审批并自动合并到${BASE_REF}"
echo
# 只合develop分支
if [ "$BASE_REF" != "develop" ]; then
echo "Skip: 目标分支不是develop"
exit 0
fi
# 判断是否纯前端改动
FILES=$(curl -s -H "Authorization: token ${GITHUB_TOKEN}" \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/files?limit=300" \
| python3 -c "import sys,json; [print(f['filename']) for f in json.load(sys.stdin)]")
TOTAL=$(echo "$FILES" | grep -cv '^$' || true)
FRONTEND_COUNT=$(echo "$FILES" | grep -c '^apps/web/' || true)
BACKEND_COUNT=$((TOTAL - FRONTEND_COUNT))
echo "变更文件: ${TOTAL} 个 (前端: ${FRONTEND_COUNT}, 后端/公共: ${BACKEND_COUNT})"
if [ "$BACKEND_COUNT" = "0" ] && [ "$FRONTEND_COUNT" -gt "0" ]; then
CONTEXTS=("CI/CD Pipeline / Frontend Lint (pull_request)")
echo "纯前端改动,只检查Frontend Lint"
else
CONTEXTS=(
"CI/CD Pipeline / Validate Code Quality And Tests (pull_request)"
"CI/CD Pipeline / Unit Tests (pull_request)"
"CI/CD Pipeline / Frontend Lint (pull_request)"
"CI/CD Pipeline / Integration Tests (pull_request)"
)
echo "检查全部四门禁"
fi
echo
# 初始等待30秒,给CI启动写status的时间,避免checkout太快导致全找不到context误判
echo "等待30秒让CI启动..."
sleep 30
# 405连续计数器:连续多次合并返回405才放弃
MERGE_405_COUNT=0
MAX_405_RETRIES=10
# 轮询等待,最多30分钟(180次x10秒)
for attempt in $(seq 1 180); do
ALL_SUCCESS=true
ANY_FAILED=false
ANY_PENDING=false
echo "--- 第${attempt}次检查 ($(date '+%H:%M:%S')) ---"
# 检查CI状态
for ctx in "${CONTEXTS[@]}"; do
STATE=$(python3 scripts/check_ci_status.py "$GITHUB_TOKEN" "$GITHUB_REPOSITORY" "$PR_HEAD_SHA" "$ctx")
echo " CI: ${ctx##*/}: $STATE"
if [ "$STATE" != "success" ]; then
ALL_SUCCESS=false
fi
if [ "$STATE" = "failure" ] || [ "$STATE" = "error" ]; then
ANY_FAILED=true
fi
if [ "$STATE" = "pending" ]; then
ANY_PENDING=true
fi
done
# 检查审批状态
APPROVAL_RESULT=$(python3 scripts/check_pr_approval.py "$MERGE_TOKEN" "$GITHUB_REPOSITORY" "$PR_NUMBER" 1)
echo " 审批: $APPROVAL_RESULT"
HAS_APPROVAL=false
if echo "$APPROVAL_RESULT" | grep -q '^approved'; then
HAS_APPROVAL=true
fi
# 全部满足 → 合并
if [ "$ALL_SUCCESS" = "true" ] && [ "$HAS_APPROVAL" = "true" ]; then
echo
echo "CI全绿 + 审批通过,执行自动合并"
echo "等待60秒冷却,给Gitea内部状态同步时间..."
sleep 60
# 幂等检查:PR是否还是open
PR_STATE=$(curl -s -H "Authorization: token ${MERGE_TOKEN}" \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}" \
| python3 -c "import sys,json; print(json.load(sys.stdin).get('state',''))")
if [ "$PR_STATE" != "open" ]; then
echo "PR状态为 ${PR_STATE},无需合并"
exit 0
fi
# 执行squash merge
HTTP_CODE=$(curl -s -o /tmp/merge_resp.json -w "%{http_code}" \
-X POST \
-H "Authorization: token ${MERGE_TOKEN}" \
-H "Content-Type: application/json" \
-d '{"do":"squash","merge_title_field":"","merge_message_field":"","delete_branch_after_merge":true,"force_merge":false}' \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/merge")
echo "合并API HTTP状态: $HTTP_CODE"
if [ "$HTTP_CODE" = "200" ]; then
echo "自动合并成功"
exit 0
elif [ "$HTTP_CODE" = "405" ]; then
MERGE_405_COUNT=$((MERGE_405_COUNT + 1))
echo "⚠️ 合并返回405(第${MERGE_405_COUNT}次),可能CI状态尚未同步或有未解决的门禁,继续等待重试..."
cat /tmp/merge_resp.json 2>/dev/null || true
echo
if [ "$MERGE_405_COUNT" -ge "$MAX_405_RETRIES" ]; then
echo "⚠️ 连续${MAX_405_RETRIES}次合并返回405,放弃自动合并(需人工确认,非代码问题)"
curl -s -X POST \
-H "Authorization: token ${MERGE_TOKEN}" \
-H "Content-Type: application/json" \
-d '{"body": "Auto merge skipped after multiple 405 errors: PR may have conflicts or unresolved checks. Please review manually. This is not a CI failure."}' \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/comments" > /dev/null 2>&1 || true
exit 0
fi
sleep 30
continue
else
echo "自动合并失败 (HTTP $HTTP_CODE)"
cat /tmp/merge_resp.json 2>/dev/null || true
curl -s -X POST \
-H "Authorization: token ${MERGE_TOKEN}" \
-H "Content-Type: application/json" \
-d "{\"body\": \"Auto merge failed (HTTP ${HTTP_CODE}), please check manually.\"}" \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/comments" > /dev/null 2>&1 || true
exit 1
fi
else
# 本轮不满足合并条件,重置405计数器
MERGE_405_COUNT=0
fi
if [ "$ANY_FAILED" = "true" ]; then
echo
echo "CI有失败项,不自动合并"
exit 0
fi
sleep 10
done
echo
echo "等待超时(30分钟)"
exit 0
- name: Report trace end (always)
if: always()
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\nif [ "${{ job.status }}" = "success" ]; then\n TRACE_STATUS="ok"\nelse\n TRACE_STATUS="error"\nfi\n\nTRACE_SERVICE="auto-merge" TRACE_NAME="自动合并 / Auto Merge on CI Green + Approved" TRACE_STATUS="$TRACE_STATUS" python3 .gitea/scripts/ci_trace_report.py --start-time "$CI_TRACE_START_TIME"\n\nexit 0\n"
File diff suppressed because one or more lines are too long
-741
View File
@@ -1,741 +0,0 @@
name: CI/CD Pipeline
on:
push:
branches:
- main
- develop
tags:
- v*
pull_request:
branches:
- main
- develop
workflow_dispatch:
inputs:
reason:
description: "触发原因"
required: false
default: "手动触发 - CI漏触发补跑"
permissions:
contents: read
concurrency:
group: ci-cd-${{ gitea.event_name }}-${{ gitea.ref }}
cancel-in-progress: true
jobs:
check-frontend-only:
name: Check if frontend-only change
runs-on: ci-check
if: github.event_name == 'pull_request'
outputs:
skip_backend: ${{ steps.check.outputs.skip_backend }}
skip_frontend: ${{ steps.check.outputs.skip_frontend }}
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: "set -eu\npython3 - <<'PY'\nimport io, os, tarfile, time, urllib.request, urllib.error\nurl = f\"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz\"\nrequest = urllib.request.Request(url, headers={\"Authorization\": f\"token {os.environ['GITHUB_TOKEN']}\"})\nlast_err = None\nfor attempt in range(5):\n try:\n with urllib.request.urlopen(request, timeout=120) as response:\n archive = response.read()\n break\n except urllib.error.HTTPError as e:\n last_err = e\n if e.code >= 500 and attempt < 4:\n wait = 2 ** attempt\n print(f\"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...\")\n time.sleep(wait)\n continue\n raise\n except Exception as e:\n last_err = e\n if attempt < 4:\n wait = 2 ** attempt\n print(f\"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...\")\n time.sleep(wait)\n continue\n raise\nelse:\n raise last_err\nwith tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:\n root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'\n for member in tar.getmembers():\n name = member.name\n if name == root_prefix[:-1]:\n continue\n if name.startswith(root_prefix):\n member.name = name[len(root_prefix):]\n if member.name:\n tar.extract(member, '.')\nPY\n"
- name: Record start time & report trace start
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\necho "CI_TRACE_START_TIME=$(date +%s)" >> $GITHUB_ENV\nTRACE_SERVICE="" TRACE_NAME="" python3 .gitea/scripts/ci_trace_report.py --status running\n\nexit 0\n"
- name: Check changed files
id: check
shell: bash
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
PR_NUMBER=$(echo "$GITHUB_REF" | sed 's|refs/pull/||; s|/.*||')
API_URL="${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/files?limit=300"
FILES=$(curl -s -H "Authorization: token ${GITHUB_TOKEN}" "$API_URL" | python3 -c "import sys,json; [print(f['filename']) for f in json.load(sys.stdin)]")
FRONTEND_COUNT=$(echo "$FILES" | grep -c '^apps/web/' || true)
BACKEND_COUNT=$(echo "$FILES" | grep -cv '^apps/web/' || true)
TOTAL=$(echo "$FILES" | grep -cv '^$' || true)
echo "变更文件: ${TOTAL} 个 (前端: ${FRONTEND_COUNT}, 后端/公共: ${BACKEND_COUNT})"
if [ "$BACKEND_COUNT" = "0" ] && [ "$FRONTEND_COUNT" -gt "0" ]; then
echo "skip_backend=true" >> $GITHUB_OUTPUT
echo "skip_frontend=false" >> $GITHUB_OUTPUT
echo "✅ 纯前端改动,跳过后端检查"
elif [ "$FRONTEND_COUNT" = "0" ] && [ "$BACKEND_COUNT" -gt "0" ]; then
echo "skip_backend=false" >> $GITHUB_OUTPUT
echo "skip_frontend=true" >> $GITHUB_OUTPUT
echo "🔧 纯后端改动,跳过前端检查"
else
echo "skip_backend=false" >> $GITHUB_OUTPUT
echo "skip_frontend=false" >> $GITHUB_OUTPUT
echo "🔧 包含全栈变更,运行完整CI"
fi
- name: Report trace end (always)
if: always()
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\nif [ "${{ job.status }}" = "success" ]; then\n TRACE_STATUS="ok"\nelse\n TRACE_STATUS="error"\nfi\n\nTRACE_SERVICE="" TRACE_NAME="" TRACE_STATUS="$TRACE_STATUS" python3 .gitea/scripts/ci_trace_report.py --start-time "$CI_TRACE_START_TIME"\n\nexit 0\n"
validate:
needs: check-frontend-only
if: always() && needs.check-frontend-only.outputs.skip_backend != 'true'
name: Validate Code Quality And Tests
runs-on: ci-check
timeout-minutes: 10
permissions:
contents: write
env:
DATABASE_URL: postgresql+psycopg://postgres:postgres@127.0.0.1:5432/xiaoxia_saas
USE_IN_MEMORY_DB: 'false'
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: "set -eu\npython3 - <<'PY'\nimport io, os, tarfile, time, urllib.request, urllib.error\nurl = f\"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz\"\nrequest = urllib.request.Request(url, headers={\"Authorization\": f\"token {os.environ['GITHUB_TOKEN']}\"})\nlast_err = None\nfor attempt in range(5):\n try:\n with urllib.request.urlopen(request, timeout=120) as response:\n archive = response.read()\n break\n except urllib.error.HTTPError as e:\n last_err = e\n if e.code >= 500 and attempt < 4:\n wait = 2 ** attempt\n print(f\"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...\")\n time.sleep(wait)\n continue\n raise\n except Exception as e:\n last_err = e\n if attempt < 4:\n wait = 2 ** attempt\n print(f\"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...\"\
)\n time.sleep(wait)\n continue\n raise\nelse:\n raise last_err\nwith tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:\n root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'\n for member in tar.getmembers():\n name = member.name\n if name == root_prefix[:-1]:\n continue\n if name.startswith(root_prefix):\n member.name = name[len(root_prefix):]\n if member.name:\n tar.extract(member, '.')\nPY\n"
- name: Record start time & report trace start
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\necho "CI_TRACE_START_TIME=$(date +%s)" >> $GITHUB_ENV\nTRACE_SERVICE="" TRACE_NAME="" python3 .gitea/scripts/ci_trace_report.py --status running\n\nexit 0\n"
- name: Record job start time
shell: sh
run: 'set -eu
echo "JOB_START_TIME=$(date +%s)" >> $GITHUB_ENV
echo "Job started at $(date)"
'
- name: Verify CI environment
shell: sh
run: 'set -eu
python3 --version
python3 -m pip --version
echo "CI environment is ready"
'
- name: Install dependencies
shell: sh
run: 'set -eu
python3 -m pip install -q -r requirements-base.txt
python3 -m pip install -q -r requirements.txt
python3 -m pip install -q -r requirements-dev.txt
# Force source install of black/isort to ensure consistent formatting
# across compiled/source installations on different machines
python3 -m pip install --no-binary :all: black==26.5.1 isort==8.0.1
python3 -m black --version
python3 -m isort --version-number
python3 -m ruff --version
bandit --version
pytest --version
'
- name: Secret detection (detect-secrets)
shell: sh
run: "set -eu\necho \"=== Installing detect-secrets ===\"\npython3 -m pip install -q detect-secrets\ndetect-secrets --version\necho \"\"\necho \"=== Running secret scan ===\"\ndetect-secrets scan \\\n --all-files \\\n --exclude-files '(^|/)(tests|test|e2e|__tests__|spec|docs|node_modules|site-packages|migrations|alembic|.gitea|.git|.pytest_cache|.next|dist|build)/' \\\n --exclude-files '\\.(md|rst|txt|lock|example|sample|min\\.js|min\\.css|spec\\.ts|test\\.ts|test\\.py)$' \\\n --exclude-files '(package-lock|yarn\\.lock|poetry\\.lock|Pipfile\\.lock)$' \\\n --disable-plugin Base64HighEntropyString \\\n --disable-plugin HexHighEntropyString \\\n --disable-plugin BasicAuthDetector \\\n --disable-plugin KeywordDetector \\\n --disable-plugin IPPublicDetector \\\n > /tmp/secrets-scan.json 2>&1\n\nFOUND=$(python3 -c \"\nimport json\ntry:\n with open('/tmp/secrets-scan.json') as f:\n data = json.load(f)\n results = data.get('results', {})\n total = sum(len(v) for\
\ v in results.values())\n print(total)\nexcept Exception:\n print('error')\n\")\necho \"\"\necho \"Secrets detected: $FOUND\"\nif [ \"$FOUND\" != \"0\" ] && [ \"$FOUND\" != \"error\" ]; then\n echo \"\"\n echo \"=== Secret details ===\"\n python3 -c \"\nimport json\nwith open('/tmp/secrets-scan.json') as f:\n data = json.load(f)\nfor fpath, items in data.get('results', {}).items():\n for item in items:\n line = item.get('line_number', '?')\n stype = item.get('type', '?')\n hashed = item.get('hashed_secret', '')[:16]\n print(f' {fpath}:{line} [{stype}] {hashed}...')\n\"\n echo \"\"\n echo \"ERROR: Potential secrets detected in code!\"\n echo \"If these are false positives, add exclusions in the CI workflow.\"\n exit 1\nfi\necho \"Secret scan completed - no secrets detected\"\n"
- name: Calculate changed Python files (incremental scan)
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: "set -eu\nSCAN_MODE=\"full\"\nCHANGED_PY_FILES=\"\"\n\nif [ \"${GITHUB_EVENT_NAME:-}\" = \"pull_request\" ] && [ -n \"${GITHUB_REF_NAME:-}\" ]; then\n echo \"PR mode (#${GITHUB_REF_NAME}) - fetching changed files from API\"\n\n PR_NUMBER=$(echo \"$GITHUB_REF\" | sed 's|refs/pull/||; s|/.*||')\n API_URL=\"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/files?limit=100\"\n\n set +e\n RESPONSE=$(curl -s -w \"\\n%{http_code}\" -H \"Authorization: token ${GITHUB_TOKEN}\" \"${API_URL}\")\n HTTP_CODE=$(echo \"$RESPONSE\" | tail -n1)\n BODY=$(echo \"$RESPONSE\" | sed '$d')\n set -e\n\n if [ \"$HTTP_CODE\" = \"200\" ]; then\n CHANGED_PY_FILES=$(echo \"$BODY\" | python3 -c \"\nimport json, sys\ntry:\n files = json.load(sys.stdin)\n py_files = [f['filename'] for f in files\n if f['filename'].endswith('.py') and f['status'] != 'removed']\n print(' '.join(py_files))\nexcept Exception:\n print('')\n\")\n if [ -n \"$CHANGED_PY_FILES\" ]; then\n SCAN_MODE=\"incremental\"\n FILE_COUNT=$(echo \"$CHANGED_PY_FILES\" | wc -w)\n echo \"Changed Python files: ${FILE_COUNT}\"\n echo \"$CHANGED_PY_FILES\" | tr ' ' '\\n' | grep -v '^$'\n else\n SCAN_MODE=\"skip_py\"\n echo \"No Python files changed in this PR\"\n fi\n else\n echo \"WARN: API returned HTTP $HTTP_CODE, falling back to full scan\"\n fi\nelse\n echo \"Full scan mode (not a PR event)\"\nfi\n\necho \"SCAN_MODE=$SCAN_MODE\" >> $GITHUB_ENV\necho \"CHANGED_PY_FILES=$CHANGED_PY_FILES\" >> $GITHUB_ENV\n"
- name: Run code quality checks
shell: sh
run: "set -eu\n\nif [ \"$SCAN_MODE\" = \"incremental\" ]; then\n echo \"=== Incremental scan mode ===\"\n\n python3 -m compileall -q $CHANGED_PY_FILES\n\n python3 -m black --check --fast $CHANGED_PY_FILES\n\n python3 -m isort --check-only $CHANGED_PY_FILES\n\n RUFF_FILES=$(echo \"$CHANGED_PY_FILES\" | tr ' ' '\\n' | grep -v '^scripts/' | tr '\\n' ' ')\n if [ -n \"$RUFF_FILES\" ]; then\n python3 -m ruff check $RUFF_FILES --statistics\n else\n echo \"No ruff-checkable files changed, skipping\"\n fi\n\nelif [ \"$SCAN_MODE\" = \"skip_py\" ]; then\n echo \"No Python files changed - skipping Python lint checks\"\n\nelse\n echo \"=== Full scan mode ===\"\n\n python3 -m compileall -q alembic apps packages tests scripts\n\n python3 -m black --check --fast alembic apps packages tests scripts\n\n python3 -m isort --check-only alembic apps packages tests scripts\n\n python3 -m ruff check apps packages tests --statistics\nfi\n"
- name: Auto-fix formatting (black + isort)
if: failure()
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: python3 scripts/ci/auto_fix_formatting.py
- name: Type check (mypy, hard gate)
shell: sh
run: "bash scripts/ci/mypy_check.sh"
- name: Run security scan (bandit)
shell: sh
run: 'set -eu
bandit -r apps packages -q -ll
'
- name: Python dependency vulnerability scan (pip-audit)
shell: sh
run: "set -eu\necho \"=== Installing pip-audit ===\"\npython3 -m pip install -q pip-audit\npip-audit --version\necho \"\"\necho \"=== Scanning Python dependencies ===\"\nEXIT_CODE=0\nfor req_file in requirements.txt requirements-base.txt requirements-dev.txt; do\n if [ -f \"$req_file\" ]; then\n echo \"--- Scanning $req_file ---\"\n pip-audit -r \"$req_file\" --desc on 2>&1 | head -40 || EXIT_CODE=$?\n echo \"\"\n fi\ndone\necho \"pip-audit scan completed (advisory mode - warnings only, not blocking CI)\"\nif [ \"$EXIT_CODE\" != \"0\" ]; then\n echo \"WARNING: Potential vulnerabilities found in dependencies.\"\nfi\nexit 0\n"
- name: Dead code detection (vulture)
if: always()
shell: sh
run: "set +e\necho \"=== Installing vulture ===\"\npython3 -m pip install -q vulture\nvulture --version\necho \"\"\necho \"=== Running vulture dead code scan (confidence >= 70%) ===\"\necho \"告警模式,不阻断CI。置信度>=90%建议尽快确认。\"\necho \"\"\n# 按置信度从高到低输出,便于优先查看高价值条目\nvulture apps packages scripts \\\n --exclude \"tests,test,migrations,.gitea,docs,node_modules,site-packages,*/test_*.py,*/conftest.py\" \\\n --min-confidence 70 \\\n 2>&1 | sort -t'(' -k2 -rn | head -80\nEXIT_CODE=$?\necho \"\"\necho \"=== vulture scan summary ===\"\nif [ \"$EXIT_CODE\" != \"0\" ]; then\n echo \"发现潜在死代码(可能包含框架装饰器注册的函数,为误报)\"\n echo \"建议:定期人工审查高置信度(>=90%)条目\"\nelse\n echo \"未发现明显死代码 ✅\"\nfi\nexit 0\n"
- name: Validate release scripts syntax
shell: sh
run: 'set -eu
bash -n scripts/backup_postgres.sh
bash -n scripts/restore_postgres_plan.sh
bash -n scripts/init_production_env.sh
'
- name: Validate Alembic migrations (with isolated PG)
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
PG_CONTAINER=ci-pg-validate-${GITHUB_RUN_ID:-$$}
docker rm -f "$PG_CONTAINER" 2>/dev/null || true
docker run -d --name "$PG_CONTAINER" \
--shm-size=256m \
-e POSTGRES_USER=postgres \
-e POSTGRES_PASSWORD=postgres \
-e POSTGRES_DB=xiaoxia_saas \
-P \
--health-cmd "pg_isready -U postgres" \
--health-interval 3s \
--health-timeout 3s \
--health-retries 20 \
postgres:16-alpine
PG_PORT=$(docker port "$PG_CONTAINER" 5432/tcp | cut -d: -f2)
echo "PostgreSQL port: $PG_PORT"
export DATABASE_URL=postgresql+psycopg://postgres:postgres@127.0.0.1:$PG_PORT/xiaoxia_saas
for i in $(seq 1 30); do
if docker inspect --format='{{.State.Health.Status}}' "$PG_CONTAINER" 2>/dev/null | grep -q healthy; then
echo "PostgreSQL is ready on port $PG_PORT"
break
fi
echo "Waiting for PostgreSQL... ($i/30)"
sleep 2
done
docker inspect --format='{{.State.Health.Status}}' "$PG_CONTAINER" | grep -q healthy
python3 -m alembic upgrade head --sql > /tmp/alembic-upgrade.sql
test -s /tmp/alembic-upgrade.sql
grep -q "Running upgrade" /tmp/alembic-upgrade.sql
python3 scripts/check_schema_metadata.py
# Initialize git for migration safety diff (CI checkout is tar.gz without .git)
git init > /dev/null 2>&1
git remote add origin https://git.xiaoxiajianji.com/xiaoxia/xiaoxia-saas.git > /dev/null 2>&1
git fetch origin develop:refs/remotes/origin/develop --depth=100 > /dev/null 2>&1
git add -A > /dev/null 2>&1
git -c user.email=ci@local -c user.name=CI commit -m "ci-tmp" > /dev/null 2>&1
python3 scripts/check_migration_safety.py --allow-medium-risk --diff-against origin/develop
docker rm -f "$PG_CONTAINER" 2>/dev/null || true
echo "PostgreSQL container cleaned up"
- name: Job duration summary
if: always()
shell: sh
run: "set +eu\nif [ -n \"$JOB_START_TIME\" ]; then\n END_TIME=$(date +%s)\n DURATION=$((END_TIME - JOB_START_TIME))\n MINS=$((DURATION / 60))\n SECS=$((DURATION % 60))\n echo \"JOB_DURATION_SECONDS=$DURATION\" >> $GITHUB_ENV\n echo \"=== Job Duration: ${MINS}m${SECS}s ===\"\nelse\n echo \"JOB_DURATION_SECONDS=0\" >> $GITHUB_ENV\n echo \"=== Job Duration: unknown ===\"\nfi\n"
- name: Notify on failure
continue-on-error: true
if: failure()
shell: sh
env:
CI_NOTIFY_WEBHOOK: ${{ secrets.CI_NOTIFY_WEBHOOK }}
run: 'set +e
NOTIFY_MODE=failure JOB_NAME="Validate Code Quality And Tests" python3 scripts/ci_notify.py
'
- name: Report trace end (always)
if: always()
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\nif [ "${{ job.status }}" = "success" ]; then\n TRACE_STATUS="ok"\nelse\n TRACE_STATUS="error"\nfi\n\nTRACE_SERVICE="" TRACE_NAME="" TRACE_STATUS="$TRACE_STATUS" python3 .gitea/scripts/ci_trace_report.py --start-time "$CI_TRACE_START_TIME"\n\nexit 0\n"
unit-tests:
needs: check-frontend-only
if: always() && needs.check-frontend-only.outputs.skip_backend != 'true'
name: Unit Tests
runs-on: ci-l2
timeout-minutes: 8
env:
USE_IN_MEMORY_DB: 'true'
OSS_ACCESS_KEY_ID: placeholder
OSS_ACCESS_KEY_SECRET: placeholder
OSS_BUCKET_NAME: xiaoxia-autocut
OSS_ENDPOINT: oss-cn-hangzhou.aliyuncs.com
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: "set -eu\npython3 - <<'PY'\nimport io, os, tarfile, time, urllib.request, urllib.error\nurl = f\"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz\"\nrequest = urllib.request.Request(url, headers={\"Authorization\": f\"token {os.environ['GITHUB_TOKEN']}\"})\nlast_err = None\nfor attempt in range(5):\n try:\n with urllib.request.urlopen(request, timeout=120) as response:\n archive = response.read()\n break\n except urllib.error.HTTPError as e:\n last_err = e\n if e.code >= 500 and attempt < 4:\n wait = 2 ** attempt\n print(f\"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...\")\n time.sleep(wait)\n continue\n raise\n except Exception as e:\n last_err = e\n if attempt < 4:\n wait = 2 ** attempt\n print(f\"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...\"\
)\n time.sleep(wait)\n continue\n raise\nelse:\n raise last_err\nwith tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:\n root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'\n for member in tar.getmembers():\n name = member.name\n if name == root_prefix[:-1]:\n continue\n if name.startswith(root_prefix):\n member.name = name[len(root_prefix):]\n if member.name:\n tar.extract(member, '.')\nPY\n"
- name: Record start time & report trace start
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\necho "CI_TRACE_START_TIME=$(date +%s)" >> $GITHUB_ENV\nTRACE_SERVICE="" TRACE_NAME="" python3 .gitea/scripts/ci_trace_report.py --status running\n\nexit 0\n"
- name: Record job start time
shell: sh
run: 'set -eu
echo "JOB_START_TIME=$(date +%s)" >> $GITHUB_ENV
echo "Job started at $(date)"
'
- name: Install ffmpeg
shell: sh
run: "set +e\nif command -v ffmpeg > /dev/null 2>&1; then\n echo \"ffmpeg already installed: $(ffmpeg -version | head -1)\"\n exit 0\nfi\nif command -v apt-get > /dev/null 2>&1; then\n apt-get update -qq && apt-get install -y -qq ffmpeg\nelif command -v yum > /dev/null 2>&1; then\n yum install -y -q epel-release 2>/dev/null\n yum install -y -q ffmpeg 2>/dev/null\n if [ $? -ne 0 ] && command -v dnf > /dev/null 2>&1; then\n dnf install -y -q --nogpgcheck https://download1.rpmfusion.org/free/el/rpmfusion-free-release-$(rpm -E %rhel).noarch.rpm 2>/dev/null\n dnf install -y -q ffmpeg 2>/dev/null\n fi\nelif command -v dnf > /dev/null 2>&1; then\n dnf install -y -q ffmpeg 2>/dev/null\nfi\nif command -v ffmpeg > /dev/null 2>&1; then\n echo \"ffmpeg installed successfully: $(ffmpeg -version | head -1)\"\nelse\n echo \"Warning: ffmpeg installation failed or not available, some tests may be skipped\"\nfi\n"
- name: Install dependencies
shell: sh
run: 'set -eu
python3 -m pip install -q -r requirements-base.txt
python3 -m pip install -q -r requirements.txt
python3 -m pip install -q -r requirements-dev.txt
pytest --version
'
- name: Select incremental test files
if: github.event_name == 'pull_request'
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set +e
PR_NUMBER=$(echo "$GITHUB_REF" | sed 's|refs/pull/||; s|/.*||')
API_URL="${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/files?limit=300"
CHANGED_FILES=$(curl -s -H "Authorization: token ${GITHUB_TOKEN}" "$API_URL" | python3 -c "import sys,json; [print(f['filename']) for f in json.load(sys.stdin) if f['status'] != 'removed']")
echo "改动文件数: $(echo "$CHANGED_FILES" | grep -c . || echo 0)"
CHANGED_FILES="$CHANGED_FILES" \
SELECTED_TESTS_OUTPUT=/tmp/selected_tests.txt \
python3 scripts/ci/select_unit_tests.py
SELECT_EXIT=$?
if [ $SELECT_EXIT -eq 0 ]; then
echo "UNIT_TEST_MODE=incremental" >> $GITHUB_ENV
TEST_FILES=$(cat /tmp/selected_tests.txt | tr '\n' ' ')
echo "SELECTED_TEST_FILES=$TEST_FILES" >> $GITHUB_ENV
echo "增量模式: $(cat /tmp/selected_tests.txt | wc -l) 个测试文件"
else
echo "UNIT_TEST_MODE=full" >> $GITHUB_ENV
echo "SELECTED_TEST_FILES=tests/unit" >> $GITHUB_ENV
echo "全量模式"
fi
- name: Run unit tests with coverage
shell: sh
run: |
set -eu
if [ "${UNIT_TEST_MODE:-full}" = "incremental" ]; then
echo "=== 增量测试模式 ==="
PYTHONPATH="$PWD/apps/api:$PWD" python3 -m coverage run \
--source=apps/api/app,packages \
--omit="*/migrations/*,*/tests/*,*/test_*.py,*/site-packages/*" \
--branch \
-m pytest $SELECTED_TEST_FILES -q
python3 -m coverage report --show-missing
python3 -m coverage xml -o coverage.xml
# 增量模式下调低覆盖率门槛(跑的文件少覆盖率自然低,不做强校验)
python3 -m coverage report --fail-under=10 > /dev/null || true
else
PYTHONPATH="$PWD/apps/api:$PWD" python3 -m coverage run \
--source=apps/api/app,packages \
--omit="*/migrations/*,*/tests/*,*/test_*.py,*/site-packages/*" \
--branch \
-m pytest tests/unit -q
python3 -m coverage report --show-missing
python3 -m coverage xml -o coverage.xml
python3 -m coverage report --fail-under=65 > /dev/null
fi
- name: Diff coverage check (增量行覆盖率)
if: github.event_name == 'pull_request' && env.HAS_APP_CHANGES == 'true'
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
PR_NUMBER=$(echo "$GITHUB_REF" | sed 's|refs/pull/||; s|/.*||')
# 获取base分支
BASE_BRANCH="${{ github.base_ref }}"
echo "Base branch: $BASE_BRANCH"
# 初始化git (CI tarball checkout没有.git目录)
# 先备份PR代码,再基于base分支建分支,确保HEAD与base有共同祖先
PR_CODE_DIR="/tmp/pr-code-$$"
mkdir -p "$PR_CODE_DIR"
# 排除隐藏文件(如.env)和后续生成的coverage文件,只备份源码
find . -maxdepth 1 -mindepth 1 ! -name 'coverage.xml' ! -name 'diff_coverage.html' -exec cp -r {} "$PR_CODE_DIR/" \;
rm -rf .git
git init > /dev/null 2>&1
git remote add origin https://git.xiaoxiajianji.com/xiaoxia/xiaoxia-saas.git > /dev/null 2>&1
git config user.email "ci@local"
git config user.name "CI"
# 拉取base分支用于对比
git fetch origin $BASE_BRANCH --depth=200
# 基于base分支创建当前分支,确保有共同祖先
git checkout -b ci-pr-branch "origin/$BASE_BRANCH" > /dev/null 2>&1
# 清除base分支的源码,用PR代码覆盖
find . -mindepth 1 -maxdepth 1 ! -name '.git' -exec rm -rf {} +
cp -r "$PR_CODE_DIR"/. .
rm -rf "$PR_CODE_DIR"
# 提交当前代码
git add -A > /dev/null 2>&1
git commit -m "ci-tmp" > /dev/null 2>&1
# 根据模式设置门槛
if [ "${UNIT_TEST_MODE:-full}" = "incremental" ]; then
# 增量测试模式覆盖不全,门槛设低一些
THRESHOLD=40
echo "增量测试模式,增量覆盖率门槛: ${THRESHOLD}%"
else
THRESHOLD=60
echo "全量测试模式,增量覆盖率门槛: ${THRESHOLD}%"
fi
# 运行diff-cover
set +e
python3 -m diff_cover.diff_cover_tool coverage.xml \
--compare-branch="origin/$BASE_BRANCH" \
--fail-under=$THRESHOLD \
--html-report diff_coverage.html \
2>&1
DIFF_EXIT=$?
set -e
if [ $DIFF_EXIT -ne 0 ]; then
echo ""
echo "❌ 增量覆盖率未达到门槛 (${THRESHOLD}%)"
echo " 请为改动的代码添加单元测试后再提交"
echo ""
echo "=== 覆盖率报告 ==="
python3 -m diff_cover.diff_cover_tool coverage.xml \
--compare-branch="origin/$BASE_BRANCH" 2>&1 | tail -30
exit 1
fi
echo "✅ 增量覆盖率达标"
- name: CI failure notification
if: failure()
shell: sh
env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
CI_WEBHOOK_URL: ${{ secrets.CI_WEBHOOK_URL }}
run: 'set +e
FAILED_JOB="Unit Tests" python3 scripts/ci_notify_failure.py
'
- name: Job duration summary
if: always()
shell: sh
run: "set +eu\nif [ -n \"$JOB_START_TIME\" ]; then\n END_TIME=$(date +%s)\n DURATION=$((END_TIME - JOB_START_TIME))\n MINS=$((DURATION / 60))\n SECS=$((DURATION % 60))\n echo \"JOB_DURATION_SECONDS=$DURATION\" >> $GITHUB_ENV\n echo \"=== Job Duration: ${MINS}m${SECS}s ===\"\nelse\n echo \"JOB_DURATION_SECONDS=0\" >> $GITHUB_ENV\n echo \"=== Job Duration: unknown ===\"\nfi\n"
- name: Notify on failure
continue-on-error: true
if: failure()
shell: sh
env:
CI_NOTIFY_WEBHOOK: ${{ secrets.CI_NOTIFY_WEBHOOK }}
run: 'set +e
NOTIFY_MODE=failure JOB_NAME="Unit Tests" python3 scripts/ci_notify.py
'
- name: Report trace end (always)
if: always()
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\nif [ "${{ job.status }}" = "success" ]; then\n TRACE_STATUS="ok"\nelse\n TRACE_STATUS="error"\nfi\n\nTRACE_SERVICE="" TRACE_NAME="" TRACE_STATUS="$TRACE_STATUS" python3 .gitea/scripts/ci_trace_report.py --start-time "$CI_TRACE_START_TIME"\n\nexit 0\n"
integration-tests:
name: Integration Tests
runs-on: ci-l2
timeout-minutes: 30
if: always() && needs.check-frontend-only.outputs.skip_backend != 'true'
needs:
- check-frontend-only
- validate
env:
DATABASE_URL: postgresql+psycopg://postgres:postgres@127.0.0.1:5432/xiaoxia_saas
USE_IN_MEMORY_DB: 'false'
OSS_ACCESS_KEY_ID: placeholder
OSS_ACCESS_KEY_SECRET: placeholder
OSS_BUCKET_NAME: xiaoxia-autocut
OSS_ENDPOINT: oss-cn-hangzhou.aliyuncs.com
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: "set -eu\npython3 - <<'PY'\nimport io, os, tarfile, time, urllib.request, urllib.error\nurl = f\"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz\"\nrequest = urllib.request.Request(url, headers={\"Authorization\": f\"token {os.environ['GITHUB_TOKEN']}\"})\nlast_err = None\nfor attempt in range(5):\n try:\n with urllib.request.urlopen(request, timeout=120) as response:\n archive = response.read()\n break\n except urllib.error.HTTPError as e:\n last_err = e\n if e.code >= 500 and attempt < 4:\n wait = 2 ** attempt\n print(f\"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...\")\n time.sleep(wait)\n continue\n raise\n except Exception as e:\n last_err = e\n if attempt < 4:\n wait = 2 ** attempt\n print(f\"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...\"\
)\n time.sleep(wait)\n continue\n raise\nelse:\n raise last_err\nwith tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:\n root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'\n for member in tar.getmembers():\n name = member.name\n if name == root_prefix[:-1]:\n continue\n if name.startswith(root_prefix):\n member.name = name[len(root_prefix):]\n if member.name:\n tar.extract(member, '.')\nPY\n"
- name: Record start time & report trace start
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\necho "CI_TRACE_START_TIME=$(date +%s)" >> $GITHUB_ENV\nTRACE_SERVICE="" TRACE_NAME="" python3 .gitea/scripts/ci_trace_report.py --status running\n\nexit 0\n"
- name: Record job start time
shell: sh
run: 'set -eu
echo "JOB_START_TIME=$(date +%s)" >> $GITHUB_ENV
echo "Job started at $(date)"
'
- name: Verify CI environment
shell: sh
run: 'set -eu
python3 --version
python3 -m pip --version
echo "CI environment is ready"
'
- name: Install dependencies
shell: sh
run: 'set -eu
python3 -m pip install -q -r requirements-base.txt
python3 -m pip install -q -r requirements.txt
python3 -m pip install -q -r requirements-dev.txt
pytest --version
'
- name: Install ffmpeg
shell: sh
run: "set +e\nif command -v ffmpeg > /dev/null 2>&1; then\n echo \"ffmpeg already installed: $(ffmpeg -version | head -1)\"\n exit 0\nfi\nif command -v apt-get > /dev/null 2>&1; then\n apt-get update -qq && apt-get install -y -qq ffmpeg\nelif command -v yum > /dev/null 2>&1; then\n yum install -y -q epel-release 2>/dev/null\n yum install -y -q ffmpeg 2>/dev/null\n if [ $? -ne 0 ] && command -v dnf > /dev/null 2>&1; then\n dnf install -y -q --nogpgcheck https://download1.rpmfusion.org/free/el/rpmfusion-free-release-$(rpm -E %rhel).noarch.rpm 2>/dev/null\n dnf install -y -q ffmpeg 2>/dev/null\n fi\nelif command -v dnf > /dev/null 2>&1; then\n dnf install -y -q ffmpeg 2>/dev/null\nfi\nif command -v ffmpeg > /dev/null 2>&1; then\n echo \"ffmpeg installed successfully: $(ffmpeg -version | head -1)\"\nelse\n echo \"Warning: ffmpeg installation failed or not available, some tests may be skipped\"\nfi\n"
- name: Start Redis
shell: sh
run: "set -eu\nREDIS_CONTAINER=\"ci-redis-${GITHUB_RUN_ID:-$$}\"\necho \"REDIS_CONTAINER=$REDIS_CONTAINER\" >> \"$GITHUB_ENV\"\ndocker rm -f \"$REDIS_CONTAINER\" 2>/dev/null || true\ndocker run -d --name \"$REDIS_CONTAINER\" \\\n -P \\\n --health-cmd \"redis-cli ping\" \\\n --health-interval 2s \\\n --health-timeout 2s \\\n --health-retries 10 \\\n redis:7-alpine\nREDIS_PORT=$(docker port \"$REDIS_CONTAINER\" 6379/tcp | cut -d: -f2)\necho \"Redis port: $REDIS_PORT\"\necho \"REDIS_URL=redis://127.0.0.1:$REDIS_PORT/0\" >> \"$GITHUB_ENV\"\nfor i in $(seq 1 15); do\n if docker inspect --format='{{.State.Health.Status}}' \"$REDIS_CONTAINER\" 2>/dev/null | grep -q healthy; then\n echo \"Redis is ready on port $REDIS_PORT\"\n break\n fi\n echo \"Waiting for Redis... ($i/15)\"\n sleep 2\ndone\ndocker inspect --format='{{.State.Health.Status}}' \"$REDIS_CONTAINER\" | grep -q healthy\n"
- name: Start PostgreSQL for integration tests
shell: sh
run: "set -eu\nPG_CONTAINER=\"ci-pg-${GITHUB_RUN_ID:-$$}\"\necho \"PG_CONTAINER=$PG_CONTAINER\" >> \"$GITHUB_ENV\"\ndocker rm -f \"$PG_CONTAINER\" 2>/dev/null || true\ndocker run -d --name \"$PG_CONTAINER\" \\\n --shm-size=256m \\\n -e POSTGRES_USER=postgres \\\n -e POSTGRES_PASSWORD=postgres \\\n -e POSTGRES_DB=xiaoxia_saas \\\n -P \\\n --health-cmd \"pg_isready -U postgres\" \\\n --health-interval 5s \\\n --health-timeout 5s \\\n --health-retries 12 \\\n postgres:16\n# 获取随机映射的端口\nPG_PORT=$(docker port \"$PG_CONTAINER\" 5432/tcp | cut -d: -f2)\necho \"PostgreSQL port: $PG_PORT\"\necho \"DATABASE_URL=postgresql+psycopg://postgres:postgres@127.0.0.1:$PG_PORT/xiaoxia_saas\" >> \"$GITHUB_ENV\"\nfor i in $(seq 1 30); do\n if docker inspect --format='{{.State.Health.Status}}' \"$PG_CONTAINER\" 2>/dev/null | grep -q healthy; then\n echo \"PostgreSQL is ready on port $PG_PORT\"\n break\n fi\n echo \"Waiting for PostgreSQL... ($i/30)\"\n sleep 2\ndone\ndocker inspect --format='{{.State.Health.Status}}'\
\ \"$PG_CONTAINER\" | grep -q healthy\n"
- name: Apply migrations for integration tests
shell: sh
run: 'set -eu
PYTHONPATH="$PWD/apps/api:$PWD" python3 -m alembic upgrade head
'
- name: Run integration tests
shell: sh
run: "set -eu\npython3 -m pip install -q pytest-rerunfailures\nPYTHONPATH=\"$PWD/apps/api:$PWD\" python3 -m coverage run --append \\\n --source=apps/api/app,packages \\\n --omit=\"*/migrations/*,*/tests/*,*/test_*.py,*/site-packages/*\" \\\n --branch \\\n -m pytest tests/integration -q --timeout=60 -x --reruns 2 --reruns-delay 1 -m \"not performance\"\npython3 -m coverage report --show-missing\npython3 -m coverage xml -o coverage.xml\npython3 -m coverage report --fail-under=40 > /dev/null # 集成测试覆盖率门槛较低,核心目标是功能验证\n"
- name: Run API performance baseline tests
shell: sh
continue-on-error: true
run: "set +e\necho \"=== API 性能基线测试 ===\"\nPERF_OUTPUT=$(mktemp)\nPYTHONPATH=\"$PWD/apps/api:$PWD\" python3 -m pytest tests/integration/test_api_performance.py \\\n -v --timeout=120 -p no:cacheprovider 2>&1 | tee \"$PERF_OUTPUT\"\nPERF_EXIT=$?\n\n# 提取性能统计\necho \"\"\necho \"=== 性能测试摘要 ===\"\ngrep \"PERF_STATS:\" \"$PERF_OUTPUT\" || echo \"PERF_STATS: 未找到统计数据\"\ngrep \"PERF_RESULT:\" \"$PERF_OUTPUT\" || echo \"PERF_RESULT: 未找到详细结果\"\n\n# 统计通过率\nTOTAL=$(grep -c \"PERF_RESULT:\" \"$PERF_OUTPUT\" || echo 0)\nPASSED=$(grep \"PERF_RESULT: PASS\" \"$PERF_OUTPUT\" | wc -l)\nFAILED=$(grep \"PERF_RESULT: FAIL\" \"$PERF_OUTPUT\" | wc -l)\n\necho \"\"\necho \"性能测试结果: $PASSED/$TOTAL 通过, $FAILED 未达标\"\n\nif [ \"$FAILED\" -gt 0 ]; then\n echo \"\"\n echo \"⚠️ 警告: $FAILED 个接口性能未达标,请关注以下接口:\"\n grep \"PERF_RESULT: FAIL\" \"$PERF_OUTPUT\" | while read line; do\n echo \" $line\"\n done\n echo \"\"\n echo \"性能测试失败不阻塞主流水线,但建议尽快优化。\"\nelse\n echo \"✅ 所有接口性能达标!\"\nfi\n\nrm -f \"$PERF_OUTPUT\"\
\n# 始终返回 0,不阻塞流水线\nexit 0\n"
- name: Cleanup PostgreSQL & Redis
if: always()
shell: sh
run: 'docker rm -f "${PG_CONTAINER:-ci-pg-validate}" 2>/dev/null || true
docker rm -f "${REDIS_CONTAINER:-ci-redis-int}" 2>/dev/null || true
echo "PostgreSQL container cleaned up"
echo "Redis container cleaned up"
'
- name: Coverage summary
if: always()
shell: sh
env:
COVERAGE_THRESHOLD: '40'
run: 'set +e
echo "=== 覆盖率汇总 ==="
python3 scripts/ci_coverage_summary.py
'
- name: Job duration summary
if: always()
shell: sh
run: "set +eu\nif [ -n \"$JOB_START_TIME\" ]; then\n END_TIME=$(date +%s)\n DURATION=$((END_TIME - JOB_START_TIME))\n MINS=$((DURATION / 60))\n SECS=$((DURATION % 60))\n echo \"JOB_DURATION_SECONDS=$DURATION\" >> $GITHUB_ENV\n echo \"=== Job Duration: ${MINS}m${SECS}s ===\"\nelse\n echo \"JOB_DURATION_SECONDS=0\" >> $GITHUB_ENV\n echo \"=== Job Duration: unknown ===\"\nfi\n"
- name: Notify on failure
continue-on-error: true
if: failure()
shell: sh
env:
CI_NOTIFY_WEBHOOK: ${{ secrets.CI_NOTIFY_WEBHOOK }}
run: 'set +e
NOTIFY_MODE=failure JOB_NAME="Integration Tests" python3 scripts/ci_notify.py
'
- name: Report trace end (always)
if: always()
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\nif [ "${{ job.status }}" = "success" ]; then\n TRACE_STATUS="ok"\nelse\n TRACE_STATUS="error"\nfi\n\nTRACE_SERVICE="" TRACE_NAME="" TRACE_STATUS="$TRACE_STATUS" python3 .gitea/scripts/ci_trace_report.py --start-time "$CI_TRACE_START_TIME"\n\nexit 0\n"
frontend-lint:
name: Frontend Lint
runs-on: ci-check
timeout-minutes: 10
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: "set -eu\npython3 - <<'PY'\nimport io, os, tarfile, time, urllib.request, urllib.error\nurl = f\"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz\"\nrequest = urllib.request.Request(url, headers={\"Authorization\": f\"token {os.environ['GITHUB_TOKEN']}\"})\nlast_err = None\nfor attempt in range(5):\n try:\n with urllib.request.urlopen(request, timeout=120) as response:\n archive = response.read()\n break\n except urllib.error.HTTPError as e:\n last_err = e\n if e.code >= 500 and attempt < 4:\n wait = 2 ** attempt\n print(f\"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...\")\n time.sleep(wait)\n continue\n raise\n except Exception as e:\n last_err = e\n if attempt < 4:\n wait = 2 ** attempt\n print(f\"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...\"\
)\n time.sleep(wait)\n continue\n raise\nelse:\n raise last_err\nwith tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:\n root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'\n for member in tar.getmembers():\n name = member.name\n if name == root_prefix[:-1]:\n continue\n if name.startswith(root_prefix):\n member.name = name[len(root_prefix):]\n if member.name:\n tar.extract(member, '.')\nPY\n"
- name: Record start time & report trace start
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\necho "CI_TRACE_START_TIME=$(date +%s)" >> $GITHUB_ENV\nTRACE_SERVICE="" TRACE_NAME="" python3 .gitea/scripts/ci_trace_report.py --status running\n\nexit 0\n"
- name: Record job start time
shell: sh
run: 'set -eu
echo "JOB_START_TIME=$(date +%s)" >> $GITHUB_ENV
echo "Job started at $(date)"
'
- name: Install dependencies
shell: sh
run: "set -eu\nNPM_CACHE_VOLUME=\"xiaoxia-npm-cache\"\nif ! docker volume inspect \"$NPM_CACHE_VOLUME\" >/dev/null 2>&1; then\n docker volume create \"$NPM_CACHE_VOLUME\" >/dev/null\n echo \"Created npm cache volume: $NPM_CACHE_VOLUME\"\nfi\n\ndocker run --rm \\\n -v \"$PWD:/workspace\" \\\n -v \"$NPM_CACHE_VOLUME:/workspace/apps/web/node_modules\" -w /workspace/apps/web \\\n docker.m.daocloud.io/library/node:20 \\\n sh -lc 'PACKAGE_LOCK_HASH=$(md5sum package-lock.json 2>/dev/null | cut -d\" \" -f1)\nCACHE_HASH_FILE=\"node_modules/.package-lock-hash\"\nCACHE_VALID=false\nif [ -f \"$CACHE_HASH_FILE\" ] && [ \"$(cat \"$CACHE_HASH_FILE\")\" = \"$PACKAGE_LOCK_HASH\" ] && [ -x \"node_modules/.bin/eslint\" ] && [ -x \"node_modules/.bin/tsc\" ] && [ -x \"node_modules/.bin/prettier\" ] && [ -x \"node_modules/.bin/vitest\" ]; then\n CACHE_VALID=true\n echo \"Cache hit: dependencies valid, skipping npm ci\"\nfi\nif [ \"$CACHE_VALID\" = \"false\" ]; then\n echo \"Cache miss or invalid: running npm ci...\"\n if ! npm ci --include=dev; then\n echo \"npm ci failed, cleaning node_modules and retrying...\"\n rm -rf node_modules\n mkdir -p node_modules\n npm ci --include=dev\n fi\n # Post-install integrity check: verify all critical tools exist\n if [ ! -x \"node_modules/.bin/eslint\" ] || [ ! -x \"node_modules/.bin/tsc\" ] || [ ! -x \"node_modules/.bin/prettier\" ] || [ ! -x \"node_modules/.bin/vitest\" ]; then\n echo \"Post-install check failed: critical binaries missing, cleaning and retrying...\"\n rm -rf node_modules\n mkdir -p node_modules\n npm ci --include=dev\n fi\n echo \"$PACKAGE_LOCK_HASH\" > \"$CACHE_HASH_FILE\"\n echo \"Dependencies installed, cache updated\"\nfi'\n"
- name: Run ESLint
shell: sh
run: "set -eu\nNPM_CACHE_VOLUME=\"xiaoxia-npm-cache\"\nif ! docker volume inspect \"$NPM_CACHE_VOLUME\" >/dev/null 2>&1; then\n docker volume create \"$NPM_CACHE_VOLUME\" >/dev/null\n echo \"Created npm cache volume: $NPM_CACHE_VOLUME\"\nfi\n\ndocker run --rm \\\n -v \"$PWD:/workspace\" \\\n -v \"$NPM_CACHE_VOLUME:/workspace/apps/web/node_modules\" -w /workspace/apps/web \\\n docker.m.daocloud.io/library/node:20 \\\n sh -lc 'npx --no-install eslint src --ext .ts,.tsx --max-warnings 0'\n"
- name: Run TypeScript type check
shell: sh
run: "set -eu\nNPM_CACHE_VOLUME=\"xiaoxia-npm-cache\"\nif ! docker volume inspect \"$NPM_CACHE_VOLUME\" >/dev/null 2>&1; then\n docker volume create \"$NPM_CACHE_VOLUME\" >/dev/null\n echo \"Created npm cache volume: $NPM_CACHE_VOLUME\"\nfi\n\ndocker run --rm \\\n -v \"$PWD:/workspace\" \\\n -v \"$NPM_CACHE_VOLUME:/workspace/apps/web/node_modules\" -w /workspace/apps/web \\\n docker.m.daocloud.io/library/node:20 \\\n sh -lc 'npx --no-install tsc --noEmit'\n"
- name: Run Prettier check
shell: sh
run: "set -eu\nNPM_CACHE_VOLUME=\"xiaoxia-npm-cache\"\nif ! docker volume inspect \"$NPM_CACHE_VOLUME\" >/dev/null 2>&1; then\n docker volume create \"$NPM_CACHE_VOLUME\" >/dev/null\n echo \"Created npm cache volume: $NPM_CACHE_VOLUME\"\nfi\n\ndocker run --rm \\\n -v \"$PWD:/workspace\" \\\n -v \"$NPM_CACHE_VOLUME:/workspace/apps/web/node_modules\" -w /workspace/apps/web \\\n docker.m.daocloud.io/library/node:20 \\\n sh -lc 'npx --no-install prettier --check \"src/**/*.{ts,tsx,md}\"'\n"
- name: Run Vitest tests
shell: sh
run: "set -eu\nNPM_CACHE_VOLUME=\"xiaoxia-npm-cache\"\nif ! docker volume inspect \"$NPM_CACHE_VOLUME\" >/dev/null 2>&1; then\n docker volume create \"$NPM_CACHE_VOLUME\" >/dev/null\n echo \"Created npm cache volume: $NPM_CACHE_VOLUME\"\nfi\n\ndocker run --rm \\\n -v \"$PWD:/workspace\" \\\n -v \"$NPM_CACHE_VOLUME:/workspace/apps/web/node_modules\" -w /workspace/apps/web \\\n docker.m.daocloud.io/library/node:20 \\\n sh -lc 'npx --no-install vitest run src/test'\n"
- name: Job duration summary
if: always()
shell: sh
run: "set +eu\nif [ -n \"$JOB_START_TIME\" ]; then\n END_TIME=$(date +%s)\n DURATION=$((END_TIME - JOB_START_TIME))\n MINS=$((DURATION / 60))\n SECS=$((DURATION % 60))\n echo \"JOB_DURATION_SECONDS=$DURATION\" >> $GITHUB_ENV\n echo \"=== Job Duration: ${MINS}m${SECS}s ===\"\nelse\n echo \"JOB_DURATION_SECONDS=0\" >> $GITHUB_ENV\n echo \"=== Job Duration: unknown ===\"\nfi\n"
- name: Notify on failure
continue-on-error: true
if: failure()
shell: sh
env:
CI_NOTIFY_WEBHOOK: ${{ secrets.CI_NOTIFY_WEBHOOK }}
run: 'set +e
NOTIFY_MODE=failure JOB_NAME="Frontend Lint" python3 scripts/ci_notify.py
'
- name: Report trace end (always)
if: always()
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\nif [ "${{ job.status }}" = "success" ]; then\n TRACE_STATUS="ok"\nelse\n TRACE_STATUS="error"\nfi\n\nTRACE_SERVICE="" TRACE_NAME="" TRACE_STATUS="$TRACE_STATUS" python3 .gitea/scripts/ci_trace_report.py --start-time "$CI_TRACE_START_TIME"\n\nexit 0\n"
frontend-unit-test:
name: Frontend Unit Tests
runs-on: ci-check
timeout-minutes: 15
needs: check-frontend-only
if: always() && needs.check-frontend-only.outputs.skip_frontend != 'true'
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: "set -eu\npython3 - <<'PY'\nimport io, os, tarfile, time, urllib.request, urllib.error\nurl = f\"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz\"\nrequest = urllib.request.Request(url, headers={\"Authorization\": f\"token {os.environ['GITHUB_TOKEN']}\"})\nlast_err = None\nfor attempt in range(5):\n try:\n with urllib.request.urlopen(request, timeout=120) as response:\n archive = response.read()\n break\n except urllib.error.HTTPError as e:\n last_err = e\n if e.code >= 500 and attempt < 4:\n wait = 2 ** attempt\n print(f\"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...\")\n time.sleep(wait)\n continue\n raise\n except Exception as e:\n last_err = e\n if attempt < 4:\n wait = 2 ** attempt\n print(f\"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...\")\n time.sleep(wait)\n continue\n raise\nelse:\n raise last_err\nwith tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:\n root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'\n for member in tar.getmembers():\n name = member.name\n if name == root_prefix[:-1]:\n continue\n if name.startswith(root_prefix):\n member.name = name[len(root_prefix):]\n if member.name:\n tar.extract(member, '.')\nPY\n"
- name: Record start time & report trace start
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\necho "CI_TRACE_START_TIME=$(date +%s)" >> $GITHUB_ENV\nTRACE_SERVICE="" TRACE_NAME="" python3 .gitea/scripts/ci_trace_report.py --status running\n\nexit 0\n"
- name: Record job start time
shell: sh
run: 'set -eu
echo "JOB_START_TIME=$(date +%s)" >> $GITHUB_ENV
echo "Job started at $(date)"
'
- name: Install dependencies
shell: sh
run: "set -eu\nNPM_CACHE_VOLUME=\"xiaoxia-npm-cache\"\nif ! docker volume inspect \"$NPM_CACHE_VOLUME\" >/dev/null 2>&1; then\n docker volume create \"$NPM_CACHE_VOLUME\" >/dev/null\n echo \"Created npm cache volume: $NPM_CACHE_VOLUME\"\nfi\n\ndocker run --rm \\\n -v \"$PWD:/workspace\" \\\n -v \"$NPM_CACHE_VOLUME:/workspace/apps/web/node_modules\" -w /workspace/apps/web \\\n docker.m.daocloud.io/library/node:20 \\\n sh -lc 'PACKAGE_LOCK_HASH=$(md5sum package-lock.json 2>/dev/null | cut -d\" \" -f1)\nCACHE_HASH_FILE=\"node_modules/.package-lock-hash\"\nCACHE_VALID=false\nif [ -f \"$CACHE_HASH_FILE\" ] && [ \"$(cat \"$CACHE_HASH_FILE\")\" = \"$PACKAGE_LOCK_HASH\" ] && [ -x \"node_modules/.bin/vitest\" ]; then\n CACHE_VALID=true\n echo \"Cache hit: dependencies valid, skipping npm ci\"\nfi\nif [ \"$CACHE_VALID\" = \"false\" ]; then\n echo \"Cache miss or invalid: running npm ci...\"\n if ! npm ci --include=dev; then\n echo \"npm ci failed, cleaning node_modules and retrying...\"\n rm -rf node_modules\n mkdir -p node_modules\n npm ci --include=dev\n fi\n # Post-install integrity check\n if [ ! -x \"node_modules/.bin/vitest\" ]; then\n echo \"Post-install check failed: vitest missing, cleaning and retrying...\"\n rm -rf node_modules\n mkdir -p node_modules\n npm ci --include=dev\n fi\n echo \"$PACKAGE_LOCK_HASH\" > \"$CACHE_HASH_FILE\"\n echo \"Dependencies installed, cache updated\"\nfi'\n"
- name: Run Vitest with coverage
shell: sh
run: "set -eu\nNPM_CACHE_VOLUME=\"xiaoxia-npm-cache\"\nif ! docker volume inspect \"$NPM_CACHE_VOLUME\" >/dev/null 2>&1; then\n docker volume create \"$NPM_CACHE_VOLUME\" >/dev/null\n echo \"Created npm cache volume: $NPM_CACHE_VOLUME\"\nfi\n\ndocker run --rm \\\n -v \"$PWD:/workspace\" \\\n -v \"$NPM_CACHE_VOLUME:/workspace/apps/web/node_modules\" -w /workspace/apps/web \\\n docker.m.daocloud.io/library/node:20 \\\n sh -lc 'npx --no-install vitest run --coverage'\n"
- name: Job duration summary
if: always()
shell: sh
run: "set +eu\nif [ -n \"$JOB_START_TIME\" ]; then\n END_TIME=$(date +%s)\n DURATION=$((END_TIME - JOB_START_TIME))\n MINS=$((DURATION / 60))\n SECS=$((DURATION % 60))\n echo \"JOB_DURATION_SECONDS=$DURATION\" >> $GITHUB_ENV\n echo \"=== Job Duration: ${MINS}m${SECS}s ===\"\nelse\n echo \"JOB_DURATION_SECONDS=0\" >> $GITHUB_ENV\n echo \"=== Job Duration: unknown ===\"\nfi\n"
- name: Notify on failure
continue-on-error: true
if: failure()
shell: sh
env:
CI_NOTIFY_WEBHOOK: ${{ secrets.CI_NOTIFY_WEBHOOK }}
run: 'set +e
NOTIFY_MODE=failure JOB_NAME="Frontend Unit Tests" python3 scripts/ci_notify.py
' - name: Report trace end (always)
if: always()
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\nif [ "${{ job.status }}" = "success" ]; then\n TRACE_STATUS="ok"\nelse\n TRACE_STATUS="error"\nfi\n\nTRACE_SERVICE="" TRACE_NAME="" TRACE_STATUS="$TRACE_STATUS" python3 .gitea/scripts/ci_trace_report.py --start-time "$CI_TRACE_START_TIME"\n\nexit 0\n"
+78
View File
@@ -0,0 +1,78 @@
name: CI Failure Monitor
on:
schedule:
- cron: '0 */6 * * *' # 每6小时检查一次
workflow_dispatch:
inputs:
days:
description: '统计最近N天的失败'
required: false
default: '7'
fail_threshold:
description: '失败次数阈值'
required: false
default: '3'
fail_rate_threshold:
description: '失败率阈值(%)'
required: false
default: '30'
permissions:
contents: read
jobs:
monitor:
name: CI重复失败检测
runs-on: ci-l2
timeout-minutes: 10
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sH "Authorization: token $GITHUB_TOKEN" \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" \
| bash
- name: Record job start time
shell: sh
run: bash scripts/ci/step_timer_start.sh
- name: Run failure detection
shell: sh
env:
GITEA_API_TOKEN: ${{ secrets.REVIEW_GITEA_TOKEN }}
GITEA_URL: https://git.xiaoxiajianji.com
GITEA_REPO: xiaoxia/xiaoxia-saas
CI_NOTIFY_WEBHOOK: ${{ secrets.CI_NOTIFY_WEBHOOK }}
FAIL_CHECK_DAYS: ${{ inputs.days || 7 }}
FAIL_THRESHOLD: ${{ inputs.fail_threshold || 3 }}
FAIL_RATE_THRESHOLD: ${{ inputs.fail_rate_threshold || 30 }}
run: |
set +e
python3 scripts/ci/ci_repeated_failure_detector.py
EXIT_CODE=$?
echo "检测完成,退出码: $EXIT_CODE"
# 0=无异常, 1=有警告, 2=有严重问题
# 监控脚本永远不fail,避免告警风暴
exit 0
- name: Job duration summary
if: always()
shell: sh
run: bash scripts/ci/step_timer_end.sh
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
+103
View File
@@ -0,0 +1,103 @@
name: CI Health Daily Report
on:
schedule:
- cron: '0 1 * * *' # UTC 01:00 = 北京时间 09:00
workflow_dispatch:
permissions:
contents: read
jobs:
ci-health-report:
name: CI健康度每日巡检
runs-on: ci-l2
timeout-minutes: 15
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
python3 - <<'PY'
import io, os, tarfile, time, urllib.request, urllib.error
url = f"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz"
request = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['GITHUB_TOKEN']}"})
last_err = None
for attempt in range(5):
try:
with urllib.request.urlopen(request, timeout=120) as response:
archive = response.read()
break
except urllib.error.HTTPError as e:
last_err = e
if e.code >= 500 and attempt < 4:
wait = 2 ** attempt
print(f"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
except Exception as e:
last_err = e
if attempt < 4:
wait = 2 ** attempt
print(f"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
else:
raise last_err
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
PY
- name: Generate CI Dashboard HTML
shell: sh
env:
GITEA_TOKEN: ${{ github.token }}
run: |
set +e
echo "=== 生成 CI 健康度 HTML 看板 ==="
echo "时间: $(date '+%Y-%m-%d %H:%M:%S')"
echo ""
python3 scripts/ci/ci_dashboard.py --days 7 --html --html-output ci_dashboard.html
EXIT_CODE=$?
if [ $EXIT_CODE -eq 0 ] && [ -f ci_dashboard.html ]; then
HTML_SIZE=$(wc -c < ci_dashboard.html)
echo ""
echo "✅ HTML 看板生成成功 (${HTML_SIZE} bytes)"
echo "路径: $(pwd)/ci_dashboard.html"
# 输出文件内容前几行,方便在 Actions 日志中确认
echo ""
echo "--- 看板预览 (前 5 行) ---"
head -5 ci_dashboard.html
echo "...(完整内容见产物文件)"
else
echo "❌ HTML 看板生成失败 (exit code: $EXIT_CODE)"
fi
echo ""
# 永远成功,看板生成失败不影响主流程
exit 0
- name: Run CI health check and report
shell: sh
env:
GITEA_TOKEN: ${{ github.token }}
CI_NOTIFY_WEBHOOK: ${{ secrets.CI_NOTIFY_WEBHOOK }}
run: |
set +e
echo "=== CI健康度每日巡检 ==="
echo "时间: $(date '+%Y-%m-%d %H:%M:%S')"
echo ""
python3 scripts/ci/ci_health_report.py --limit 30
EXIT_CODE=$?
echo ""
echo "巡检完成 (exit code: $EXIT_CODE)"
# 永远成功,不影响CI状态(通知失败不应该标红)
exit 0
+1821
View File
File diff suppressed because it is too large Load Diff
+14 -15
View File
@@ -16,20 +16,16 @@ permissions:
jobs:
monitor:
name: Monitor CI Trigger Reliability
runs-on: ubuntu-latest
runs-on: ci-l2
timeout-minutes: 5
steps:
- name: Checkout code
uses: actions/checkout@v3
- name: Record start time & report trace start
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\necho "CI_TRACE_START_TIME=$(date +%s)" >> $GITHUB_ENV\nTRACE_SERVICE="ci-trigger-monitor" TRACE_NAME="CI触发监控 / Monitor CI Trigger Reliability" python3 .gitea/scripts/ci_trace_report.py --status running\n\nexit 0\n"
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sH "Authorization: token $GITHUB_TOKEN" "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" | bash
- name: Check CI trigger status for all open PRs
env:
GITEA_API_TOKEN: ${{ secrets.REVIEW_GITEA_TOKEN }}
@@ -42,12 +38,15 @@ jobs:
python3 scripts/ci_trigger_monitor.py
# 监控脚本永远不fail,避免告警风暴
exit 0
- name: Report trace end (always)
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\nif [ "${{ job.status }}" = "success" ]; then\n TRACE_STATUS="ok"\nelse\n TRACE_STATUS="error"\nfi\n\nTRACE_SERVICE="ci-trigger-monitor" TRACE_NAME="CI触发监控 / Monitor CI Trigger Reliability" TRACE_STATUS="$TRACE_STATUS" python3 .gitea/scripts/ci_trace_report.py --start-time "$CI_TRACE_START_TIME"\n\nexit 0\n"
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
+29 -20
View File
@@ -15,26 +15,29 @@ concurrency:
jobs:
code-review:
name: AI Code Review
runs-on: ubuntu-latest
runs-on: ci-l2
# 跳过草稿 PR
if: ${{ !gitea.event.pull_request.draft }}
steps:
- name: Checkout
uses: actions/checkout@v3
with:
fetch-depth: 0
- name: Record start time & report trace start
# actions/checkout 由 runner 在宿主机层面处理,不受容器网络影响
- name: Checkout code
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\necho "CI_TRACE_START_TIME=$(date +%s)" >> $GITHUB_ENV\nTRACE_SERVICE="code-review" TRACE_NAME="AI代码审查 / AI Code Review" python3 .gitea/scripts/ci_trace_report.py --status running\n\nexit 0\n"
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sH "Authorization: token $GITHUB_TOKEN" "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" | bash
- name: Install dependencies
run: |
# 确保 python3-pip 可用(兼容不同基础镜像)
if ! python3 -m pip --version >/dev/null 2>&1; then
apt-get update -qq && apt-get install -y -qq python3-pip python3-venv >/dev/null 2>&1
fi
# 部分镜像 ensurepip 方式兜底
if ! python3 -m pip --version >/dev/null 2>&1; then
python3 -m ensurepip --upgrade 2>/dev/null || curl -sS https://bootstrap.pypa.io/get-pip.py | python3
fi
python3 -m pip install --upgrade pip
python3 -m pip install requests
@@ -45,6 +48,7 @@ jobs:
GITEA_TOKEN: ${{ secrets.REVIEW_GITEA_TOKEN }}
REPO_NAME: ${{ gitea.repository }}
PR_NUMBER: ${{ gitea.event.pull_request.number }}
PR_HEAD_SHA: ${{ gitea.event.pull_request.head.sha }}
# LLM 提供商: coze (扣子原生Bot) / openai (OpenAI兼容)
LLM_PROVIDER: "coze"
# 扣子模式配置(默认国内站 api.coze.cn
@@ -57,14 +61,19 @@ jobs:
LLM_TIMEOUT: "120"
run: |
python3 scripts/ci_code_review.py
# 审查脚本异常不影响 CI 通过
continue-on-error: true
- name: Report trace end (always)
# 注意:脚本退出码决定job状态
# - 有阻塞级问题 → exit 1 → job失败 → 门禁拦截
# - 无阻塞级问题/LLM异常 → exit 0 → 通过(fail-open
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\nif [ "${{ job.status }}" = "success" ]; then\n TRACE_STATUS="ok"\nelse\n TRACE_STATUS="error"\nfi\n\nTRACE_SERVICE="code-review" TRACE_NAME="AI代码审查 / AI Code Review" TRACE_STATUS="$TRACE_STATUS" python3 .gitea/scripts/ci_trace_report.py --start-time "$CI_TRACE_START_TIME"\n\nexit 0\n"
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
+102 -221
View File
@@ -1,4 +1,5 @@
name: Daily Health Check
# 注意:使用 curl step_checkout.sh 方式以兼容 docker runner
on:
schedule:
@@ -12,7 +13,7 @@ jobs:
# ── 1. 生产环境冒烟测试 ─────────────────────────────────────────────
production-smoke:
name: Production Smoke Test
runs-on: saas
runs-on: ci-l2
timeout-minutes: 8
outputs:
report: ${{ steps.smoke.outputs.report }}
@@ -23,58 +24,12 @@ jobs:
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
python3 - <<'PY'
import io, os, tarfile, time, urllib.request, urllib.error
url = f"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz"
request = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['GITHUB_TOKEN']}"})
last_err = None
for attempt in range(5):
try:
with urllib.request.urlopen(request, timeout=120) as response:
archive = response.read()
break
except urllib.error.HTTPError as e:
last_err = e
if e.code >= 500 and attempt < 4:
wait = 2 ** attempt
print(f"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
except Exception as e:
last_err = e
if attempt < 4:
wait = 2 ** attempt
print(f"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
else:
raise last_err
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
PY
- name: Record start time & report trace start
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\necho "CI_TRACE_START_TIME=$(date +%s)" >> $GITHUB_ENV\nTRACE_SERVICE="daily-check" TRACE_NAME="每日健康检查 / Production Smoke Test" python3 .gitea/scripts/ci_trace_report.py --status running\n\nexit 0\n"
curl -sH "Authorization: token $GITHUB_TOKEN" \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" \
| bash
- name: Production health check & smoke test
id: smoke
shell: sh
shell: bash
env:
SMOKE_ENV: production
EXISTING_TOKEN: ${{ secrets.PROD_E2E_TOKEN }}
@@ -114,19 +69,22 @@ jobs:
echo "======================================"
exit $SMOKE_EXIT
# ── 2. Staging API 集成测试 ─────────────────────────────────────────
- name: Report trace end (always)
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\nif [ "${{ job.status }}" = "success" ]; then\n TRACE_STATUS="ok"\nelse\n TRACE_STATUS="error"\nfi\n\nTRACE_SERVICE="daily-check" TRACE_NAME="每日健康检查 / Production Smoke Test" TRACE_STATUS="$TRACE_STATUS" python3 .gitea/scripts/ci_trace_report.py --start-time "$CI_TRACE_START_TIME"\n\nexit 0\n"
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
# ── 2. Staging API 集成测试 ─────────────────────────────────────────
staging-api-tests:
name: Staging API Integration Tests
runs-on: saas
runs-on: ci-l2
timeout-minutes: 10
outputs:
report: ${{ steps.smoke.outputs.report }}
@@ -137,76 +95,36 @@ jobs:
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
python3 - <<'PY'
import io, os, tarfile, time, urllib.request, urllib.error
url = f"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz"
request = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['GITHUB_TOKEN']}"})
last_err = None
for attempt in range(5):
try:
with urllib.request.urlopen(request, timeout=120) as response:
archive = response.read()
break
except urllib.error.HTTPError as e:
last_err = e
if e.code >= 500 and attempt < 4:
wait = 2 ** attempt
print(f"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
except Exception as e:
last_err = e
if attempt < 4:
wait = 2 ** attempt
print(f"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
else:
raise last_err
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
PY
- name: Record start time & report trace start
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\necho "CI_TRACE_START_TIME=$(date +%s)" >> $GITHUB_ENV\nTRACE_SERVICE="daily-check" TRACE_NAME="每日健康检查 / Staging API Integration Tests" python3 .gitea/scripts/ci_trace_report.py --status running\n\nexit 0\n"
curl -sH "Authorization: token $GITHUB_TOKEN" \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" \
| bash
- name: Run API smoke test on staging
id: smoke
shell: sh
shell: bash
env:
STAGING_TEST_USER: ${{ secrets.STAGING_TEST_USER }}
STAGING_TEST_PASSWORD: ${{ secrets.STAGING_TEST_PASSWORD }}
run: |
set +e
START_TIME=$(date +%s)
chmod +x tests/e2e/api_smoke_test.sh
docker run --rm \
CONTAINER_NAME="ci-test-$$"
docker create --name "$CONTAINER_NAME" \
-e BASE_URL=https://staging-api.xiaoxiajianji.com \
-e WEB_URL=https://staging.xiaoxiajianji.com \
-e TEST_USER=18314979086@163.com \
-e TEST_PASSWORD=Ying1234 \
-e TEST_USER="$STAGING_TEST_USER" \
-e TEST_PASSWORD="$STAGING_TEST_PASSWORD" \
-e CLEANUP_ENABLED=1 \
-e PERF_CHECK_ENABLED=1 \
-e PERF_WARN_THRESHOLD_MS=500 \
-e PERF_FAIL_THRESHOLD_MS=3000 \
-v "$PWD:/workspace" \
-w /workspace \
git.xiaoxiajianji.com/xiaoxia/base/playwright:v1.45.0-jammy \
bash tests/e2e/api_smoke_test.sh 2>&1 | tee /tmp/staging-api-smoke.log
bash tests/e2e/api_smoke_test.sh 2>&1
docker cp . "$CONTAINER_NAME:/workspace"
docker start -a "$CONTAINER_NAME" 2>&1 | tee /tmp/staging-api-smoke.log
SMOKE_EXIT=${PIPESTATUS[0]}
docker rm "$CONTAINER_NAME" > /dev/null 2>&1 || true
END_TIME=$(date +%s)
ELAPSED=$((END_TIME - START_TIME))
@@ -228,18 +146,21 @@ jobs:
- name: Run Staging API Integration Tests (Playwright)
id: e2e_api
shell: sh
shell: bash
run: |
set +e
START_TIME=$(date +%s)
docker run --rm \
CONTAINER_NAME="ci-test-$$"
docker create --name "$CONTAINER_NAME" \
-e E2E_BASE_URL=https://staging.xiaoxiajianji.com \
-e E2E_API_BASE=https://staging-api.xiaoxiajianji.com/api/v1 \
-v "$PWD:/workspace" \
-w /workspace/apps/web \
git.xiaoxiajianji.com/xiaoxia/base/playwright:v1.45.0-jammy \
sh -lc "npm ci && npx playwright test --reporter=line e2e/test_auth.spec.ts e2e/test_asset.spec.ts e2e/test_project.spec.ts" 2>&1 | tee /tmp/staging-api-e2e.log
sh -lc "npm ci && npx playwright test --reporter=line e2e/test_auth.spec.ts e2e/test_asset.spec.ts e2e/test_project.spec.ts" 2>&1
docker cp . "$CONTAINER_NAME:/workspace"
docker start -a "$CONTAINER_NAME" 2>&1 | tee /tmp/staging-api-e2e.log
EXIT_CODE=${PIPESTATUS[0]}
docker rm "$CONTAINER_NAME" > /dev/null 2>&1 || true
END_TIME=$(date +%s)
ELAPSED=$((END_TIME - START_TIME))
@@ -268,19 +189,22 @@ jobs:
echo "report=FAIL" >> "${GITHUB_OUTPUT}"
fi
# ── 3. Staging 浏览器 E2E ──────────────────────────────────────────
- name: Report trace end (always)
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\nif [ "${{ job.status }}" = "success" ]; then\n TRACE_STATUS="ok"\nelse\n TRACE_STATUS="error"\nfi\n\nTRACE_SERVICE="daily-check" TRACE_NAME="每日健康检查 / Staging API Integration Tests" TRACE_STATUS="$TRACE_STATUS" python3 .gitea/scripts/ci_trace_report.py --start-time "$CI_TRACE_START_TIME"\n\nexit 0\n"
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
# ── 3. Staging 浏览器 E2E ──────────────────────────────────────────
staging-e2e:
name: Staging Browser E2E
runs-on: saas
runs-on: ci-l2
timeout-minutes: 15
outputs:
report: ${{ steps.smoke.outputs.report }}
@@ -291,71 +215,28 @@ jobs:
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
python3 - <<'PY'
import io, os, tarfile, time, urllib.request, urllib.error
url = f"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz"
request = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['GITHUB_TOKEN']}"})
last_err = None
for attempt in range(5):
try:
with urllib.request.urlopen(request, timeout=120) as response:
archive = response.read()
break
except urllib.error.HTTPError as e:
last_err = e
if e.code >= 500 and attempt < 4:
wait = 2 ** attempt
print(f"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
except Exception as e:
last_err = e
if attempt < 4:
wait = 2 ** attempt
print(f"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
else:
raise last_err
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
PY
- name: Record start time & report trace start
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\necho "CI_TRACE_START_TIME=$(date +%s)" >> $GITHUB_ENV\nTRACE_SERVICE="daily-check" TRACE_NAME="每日健康检查 / Staging Browser E2E" python3 .gitea/scripts/ci_trace_report.py --status running\n\nexit 0\n"
curl -sH "Authorization: token $GITHUB_TOKEN" \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" \
| bash
- name: Run Playwright E2E on staging
id: e2e
shell: sh
shell: bash
run: |
set +e
START_TIME=$(date +%s)
docker run --rm --ipc=host \
CONTAINER_NAME="ci-test-$$"
docker create --name "$CONTAINER_NAME" --ipc=host \
-e E2E_BASE_URL=https://staging.xiaoxiajianji.com \
-e E2E_API_BASE=https://staging-api.xiaoxiajianji.com/api/v1 \
-e E2E_BROWSER_CHANNEL=chromium \
-e PLAYWRIGHT_HEADLESS=1 \
-v "$PWD:/workspace" \
-w /workspace/apps/web \
git.xiaoxiajianji.com/xiaoxia/base/playwright:v1.45.0-jammy \
sh -lc 'npm ci && npx playwright test --reporter=line --project=chromium e2e/auth.spec.ts e2e/auth-guard.spec.ts e2e/core-upload.spec.ts e2e/core-generation.spec.ts e2e/core-titles.spec.ts' 2>&1 | tee /tmp/staging-e2e.log
sh -lc 'npm ci && npx playwright test --reporter=line --project=chromium e2e/auth.spec.ts e2e/auth-guard.spec.ts e2e/core-upload.spec.ts e2e/core-generation.spec.ts e2e/core-titles.spec.ts' 2>&1
docker cp . "$CONTAINER_NAME:/workspace"
docker start -a "$CONTAINER_NAME" 2>&1 | tee /tmp/staging-e2e.log
EXIT_CODE=${PIPESTATUS[0]}
docker rm "$CONTAINER_NAME" > /dev/null 2>&1 || true
END_TIME=$(date +%s)
ELAPSED=$((END_TIME - START_TIME))
@@ -374,19 +255,22 @@ jobs:
echo "=========================================="
exit $EXIT_CODE
# ── 4. 性能基线巡检 ────────────────────────────────────────────────
- name: Report trace end (always)
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\nif [ "${{ job.status }}" = "success" ]; then\n TRACE_STATUS="ok"\nelse\n TRACE_STATUS="error"\nfi\n\nTRACE_SERVICE="daily-check" TRACE_NAME="每日健康检查 / Staging Browser E2E" TRACE_STATUS="$TRACE_STATUS" python3 .gitea/scripts/ci_trace_report.py --start-time "$CI_TRACE_START_TIME"\n\nexit 0\n"
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
# ── 4. 性能基线巡检 ────────────────────────────────────────────────
performance-check:
name: Performance Baseline Check
runs-on: saas
runs-on: ci-l2
timeout-minutes: 8
outputs:
report: ${{ steps.report.outputs.report }}
@@ -395,6 +279,9 @@ jobs:
- name: Run performance baseline checks
id: perf
shell: sh
env:
STAGING_TEST_USER: ${{ secrets.STAGING_TEST_USER }}
STAGING_TEST_PASSWORD: ${{ secrets.STAGING_TEST_PASSWORD }}
run: |
set +e
START_TIME=$(date +%s)
@@ -430,9 +317,10 @@ jobs:
# 先登录获取 token
echo "--- 准备: 获取测试 Token ---"
LOGIN_BODY="{\"email\":\"$STAGING_TEST_USER\",\"password\":\"$STAGING_TEST_PASSWORD\"}"
AUTH_RESP=$(curl -s -w "\n%{http_code}" -X POST \
-H "Content-Type: application/json" \
-d '{"email":"18314979086@163.com","password":"Ying1234"}' \
-d "$LOGIN_BODY" \
"https://staging-api.xiaoxiajianji.com/api/v1/auth/login" \
--max-time 10 2>&1)
AUTH_CODE=$(echo "$AUTH_RESP" | tail -1)
@@ -462,7 +350,7 @@ jobs:
# 构建 curl 命令
CURL_ARGS="-s -o /dev/null -w '%{http_code} %{time_total}' --max-time 30"
if [ "$method" = "POST" ]; then
CURL_ARGS="$CURL_ARGS -X POST -H 'Content-Type: application/json' -d '{\"email\":\"18314979086@163.com\",\"password\":\"Ying1234\"}'"
CURL_ARGS="$CURL_ARGS -X POST -H 'Content-Type: application/json' -d \"$LOGIN_BODY\""
fi
if [ -n "$TOKEN" ] && [ "$name" != "健康检查" ]; then
CURL_ARGS="$CURL_ARGS -H 'Authorization: Bearer $TOKEN'"
@@ -507,17 +395,12 @@ jobs:
echo "耗时: ${ELAPSED}s"
echo "======================================"
- name: Record start time & report trace start
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\necho "CI_TRACE_START_TIME=$(date +%s)" >> $GITHUB_ENV\nTRACE_SERVICE="daily-check" TRACE_NAME="每日健康检查 / Performance Baseline Check" python3 .gitea/scripts/ci_trace_report.py --status running\n\nexit 0\n"
- name: Generate performance report
id: report
shell: sh
env:
STAGING_TEST_USER: ${{ secrets.STAGING_TEST_USER }}
STAGING_TEST_PASSWORD: ${{ secrets.STAGING_TEST_PASSWORD }}
run: |
set +e
echo ""
@@ -532,10 +415,11 @@ jobs:
RESULTS=""
START_TIME=$(date +%s)
LOGIN_BODY="{\"email\":\"$STAGING_TEST_USER\",\"password\":\"$STAGING_TEST_PASSWORD\"}"
# 先登录获取 token
AUTH_RESP=$(curl -s -w "\n%{http_code}" -X POST \
-H "Content-Type: application/json" \
-d '{"email":"18314979086@163.com","password":"Ying1234"}' \
-d "$LOGIN_BODY" \
"https://staging-api.xiaoxiajianji.com/api/v1/auth/login" \
--max-time 10 2>&1)
AUTH_CODE=$(echo "$AUTH_RESP" | tail -1)
@@ -551,7 +435,7 @@ jobs:
local CURL_ARGS="-s -o /dev/null -w '%{http_code} %{time_total}' --max-time 30"
if [ "$method" = "POST" ]; then
CURL_ARGS="$CURL_ARGS -X POST -H 'Content-Type: application/json' -d '{\"email\":\"18314979086@163.com\",\"password\":\"Ying1234\"}'"
CURL_ARGS="$CURL_ARGS -X POST -H 'Content-Type: application/json' -d \"$LOGIN_BODY\""
fi
if [ -n "$TOKEN" ] && [ "$name" != "健康检查" ]; then
CURL_ARGS="$CURL_ARGS -H 'Authorization: Bearer $TOKEN'"
@@ -639,19 +523,22 @@ jobs:
exit 0
fi
# ── 5. 每日巡检汇总报告 ────────────────────────────────────────────
- name: Report trace end (always)
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\nif [ "${{ job.status }}" = "success" ]; then\n TRACE_STATUS="ok"\nelse\n TRACE_STATUS="error"\nfi\n\nTRACE_SERVICE="daily-check" TRACE_NAME="每日健康检查 / Performance Baseline Check" TRACE_STATUS="$TRACE_STATUS" python3 .gitea/scripts/ci_trace_report.py --start-time "$CI_TRACE_START_TIME"\n\nexit 0\n"
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
# ── 5. 每日巡检汇总报告 ────────────────────────────────────────────
daily-report:
name: Daily Check Report
runs-on: saas
runs-on: ci-l2
timeout-minutes: 2
if: always()
needs:
@@ -724,20 +611,14 @@ jobs:
# 不 exit 1,因为我们用了 always(),保持 report job 成功,
# 但其他失败的 job 已经让整体流水线标记为失败
fi
- name: Record start time & report trace start
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\necho "CI_TRACE_START_TIME=$(date +%s)" >> $GITHUB_ENV\nTRACE_SERVICE="daily-check" TRACE_NAME="每日健康检查 / Daily Check Report" python3 .gitea/scripts/ci_trace_report.py --status running\n\nexit 0\n"
- name: Report trace end (always)
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\nif [ "${{ job.status }}" = "success" ]; then\n TRACE_STATUS="ok"\nelse\n TRACE_STATUS="error"\nfi\n\nTRACE_SERVICE="daily-check" TRACE_NAME="每日健康检查 / Daily Check Report" TRACE_STATUS="$TRACE_STATUS" python3 .gitea/scripts/ci_trace_report.py --start-time "$CI_TRACE_START_TIME"\n\nexit 0\n"
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
+56
View File
@@ -0,0 +1,56 @@
name: PR Auto Scan
# 定时扫描所有open PR,对CI全绿的触发审批/合并
# 作为短作业模式的兜底,防止事件驱动遗漏
on:
schedule:
- cron: "*/5 * * * *" # 每5分钟扫描一次
workflow_dispatch:
permissions:
contents: read
jobs:
auto-scan:
name: Auto Scan Open PRs
runs-on: ci-check
timeout-minutes: 5
if: github.repository == 'xiaoxia/xiaoxia-saas'
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sH "Authorization: token $GITHUB_TOKEN" "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/pr_auto_scan.py?ref=develop" -o /tmp/pr_auto_scan.py
python3 /tmp/pr_auto_scan.py --help > /dev/null 2>&1 || {
# fallback: checkout
echo "使用checkout方式"
curl -sH "Authorization: token $GITHUB_TOKEN" "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=develop" | bash
}
- name: Scan and auto process PRs
shell: bash
env:
GITHUB_TOKEN: ${{ github.token }}
REVIEW_TOKEN: ${{ secrets.REVIEW_GITEA_TOKEN }}
MERGE_TOKEN: ${{ secrets.REVIEW_GITEA_TOKEN }}
run: |
set -eu
echo "=== 扫描所有open PR并自动处理 ==="
echo "时间: $(date)"
echo
python3 /tmp/pr_auto_scan.py --token "$REVIEW_TOKEN" --repo "$GITHUB_REPOSITORY" --base develop --approve --merge --dry-run false
echo ""
echo "✅ 扫描完成"
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "" || true
+113
View File
@@ -0,0 +1,113 @@
name: PR Automation
on:
pull_request:
types: [synchronize, opened, ready_for_review, review_requested]
workflow_dispatch:
permissions:
contents: read
jobs:
auto-approve:
name: Auto Approve on CI Green
runs-on: ci-check
if: github.event_name == 'pull_request' && !github.event.pull_request.draft
timeout-minutes: 3 # 长等待模式:等CI全绿后自动合并,不遗漏任何PR
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sH "Authorization: token $GITHUB_TOKEN" "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" | bash
- name: "🔍 脚本语法自检"
shell: bash
run: |
ERROR=0
for f in scripts/ci/*.sh; do [ -f "$f" ] && bash -n "$f" 2>&1 || ERROR=$((ERROR+1)); done
for f in scripts/ci/*.py; do [ -f "$f" ] && python3 -m py_compile "$f" 2>&1 || ERROR=$((ERROR+1)); done
if [ "$ERROR" -ne 0 ]; then echo "❌ 语法自检失败 ($ERROR个)"; exit 1; fi
echo "✅ 脚本语法自检通过"
- name: Auto approve when CI passes
shell: bash
env:
GITHUB_TOKEN: ${{ github.token }}
REVIEW_TOKEN: ${{ secrets.REVIEW_GITEA_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
run: |
bash scripts/ci/auto_approve.sh
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
auto-merge:
name: Auto Merge on CI Green + Approved
runs-on: ci-check
if: github.event_name == 'pull_request' && !github.event.pull_request.draft && github.event.pull_request.base.ref == 'develop'
timeout-minutes: 45 # 长等待模式:等CI全绿后自动合并,不遗漏任何PR
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sH "Authorization: token $GITHUB_TOKEN" "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" | bash
- name: "🔍 脚本语法自检(防止脚本bug导致所有PR挂掉)"
shell: bash
run: |
echo "=== CI脚本语法自检 ==="
ERROR=0
for f in scripts/ci/*.sh; do
[ -f "$f" ] || continue
if ! bash -n "$f" 2>&1; then
echo "FAIL: $f"
ERROR=1
fi
done
for f in scripts/ci/*.py; do
[ -f "$f" ] || continue
if ! python3 -m py_compile "$f" 2>&1; then
echo "FAIL: $f"
ERROR=1
fi
done
if [ "$ERROR" -ne 0 ]; then
echo "❌ 脚本语法自检失败"
exit 1
fi
echo "✅ 所有CI脚本语法自检通过"
- name: Auto merge when CI passes and approved
shell: bash
env:
GITHUB_TOKEN: ${{ github.token }}
MERGE_TOKEN: ${{ secrets.REVIEW_GITEA_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
BASE_REF: ${{ github.event.pull_request.base.ref }}
run: |
bash scripts/ci/auto_merge.sh
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
+9 -14
View File
@@ -61,14 +61,6 @@ jobs:
tar.extract(member, '.')
PY
- name: Record start time & report trace start
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\necho "CI_TRACE_START_TIME=$(date +%s)" >> $GITHUB_ENV\nTRACE_SERVICE="" TRACE_NAME="" python3 .gitea/scripts/ci_trace_report.py --status running\n\nexit 0\n"
- name: Extract PR number
shell: sh
run: |
@@ -201,12 +193,15 @@ jobs:
> /dev/null
echo "Cleanup comment posted"
- name: Report trace end (always)
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\nif [ "${{ job.status }}" = "success" ]; then\n TRACE_STATUS="ok"\nelse\n TRACE_STATUS="error"\nfi\n\nTRACE_SERVICE="" TRACE_NAME="" TRACE_STATUS="$TRACE_STATUS" python3 .gitea/scripts/ci_trace_report.py --start-time "$CI_TRACE_START_TIME"\n\nexit 0\n"
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
+21 -16
View File
@@ -72,14 +72,6 @@ jobs:
tar.extract(member, '.')
PY
- name: Record start time & report trace start
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\necho "CI_TRACE_START_TIME=$(date +%s)" >> $GITHUB_ENV\nTRACE_SERVICE="" TRACE_NAME="" python3 .gitea/scripts/ci_trace_report.py --status running\n\nexit 0\n"
- name: Record job start time
shell: sh
run: |
@@ -165,8 +157,8 @@ jobs:
PREVIEW_SSH_KEY: ${{ secrets.PREVIEW_SSH_KEY }}
run: |
set -eux
preview_host="${PREVIEW_SSH_HOST:-172.30.18.197}"
preview_user="${PREVIEW_SSH_USER:-deploy}"
preview_host="${PREVIEW_SSH_HOST:-47.98.113.167}"
preview_user="${PREVIEW_SSH_USER:-root}"
preview_port="${PREVIEW_SSH_PORT:-22222}"
preview_dir="/var/www/preview/pr-${PR_NUMBER}"
@@ -192,6 +184,16 @@ jobs:
exit 1
fi
# SSH密钥完整性自检
if ! ssh-keygen -y -f "$key_path" > /dev/null 2>&1; then
echo "ERROR: SSH密钥损坏(private key contents do not match public"
echo "请检查 PREVIEW_SSH_KEY secret 中的私钥是否完整正确"
echo "私钥文件大小: $(wc -c < "$key_path") 字节"
head -2 "$key_path"
exit 1
fi
echo "SSH key integrity check passed"
ssh-keyscan -p "$preview_port" -H "$preview_host" >> ~/.ssh/known_hosts 2>/dev/null
echo "SSH keyscan done"
@@ -280,12 +282,15 @@ jobs:
run: |
set +e
NOTIFY_MODE=failure JOB_NAME="Deploy Preview Environment" python3 scripts/ci_notify.py
- name: Report trace end (always)
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_ENDPOINT: ${ secrets.AGENTLOOP_ENDPOINT }
AGENTLOOP_LICENSE_KEY: ${ secrets.AGENTLOOP_LICENSE_KEY }
AGENTLOOP_PROJECT: ${ secrets.AGENTLOOP_PROJECT }
AGENTLOOP_WORKSPACE: ${ secrets.AGENTLOOP_WORKSPACE }
run: "set +e\n\nif [ "${{ job.status }}" = "success" ]; then\n TRACE_STATUS="ok"\nelse\n TRACE_STATUS="error"\nfi\n\nTRACE_SERVICE="" TRACE_NAME="" TRACE_STATUS="$TRACE_STATUS" python3 .gitea/scripts/ci_trace_report.py --start-time "$CI_TRACE_START_TIME"\n\nexit 0\n"
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
+103
View File
@@ -0,0 +1,103 @@
name: Worker Base Image Build
on:
push:
branches:
- develop
- main
paths:
- 'requirements-base.txt'
- 'requirements-worker.txt'
- 'infra/docker/worker-base-builder.Dockerfile'
- 'infra/docker/worker-base-runtime.Dockerfile'
workflow_dispatch: # 支持手动触发
jobs:
build-worker-base:
name: Build Worker Base Images
runs-on: runtime-builder
timeout-minutes: 30
strategy:
fail-fast: false
matrix:
include:
- name: builder
dockerfile: infra/docker/worker-base-builder.Dockerfile
image_name: worker-base-builder
cache_name: worker-base-builder-cache
- name: runtime
dockerfile: infra/docker/worker-base-runtime.Dockerfile
image_name: worker-base-runtime
cache_name: worker-base-runtime-cache
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sH "Authorization: token $GITHUB_TOKEN" "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" | bash
- name: Docker login to Registry
shell: sh
env:
ACR_USERNAME: ${{ secrets.ACR_USERNAME }}
ACR_PASSWORD: ${{ secrets.ACR_PASSWORD }}
GITEA_REGISTRY_USER: xiaoxia
GITEA_REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
set -eu
for i in 1 2 3; do
echo "=== Docker login 尝试 $i/3 ==="
if printf '%s' "${ACR_PASSWORD}" | docker login xiaoxia-registry.cn-hangzhou.cr.aliyuncs.com -u "${ACR_USERNAME}" --password-stdin && docker login git.xiaoxiajianji.com -u "${GITEA_REGISTRY_USER}" -p "${GITEA_REGISTRY_TOKEN}"; then
echo "✅ Docker login successful"
break
fi
echo "❌ Docker login 失败(尝试 $i/3),5s 后重试..."
sleep 5
done
- name: Setup buildx builder
shell: sh
run: |
set -eu
BUILDER_NAME="ci-builder-${GITHUB_RUN_ID}-${{ matrix.name }}"
if ! docker buildx inspect "$BUILDER_NAME" > /dev/null 2>&1; then
docker buildx create --use --name "$BUILDER_NAME" --driver docker-container
echo "Created $BUILDER_NAME"
else
docker buildx use "$BUILDER_NAME"
echo "Using existing $BUILDER_NAME"
fi
docker buildx inspect --bootstrap
- name: Build and push base image
shell: sh
run: |
set -eu
REGISTRY="xiaoxia-registry.cn-hangzhou.cr.aliyuncs.com/xiaoxiakeji"
IMAGE_TAG="${REGISTRY}/${{ matrix.image_name }}:latest"
SAFE_REF_NAME=$(echo "${GITHUB_REF_NAME}" | tr '/' '-')
CACHE_REF="${REGISTRY}/${{ matrix.cache_name }}:${SAFE_REF_NAME}"
echo "=== Building ${{ matrix.name }} base image ==="
echo "Image: ${IMAGE_TAG}"
echo "Cache: ${CACHE_REF}"
# 用通用构建脚本
bash scripts/ci/docker_build_push.sh ${{ matrix.dockerfile }} "${IMAGE_TAG}" "${CACHE_REF}"
# 同时推送到 Gitea Packages 作为备份(可选)
GITEA_IMAGE="git.xiaoxiajianji.com/xiaoxia-saas/${{ matrix.image_name }}:latest"
docker tag "${IMAGE_TAG}" "${GITEA_IMAGE}"
docker push "${GITEA_IMAGE}" || echo "Gitea Packages push failed (non-fatal)"
echo ""
echo "✅ ${{ matrix.name }} base image built and pushed"
- name: Cleanup buildx builder
if: always()
shell: sh
run: |
docker buildx rm "ci-builder-${GITHUB_RUN_ID}-${{ matrix.name }}" 2>/dev/null || true
docker buildx prune -f 2>/dev/null || true
echo "Builder cleanup done"
-5
View File
@@ -6,7 +6,6 @@ dist/
coverage/
# Python / backend
.cache/
.venv/
venv/
.venv-ci-root/
@@ -49,7 +48,3 @@ build/
tracker_tasks.json
frontend-v21-ui-prototype-final.html
!.vscode/
!.vscode/settings.json
.vscode/extensions.json
-29
View File
@@ -1,29 +0,0 @@
repos:
- repo: https://github.com/psf/black
rev: 26.5.1
hooks:
- id: black
language_version: python3.12
- repo: https://github.com/pycqa/isort
rev: 8.0.1
hooks:
- id: isort
args: ["--profile", "black"]
language_version: python3.12
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.14.0
hooks:
- id: ruff
args: [--fix]
language_version: python3.12
- repo: https://github.com/pre-commit/mirrors-prettier
rev: v4.0.0-alpha.11
hooks:
- id: prettier
name: prettier (frontend)
files: ^apps/web/.*\.(ts|tsx|js|jsx|css|scss|less|json|html|md|yaml|yml)$
additional_dependencies:
- prettier@3.4.2
-49
View File
@@ -1,49 +0,0 @@
{
"[python]": {
"editor.defaultFormatter": "ms-python.black-formatter",
"editor.formatOnSave": true,
"editor.codeActionsOnSave": {
"source.organizeImports": "explicit"
}
},
"[typescript]": {
"editor.defaultFormatter": "esbenp.prettier-vscode",
"editor.formatOnSave": true
},
"[typescriptreact]": {
"editor.defaultFormatter": "esbenp.prettier-vscode",
"editor.formatOnSave": true
},
"[javascript]": {
"editor.defaultFormatter": "esbenp.prettier-vscode",
"editor.formatOnSave": true
},
"[javascriptreact]": {
"editor.defaultFormatter": "esbenp.prettier-vscode",
"editor.formatOnSave": true
},
"[json]": {
"editor.defaultFormatter": "esbenp.prettier-vscode",
"editor.formatOnSave": true
},
"[css]": {
"editor.defaultFormatter": "esbenp.prettier-vscode",
"editor.formatOnSave": true
},
"[scss]": {
"editor.defaultFormatter": "esbenp.prettier-vscode",
"editor.formatOnSave": true
},
"[html]": {
"editor.defaultFormatter": "esbenp.prettier-vscode",
"editor.formatOnSave": true
},
"[markdown]": {
"editor.defaultFormatter": "esbenp.prettier-vscode",
"editor.formatOnSave": true
},
"prettier.requireConfig": true,
"isort.args": ["--profile", "black"],
"python.linting.ruffEnabled": true,
"python.analysis.typeCheckingMode": "basic"
}
@@ -1,28 +0,0 @@
"""CMS Enhancements (placeholder - manually applied on production)
Revision ID: 034_cms_enhance
Revises: 033
Create Date: 2026-07-09
占位迁移文件:生产数据库已手动升级到此版本,
此文件用于让 alembic 识别当前版本,避免部署时迁移失败。
实际的表结构变更(helpcenter, tickets, partners, site_settings 等)
已在生产环境手动执行。
"""
from alembic import op
revision = "034_cms_enhance"
down_revision = "033"
branch_labels = None
depends_on = None
def upgrade() -> None:
"""占位 - 变更已在生产环境手动应用"""
pass
def downgrade() -> None:
"""占位 - 不执行实际回退"""
pass
@@ -1,26 +0,0 @@
"""Add editing_mode to edit_templates
Revision ID: 035_editing_mode
Revises: 034_cms_enhance
Create Date: 2026-07-09
"""
import sqlalchemy as sa
from alembic import op
revision = "035_editing_mode"
down_revision = "034_cms_enhance"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"edit_templates",
sa.Column("editing_mode", sa.String(20), nullable=False, server_default="one_take"),
)
def downgrade() -> None:
op.drop_column("edit_templates", "editing_mode")
@@ -1,68 +0,0 @@
"""Expand UUID fields from varchar(32) to varchar(36)
All UUID fields across all tables were varchar(32), but standard UUIDs with
hyphens are 36 characters (e.g. 550e8400-e29b-41d4-a716-446655440000).
This caused StringDataRightTruncation errors on insert.
Revision ID: 036_expand_uuid_36
Revises: 035_editing_mode
Create Date: 2026-07-10
"""
import sqlalchemy as sa
from alembic import op
revision = "036_expand_uuid_36"
down_revision = "035_editing_mode"
branch_labels = None
depends_on = None
# ── 表 → 需要扩容的列 ─────────────────────────────────────────────────────────
_TABLES: dict[str, list[str]] = {
"projects": ["id", "owner_user_id"],
"edit_templates": ["id"],
"edit_plans": ["id", "template_id", "source_edit_plan_id", "project_id", "created_by_user_id"],
"template_clip_configs": ["id", "template_id"],
"edit_plan_clips": ["id", "plan_id", "template_clip_config_id", "asset_id"],
"ingest_jobs": ["id", "project_id", "library_id", "result_asset_id"],
"classification_jobs": ["id", "project_id", "asset_id"],
"generation_tasks": [
"id",
"project_id",
"strategy_id",
"asset_library_id",
"voice_library_id",
"created_by_user_id",
"source_edit_plan_id",
"batch_id",
],
"generated_videos": ["id", "project_id", "generation_task_id", "duplicate_of"],
"jobs": ["id", "project_id", "source_id", "created_by_user_id"],
}
def upgrade() -> None:
for table, columns in _TABLES.items():
for col in columns:
op.alter_column(
table,
col,
existing_type=sa.String(32),
type_=sa.String(36),
existing_nullable=None,
)
def downgrade() -> None:
for table, columns in reversed(list(_TABLES.items())):
for col in columns:
op.alter_column(
table,
col,
existing_type=sa.String(36),
type_=sa.String(32),
existing_nullable=None,
)
@@ -1,26 +0,0 @@
"""Add logs field to generation_tasks
Revision ID: 037_generation_logs
Revises: 036_expand_uuid_36
Create Date: 2026-07-10
"""
import sqlalchemy as sa
from alembic import op
revision = "037_generation_logs"
down_revision = "036_expand_uuid_36"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"generation_tasks",
sa.Column("logs", sa.Text(), nullable=False, server_default="[]"),
)
def downgrade() -> None:
op.drop_column("generation_tasks", "logs")
@@ -1,47 +0,0 @@
"""add error_info and retry fields to generation_tasks
Revision ID: 038_error_retry
Revises: 037_generation_logs
Create Date: 2026-07-13 22:15:00.000000
"""
import sqlalchemy as sa
from sqlalchemy.dialects.mysql import JSON as MySQLJSON
from alembic import op
# revision identifiers, used by Alembic.
revision = "038_error_retry"
down_revision = "037_generation_logs"
branch_labels = None
depends_on = None
def upgrade():
# error_info: 结构化错误信息(error_type, message, stack_trace, failed_at, stage等)
op.add_column(
"generation_tasks",
sa.Column("error_info", sa.JSON(), nullable=True),
)
# retry_count: 重试次数
op.add_column(
"generation_tasks",
sa.Column("retry_count", sa.Integer(), nullable=False, server_default="0"),
)
# auto_retry_enabled: 是否开启自动重试
op.add_column(
"generation_tasks",
sa.Column("auto_retry_enabled", sa.Boolean(), nullable=False, server_default=sa.text("false")),
)
# auto_retry_max: 最大自动重试次数
op.add_column(
"generation_tasks",
sa.Column("auto_retry_max", sa.Integer(), nullable=False, server_default="0"),
)
def downgrade():
op.drop_column("generation_tasks", "auto_retry_max")
op.drop_column("generation_tasks", "auto_retry_enabled")
op.drop_column("generation_tasks", "retry_count")
op.drop_column("generation_tasks", "error_info")
@@ -1,34 +0,0 @@
"""add transition_duration to edit_plan_clips
Revision ID: 039_transition_duration
Revises: 038_error_retry
Create Date: 2026-07-14 09:00:00.000000
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision = "039_transition_duration"
down_revision = "038_error_retry"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"edit_plan_clips",
sa.Column(
"transition_duration",
sa.Float(),
nullable=False,
server_default="0.0",
),
)
def downgrade() -> None:
op.drop_column("edit_plan_clips", "transition_duration")
@@ -1,29 +0,0 @@
"""add playback_speed to edit_plan_clips
Revision ID: 040_playback_speed
Revises: 039_transition_duration
Create Date: 2026-07-14 10:00:00.000000
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision = "040_playback_speed"
down_revision = "039_transition_duration"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"edit_plan_clips",
sa.Column("playback_speed", sa.Float(), nullable=False, server_default="1.0"),
)
def downgrade() -> None:
op.drop_column("edit_plan_clips", "playback_speed")
@@ -1,29 +0,0 @@
"""add result_count to edit_plans
Revision ID: 041_result_count
Revises: 040_playback_speed
Create Date: 2026-07-15 14:05:00.000000
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision = "041_result_count"
down_revision = "040_playback_speed"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"edit_plans",
sa.Column("result_count", sa.Integer(), nullable=False, server_default="0"),
)
def downgrade() -> None:
op.drop_column("edit_plans", "result_count")
@@ -1,29 +0,0 @@
"""add storage_key to assets
Revision ID: 042_storage_key
Revises: 041_result_count
Create Date: 2026-07-17 18:10:00.000000
"""
from __future__ import annotations
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision = "042_storage_key"
down_revision = "041_result_count"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"assets",
sa.Column("storage_key", sa.String(500), nullable=False, server_default=""),
)
def downgrade() -> None:
op.drop_column("assets", "storage_key")
@@ -1,33 +0,0 @@
"""add updated_at to generation_tasks
Revision ID: 043_updated_at_generation_tasks
Revises: 042_storage_key
Create Date: 2026-07-18 19:30:00.000000
"""
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision = "043_updated_at_generation_tasks"
down_revision = "042_storage_key"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"generation_tasks",
sa.Column(
"updated_at",
sa.DateTime,
nullable=False,
server_default=sa.func.now(),
),
)
def downgrade() -> None:
op.drop_column("generation_tasks", "updated_at")
@@ -1,34 +0,0 @@
"""add user_id to generated_videos
Revision ID: 044_user_id_generated_videos
Revises: 043_updated_at_generation_tasks
Create Date: 2026-07-19 08:00:00.000000
"""
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision = "044_user_id_generated_videos"
down_revision = "043_updated_at_generation_tasks"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"generated_videos",
sa.Column(
"user_id",
sa.String(36),
nullable=False,
server_default="",
index=True,
),
)
def downgrade() -> None:
op.drop_column("generated_videos", "user_id")
-36
View File
@@ -1,36 +0,0 @@
"""backfill user_id for generated_videos from generation_tasks
Revision ID: 045_backfill_user_id_generated_videos
Revises: 044_user_id_generated_videos
Create Date: 2026-07-19 10:50:00.000000
"""
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision = "045_backfill_user_id"
down_revision = "044_user_id_generated_videos"
branch_labels = None
depends_on = None
def upgrade() -> None:
# 回填 generated_videos.user_id:通过 generation_task_id 关联 generation_tasks 表
# 取 generation_tasks.created_by_user_id 作为 user_id
# 回填不到的(无关联task的兜底记录)保持空字符串
op.execute("""
UPDATE generated_videos gv
SET user_id = gt.created_by_user_id
FROM generation_tasks gt
WHERE gv.generation_task_id = gt.id
AND gv.user_id = ''
AND gt.created_by_user_id != ''
""")
def downgrade() -> None:
# 降级不做处理(无法精确区分哪些是回填的)
pass
-33
View File
@@ -1,33 +0,0 @@
"""add video_title to generation_tasks
Revision ID: 046_add_video_title_to_generation_tasks
Revises: 045_backfill_user_id_generated_videos
Create Date: 2026-07-19 11:20:00.000000
"""
import sqlalchemy as sa
from alembic import op
# revision identifiers, used by Alembic.
revision = "046_task_title"
down_revision = "045_backfill_user_id"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column(
"generation_tasks",
sa.Column(
"video_title",
sa.String(255),
nullable=False,
server_default="",
),
)
def downgrade() -> None:
op.drop_column("generation_tasks", "video_title")
+1
View File
@@ -0,0 +1 @@
"""API application package."""
+1
View File
@@ -0,0 +1 @@
"""API package."""
+29 -15
View File
@@ -4,14 +4,17 @@ from app.api.routes.assets import router as assets_router
from app.api.routes.auth import router as auth_router
from app.api.routes.chunked_upload import router as chunked_upload_router
from app.api.routes.classification_jobs import router as classification_jobs_router
from app.api.routes.dashboard import router as dashboard_router
from app.api.routes.duplication import router as duplication_router
from app.api.routes.edit_plans import router as edit_plans_router
from app.api.routes.feature_flags import router as feature_flags_router
from app.api.routes.edit_templates import router as edit_templates_router
from app.api.routes.generated_videos import router as generated_videos_router
from app.api.routes.generation_tasks import router as generation_tasks_router
from app.api.routes.health import router as health_check_router
from app.api.routes.ingest_jobs import router as ingest_jobs_router
from app.api.routes.internal_render import router as internal_render_router
from app.api.routes.jobs import router as jobs_router
from app.api.routes.projects import router as projects_router
from app.api.routes.recipes import router as recipes_router
from app.api.routes.subscription import router as subscription_router
from app.api.routes.tags import router as tags_router
from app.api.routes.task_center import router as task_center_router
@@ -19,7 +22,6 @@ from app.api.routes.templates import router as templates_router
from app.api.routes.titles import router as titles_router
from app.api.routes.tts import router as tts_router
from app.api.routes.upload import router as upload_router
from app.api.routes.videos import router as videos_router
from app.api.routes.voice_clones import router as voice_clones_router
from app.api.routes.voices import router as voices_router
from fastapi import APIRouter
@@ -85,6 +87,15 @@ api_router.include_router(
prefix="/generation",
tags=["Generation"],
)
api_router.include_router(
jobs_router,
tags=["Job"],
)
api_router.include_router(
generated_videos_router,
prefix="/generated-videos",
tags=["GeneratedVideo"],
)
api_router.include_router(
titles_router,
prefix="/titles",
@@ -100,10 +111,6 @@ api_router.include_router(
prefix="/voice-clones",
tags=["VoiceClone"],
)
api_router.include_router(
videos_router,
tags=["VideoCenter"],
)
api_router.include_router(
duplication_router,
prefix="/duplication",
@@ -114,11 +121,26 @@ api_router.include_router(
prefix="/subscription",
tags=["Subscription"],
)
api_router.include_router(
recipes_router,
prefix="/recipes",
tags=["Recipe"],
)
api_router.include_router(
templates_router,
prefix="/templates",
tags=["Template"],
)
api_router.include_router(
dashboard_router,
prefix="/dashboard",
tags=["Dashboard"],
)
api_router.include_router(
edit_templates_router,
prefix="/edit-templates",
tags=["EditTemplate"],
)
api_router.include_router(
edit_plans_router,
prefix="/edit-plans",
@@ -129,11 +151,3 @@ api_router.include_router(
prefix="/tts",
tags=["TTS"],
)
api_router.include_router(
feature_flags_router,
tags=["Internal"],
)
api_router.include_router(
internal_render_router,
tags=["Internal"],
)
-141
View File
@@ -1,141 +0,0 @@
"""路由层共享辅助函数 — 消除跨文件重复定义。"""
from datetime import datetime, timezone
from typing import Any
from fastapi import HTTPException, status
from packages.application import GetProjectUseCase
from packages.ports.user_repository import UserRepository
def check_project_access(project_id: str, user_id: str, project_repository) -> None:
"""检查用户是否有项目访问权限。
合并自 asset_libraries.py / edit_plans.py 的同名函数。
- 空 project_id 直接放行(兼容 edit_plans 中 project_id 可选的场景)
- 错误信息使用中文,与项目其他路由保持一致
"""
if not project_id or not project_id.strip():
return
project = project_repository.find_by_id(project_id)
if project is None:
raise HTTPException(status_code=404, detail="项目不存在")
if not project.can_access(user_id):
raise HTTPException(status_code=403, detail="无权访问该项目")
def get_user_plan(user_id: str, user_repository: UserRepository) -> str:
"""获取用户的订阅计划名称。"""
user = user_repository.find_by_id(user_id)
if user is None:
return "free"
return getattr(user, "subscription_plan", "free") or "free"
def require_project_and_library(
project_id: str,
library_id: str,
project_repository: Any,
asset_library_repository: Any,
) -> None:
"""Verify project and asset library exist."""
project = GetProjectUseCase(project_repository).execute(project_id)
if project is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Project not found")
libraries = asset_library_repository.find_by_project(project_id)
if not any(item.id == library_id for item in libraries):
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Asset library not found")
def auto_select_video_assets(
*,
project_id: str,
asset_library_repo: Any,
asset_repo: Any,
logger=None,
) -> list[str]:
"""从项目视频素材库自动选取 ready 状态的视频素材。
Args:
project_id: 项目 ID
asset_library_repo: 素材库仓储
asset_repo: 素材仓储
logger: 可选的 logger 实例,用于记录警告
Returns:
选中的素材 ID 列表,无可用素材时返回空列表
"""
if not project_id:
return []
# 找到项目的视频素材库
libs = asset_library_repo.find_by_project(project_id)
video_lib = None
for lib in libs:
lib_kind = lib.kind.value if hasattr(lib.kind, "value") else lib.kind
if lib_kind == "video":
video_lib = lib
break
if not video_lib:
if logger:
logger.warning("自动选素材: 项目 %s 无视频素材库", project_id)
return []
# 从素材库中选取 ready 状态的视频素材
assets = asset_repo.find_by_library(video_lib.id)
ready_videos = [
a
for a in assets
if (a.status.value if hasattr(a.status, "value") else a.status) == "ready"
and a.mime_type
and a.mime_type.startswith("video")
]
# 过滤横屏素材(只保留竖屏/正方形)
# 移动端短视频场景默认竖屏,横屏素材裁剪后画面不可用
# 注意:这只是选素材优化,渲染引擎本身支持任何分辨率的素材
filtered_videos = []
skipped_landscape = 0
for a in ready_videos:
width = a.width if hasattr(a, "width") and a.width else 0
height = a.height if hasattr(a, "height") and a.height else 0
if not width or not height:
# 从 metadata 兜底
if a.metadata and isinstance(a.metadata, dict):
width = int(a.metadata.get("width", 0) or 0)
height = int(a.metadata.get("height", 0) or 0)
if width and height and width > height:
skipped_landscape += 1
continue
filtered_videos.append(a)
if skipped_landscape and logger:
logger.warning("自动选素材: 跳过 %d 个横屏素材", skipped_landscape)
if not filtered_videos:
if logger:
logger.warning("自动选素材: 素材库 %s 无可用视频素材", video_lib.name)
return []
# 按创建时间降序(新素材在前)
filtered_videos.sort(key=lambda a: a.created_at, reverse=True)
return [a.id for a in filtered_videos]
def format_utc_datetime(dt: datetime | None) -> str:
"""将数据库读出的 UTC naive datetime 格式化为带时区的 ISO 8601 字符串。
数据库 DateTime 列不带时区信息,但存的是 UTC 时间。
直接 .isoformat() 输出无时区标识,前端会按本地时间解析,导致差 8 小时。
输出带 Z 后缀,前端 new Date() 自动转本地时间。
"""
if dt is None:
return ""
if isinstance(dt, str):
return dt
if dt.tzinfo is None:
return dt.isoformat() + "Z"
return dt.astimezone(timezone.utc).isoformat().replace("+00:00", "Z")
+11 -34
View File
@@ -1,18 +1,14 @@
from typing import Any
from app.auth import AuthenticatedUser, get_current_user
from app.dependencies import (
get_asset_library_repository,
get_asset_repository,
get_project_repository,
)
from app.dependencies import get_asset_library_repository, get_project_repository
from app.schemas.asset_library import (
AssetLibraryResponse,
CreateAssetLibraryRequest,
EnsureDefaultLibraryRequest,
ListAssetLibrariesResponse,
)
from fastapi import APIRouter, Depends, HTTPException, Query, Response, status
from fastapi import APIRouter, Depends, HTTPException, Query, status
from packages.application import (
CreateAssetLibraryCommand,
@@ -22,11 +18,18 @@ from packages.application import (
)
from packages.domain import AssetLibrary, AssetLibraryKind
from ._helpers import check_project_access
router = APIRouter()
def _check_project_access(project_id: str, user_id: str, project_repository) -> None:
"""检查用户是否有项目访问权限"""
project = project_repository.find_by_id(project_id)
if project is None:
raise HTTPException(status_code=404, detail=f"Project {project_id} not found")
if not project.can_access(user_id):
raise HTTPException(status_code=status.HTTP_403_FORBIDDEN, detail="Access denied to project")
def _to_asset_library_response(item) -> AssetLibraryResponse:
return AssetLibraryResponse(
id=item.id,
@@ -144,29 +147,3 @@ def ensure_default_library(
)
created = asset_library_repository.create(library)
return _to_asset_library_response(created)
@router.delete("/{library_id}", status_code=status.HTTP_204_NO_CONTENT, response_class=Response)
def delete_asset_library(
library_id: str,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
asset_library_repository: Any = Depends(get_asset_library_repository),
asset_repository: Any = Depends(get_asset_repository),
project_repository: Any = Depends(get_project_repository),
) -> None:
"""删除素材库,同时删除库内所有素材。"""
# 查找素材库
library = asset_library_repository.find_by_id(library_id)
if library is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="素材库不存在")
# 权限校验:检查用户是否有项目访问权限
check_project_access(library.project_id, authenticated_user.user.id, project_repository)
# 删除库内所有素材(硬删除,素材库已删除,无需保留软删除状态)
assets_in_library = asset_repository.find_by_library(library_id)
for asset in assets_in_library:
asset_repository.delete(asset.id)
# 删除素材库本身
asset_library_repository.delete(library_id)
+71 -293
View File
@@ -1,7 +1,6 @@
import logging
from typing import Any, Optional
from app.api.routes._helpers import check_project_access, format_utc_datetime
from app.auth import AuthenticatedUser, get_current_user
from app.core.storage import get_storage_service
from app.dependencies import (
@@ -12,18 +11,15 @@ from app.dependencies import (
)
from app.schemas.asset import (
AssetResponse,
BatchClassifyRequest,
BatchDeleteRequest,
BatchMarkRequest,
BatchOperationResponse,
BatchTagRequest,
BatchDeleteResponse,
CreateAssetRequest,
ListAssetsResponse,
UpdateAssetRequest,
UpdateAssetReviewRequest,
)
from app.schemas.tag import TagAssetsRequest
from fastapi import APIRouter, Depends, HTTPException, Query, Response
from fastapi import APIRouter, Depends, HTTPException, Query
from packages.application import (
CreateAssetCommand,
@@ -71,12 +67,20 @@ def _to_asset_response(item, storage_service=None) -> AssetResponse:
status=item.status.value,
classification_status=item.classification_status.value,
quality_score=item.quality_score,
created_at=format_utc_datetime(item.created_at),
uploaded_by_user_id=item.uploaded_by_user_id,
tag_ids=getattr(item, "tag_ids", []),
)
def _check_project_access(project_id: str, user_id: str, project_repository) -> None:
"""检查用户是否有项目访问权限"""
project = project_repository.find_by_id(project_id)
if project is None:
raise HTTPException(status_code=404, detail=f"Project {project_id} not found")
if not project.can_access(user_id):
raise HTTPException(status_code=403, detail="Access denied to project")
@router.get("", response_model=ListAssetsResponse)
def list_assets(
library_id: Optional[str] = Query(None),
@@ -86,21 +90,6 @@ def list_assets(
gender: Optional[str] = Query(None, description="按 metadata.gender 筛选"),
style: Optional[str] = Query(None, description="按 metadata.style 筛选"),
tag_ids: Optional[str] = Query(None, description="按标签 ID 筛选(逗号分隔,取交集)"),
smart_view: Optional[str] = Query(
None,
description="智能视图筛选:recommended=推荐(质量分≥80)、cautious=慎用(60-79)、risky=高风险(<60或已驳回)、unused=未使用、used=已使用、pending_review=待复核",
pattern="^(recommended|cautious|risky|unused|used|pending_review)$",
),
classification: Optional[str] = Query(
None,
description="按内容分类筛选:scenic=风景、product=产品、person=人物、animal=动物、food=美食、tech=科技、sport=运动、music=音乐、other=其他",
),
status: Optional[str] = Query(
"default",
description="按状态筛选,逗号分隔多值;默认返回除deleted外的所有状态;传deleted查看回收站;传all返回所有状态",
),
page: Optional[int] = Query(None, ge=1, description="页码,从1开始;与 page_size 配对使用,优先于 skip/limit"),
page_size: Optional[int] = Query(None, ge=1, le=500, description="每页数量;与 page 配对使用"),
skip: int = Query(0, ge=0),
limit: int = Query(100, ge=1, le=500),
authenticated_user: AuthenticatedUser = Depends(get_current_user),
@@ -110,26 +99,6 @@ def list_assets(
) -> ListAssetsResponse:
user_id = authenticated_user.user.id
# ── 分页:page/page_size 优先于 skip/limit
if page is not None and page_size is not None:
skip = (page - 1) * page_size
limit = page_size
# ── 解析 status 过滤
status_list: list[str] | None
if status and status.lower() == "all":
status_list = None # None = 不过滤,返回所有状态
elif status and status.lower() == "deleted":
status_list = ["deleted"] # 仅查回收站
elif status and status.lower() == "default":
status_list = ["ready", "uploading", "processing", "error"] # 默认排除deleted
elif status:
status_list = [s.strip() for s in status.split(",") if s.strip()]
if not status_list:
status_list = ["ready", "uploading", "processing", "error"]
else:
status_list = ["ready", "uploading", "processing", "error"]
# kind → file_type 映射(voice 对应 audio
kind_to_file_type = {"video": "video", "voice": "audio", "image": "image"}
@@ -140,11 +109,11 @@ def list_assets(
if not filter_tag_ids:
filter_tag_ids = None
# 需要内存过滤的标志(keyword/gender/style/tag_ids/smart_view/classification 无法在 DB 层过滤)
needs_memory_filter = bool(keyword or gender or style or filter_tag_ids or smart_view or classification)
# 需要内存过滤的标志(keyword/gender/style/tag_ids 无法在 DB 层过滤)
needs_memory_filter = bool(keyword or gender or style or filter_tag_ids)
def _apply_memory_filters(items):
"""应用 keyword / gender / style / tag_ids / smart_view / classification 内存过滤。"""
"""应用 keyword / gender / style / tag_ids 内存过滤。"""
result = items
if keyword:
kw = keyword.lower()
@@ -153,38 +122,9 @@ def list_assets(
result = [i for i in result if (i.metadata or {}).get("gender") == gender]
if style:
result = [i for i in result if (i.metadata or {}).get("style") == style]
if classification:
result = [i for i in result if (i.metadata or {}).get("classification") == classification]
if filter_tag_ids:
tag_set = set(filter_tag_ids)
result = [i for i in result if tag_set.issubset(set(getattr(i, "tag_ids", [])))]
if smart_view:
def __meta(a):
return a.metadata or {}
def __use_count(a):
return int(__meta(a).get("generation_use_count") or 0)
def __review_status(a):
return __meta(a).get("review_status", "")
if smart_view == "recommended":
result = [i for i in result if i.quality_score is not None and i.quality_score >= 80]
elif smart_view == "cautious":
result = [i for i in result if i.quality_score is not None and 60 <= i.quality_score < 80]
elif smart_view == "risky":
result = [
i
for i in result
if (i.quality_score is not None and i.quality_score < 60) or __review_status(i) == "rejected"
]
elif smart_view == "unused":
result = [i for i in result if __use_count(i) == 0]
elif smart_view == "used":
result = [i for i in result if __use_count(i) > 0]
elif smart_view == "pending_review":
result = [i for i in result if __review_status(i) == "pending_review"]
return result
# ── 优化路径:无内存过滤时,使用 DB 级分页 ──
@@ -196,15 +136,13 @@ def list_assets(
library = asset_library_repository.get(library_id)
if library is None:
raise HTTPException(status_code=404, detail=f"AssetLibrary {library_id} not found")
check_project_access(library.project_id, user_id, project_repository)
_check_project_access(library.project_id, user_id, project_repository)
if ft:
items = asset_repository.find_by_library_and_file_type(
library_id, ft, skip=skip, limit=limit, status=status_list
)
total = asset_repository.count_by_library_and_file_type(library_id, ft, status=status_list)
items = asset_repository.find_by_library_and_file_type(library_id, ft, skip=skip, limit=limit)
total = asset_repository.count_by_project(library.project_id) if not kind else len(items)
else:
items = asset_repository.find_by_library(library_id, skip=skip, limit=limit, status=status_list)
total = asset_repository.count_by_project(library.project_id, status=status_list)
items = asset_repository.find_by_library(library_id, skip=skip, limit=limit)
total = asset_repository.count_by_project(library.project_id)
return ListAssetsResponse(
items=[_to_asset_response(item) for item in items],
total=total,
@@ -214,16 +152,16 @@ def list_assets(
# 模式2:指定 project_id
if project_id:
check_project_access(project_id, user_id, project_repository)
_check_project_access(project_id, user_id, project_repository)
if ft:
items = asset_repository.find_by_project_and_file_type(
project_id, ft, skip=skip, limit=limit, status=status_list
)
total = asset_repository.count_by_project_and_file_type(project_id, ft, status=status_list)
paged = items
# 无直接方法,加载后按 file_type 过滤(仍比全量加载好)
all_items = asset_repository.find_by_project(project_id)
items = [i for i in all_items if i.mime_type and i.mime_type.startswith(ft)]
total = len(items)
paged = items[skip : skip + limit]
else:
items = asset_repository.find_by_project(project_id, skip=skip, limit=limit, status=status_list)
total = asset_repository.count_by_project(project_id, status=status_list)
items = asset_repository.find_by_project(project_id, skip=skip, limit=limit)
total = asset_repository.count_by_project(project_id)
paged = items
return ListAssetsResponse(
items=[_to_asset_response(item) for item in paged],
@@ -243,43 +181,22 @@ def list_assets(
if not project_ids:
return ListAssetsResponse(items=[], total=0, skip=skip, limit=limit)
if ft:
# 有 kind 过滤:逐项目查 file_type凑够一页
total = 0
paged_items: list = []
offset = skip
remaining = limit
for pid in project_ids:
proj_total = asset_repository.count_by_project_and_file_type(pid, ft, status=status_list)
total += proj_total
if offset >= proj_total:
offset -= proj_total
continue
proj_items = asset_repository.find_by_project_and_file_type(
pid, ft, skip=offset, limit=remaining, status=status_list
)
paged_items.extend(proj_items)
remaining -= len(proj_items)
offset = 0
if remaining <= 0:
break
else:
total = asset_repository.count_by_project_ids(project_ids, status=status_list)
# 跨项目分页:逐项目累积直到凑够一页
paged_items: list = []
offset = skip
remaining = limit
for pid in project_ids:
proj_total = asset_repository.count_by_project(pid, status=status_list)
if offset >= proj_total:
offset -= proj_total
continue
proj_items = asset_repository.find_by_project(pid, skip=offset, limit=remaining, status=status_list)
paged_items.extend(proj_items)
remaining -= len(proj_items)
offset = 0
if remaining <= 0:
break
total = asset_repository.count_by_project_ids(project_ids)
# 跨项目分页:逐项目累积直到凑够一页
paged_items: list = []
offset = skip
remaining = limit
for pid in project_ids:
proj_total = asset_repository.count_by_project(pid)
if offset >= proj_total:
offset -= proj_total
continue
proj_items = asset_repository.find_by_project(pid, skip=offset, limit=remaining)
paged_items.extend(proj_items)
remaining -= len(proj_items)
offset = 0
if remaining <= 0:
break
return ListAssetsResponse(
items=[_to_asset_response(item) for item in paged_items],
@@ -293,23 +210,14 @@ def list_assets(
library = asset_library_repository.get(library_id)
if library is None:
raise HTTPException(status_code=404, detail=f"AssetLibrary {library_id} not found")
check_project_access(library.project_id, user_id, project_repository)
_check_project_access(library.project_id, user_id, project_repository)
if kind:
all_items = asset_repository.find_by_library_and_file_type(
library_id, kind_to_file_type[kind], status=status_list
)
all_items = asset_repository.find_by_library_and_file_type(library_id, kind_to_file_type[kind])
else:
all_items = asset_repository.find_by_library(library_id, status=status_list)
all_items = asset_repository.find_by_library(library_id)
elif project_id:
check_project_access(project_id, user_id, project_repository)
if kind:
ft = kind_to_file_type.get(kind)
if ft:
all_items = asset_repository.find_by_project_and_file_type(project_id, ft, status=status_list)
else:
all_items = asset_repository.find_by_project(project_id, status=status_list)
else:
all_items = asset_repository.find_by_project(project_id, status=status_list)
_check_project_access(project_id, user_id, project_repository)
all_items = asset_repository.find_by_project(project_id)
else:
try:
projects = project_repository.find_accessible_projects(user_id)
@@ -318,18 +226,12 @@ def list_assets(
return ListAssetsResponse(items=[], total=0, skip=skip, limit=limit)
all_items = []
for proj in projects:
if kind and kind_to_file_type.get(kind):
all_items.extend(
asset_repository.find_by_project_and_file_type(proj.id, kind_to_file_type[kind], status=status_list)
)
else:
all_items.extend(asset_repository.find_by_project(proj.id, status=status_list))
all_items.extend(asset_repository.find_by_project(proj.id))
# 应用 kind 过滤(如果有)+ keyword/gender/style
if kind:
ft = kind_to_file_type.get(kind)
if ft:
all_items = [i for i in all_items if i.file_type == ft]
all_items = [i for i in all_items if i.mime_type and i.mime_type.startswith(ft or "")]
filtered = _apply_memory_filters(all_items)
total = len(filtered)
paged = filtered[skip : skip + limit]
@@ -360,163 +262,39 @@ def update_asset_review_status(
item = asset_repository.get(asset_id)
if item is None:
raise HTTPException(status_code=404, detail=f"Asset {asset_id} not found")
check_project_access(item.project_id, authenticated_user.user.id, project_repository)
_check_project_access(item.project_id, authenticated_user.user.id, project_repository)
_apply_asset_review_status(item, request.review_status)
updated = asset_repository.update(item)
return _to_asset_response(updated)
@router.post("/batch-delete", response_model=BatchOperationResponse)
@router.post("/batch-delete", response_model=BatchDeleteResponse)
def batch_delete_assets(
request: BatchDeleteRequest,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
asset_repository: Any = Depends(get_asset_repository),
project_repository: Any = Depends(get_project_repository),
) -> BatchOperationResponse:
"""批量删除素材(软删除,标记 status=deleted),需逐项校验项目权限。"""
) -> BatchDeleteResponse:
"""批量删除素材(配音素材等),需逐项校验项目权限。"""
user_id = authenticated_user.user.id
success_ids: list[str] = []
failed_details: dict[str, str] = {}
deleted_ids: list[str] = []
failed_ids: list[str] = []
for asset_id in request.asset_ids:
for asset_id in request.ids:
item = asset_repository.find_by_id(asset_id)
if item is None:
failed_details[asset_id] = "not_found"
failed_ids.append(asset_id)
continue
try:
check_project_access(item.project_id, user_id, project_repository)
success_ids.append(asset_id)
_check_project_access(item.project_id, user_id, project_repository)
deleted_ids.append(asset_id)
except HTTPException:
failed_details[asset_id] = "access_denied"
failed_ids.append(asset_id)
if success_ids:
asset_repository.batch_delete(success_ids)
if deleted_ids:
asset_repository.batch_delete(deleted_ids)
return BatchOperationResponse(
success_count=len(success_ids),
failed_ids=list(failed_details.keys()),
failed_details=failed_details,
)
@router.post("/batch-tag", response_model=BatchOperationResponse)
def batch_tag_assets(
request: BatchTagRequest,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
asset_repository: Any = Depends(get_asset_repository),
project_repository: Any = Depends(get_project_repository),
tag_repository: Any = Depends(get_tag_repository),
) -> BatchOperationResponse:
"""批量打标签(添加或替换模式),需逐项校验项目权限和标签权限。"""
user_id = authenticated_user.user.id
success_ids: list[str] = []
failed_details: dict[str, str] = {}
# 校验标签存在且属于当前用户
for tag_id in request.tag_ids:
tag = tag_repository.get(tag_id)
if tag is None:
return BatchOperationResponse(
success_count=0,
failed_ids=list(request.asset_ids),
failed_details={aid: f"tag_not_found:{tag_id}" for aid in request.asset_ids},
)
if tag.user_id != user_id:
return BatchOperationResponse(
success_count=0,
failed_ids=list(request.asset_ids),
failed_details={aid: f"tag_access_denied:{tag_id}" for aid in request.asset_ids},
)
# 校验素材权限
for asset_id in request.asset_ids:
item = asset_repository.find_by_id(asset_id)
if item is None:
failed_details[asset_id] = "not_found"
continue
try:
check_project_access(item.project_id, user_id, project_repository)
success_ids.append(asset_id)
except HTTPException:
failed_details[asset_id] = "access_denied"
if success_ids:
if request.mode == "replace":
asset_repository.batch_replace_tags(success_ids, request.tag_ids)
else:
asset_repository.batch_add_tags(success_ids, request.tag_ids)
return BatchOperationResponse(
success_count=len(success_ids),
failed_ids=list(failed_details.keys()),
failed_details=failed_details,
)
@router.post("/batch-classify", response_model=BatchOperationResponse)
def batch_classify_assets(
request: BatchClassifyRequest,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
asset_repository: Any = Depends(get_asset_repository),
project_repository: Any = Depends(get_project_repository),
) -> BatchOperationResponse:
"""批量修改素材内容分类(person/scenic/product等),存在metadata.category中。"""
user_id = authenticated_user.user.id
success_ids: list[str] = []
failed_details: dict[str, str] = {}
for asset_id in request.asset_ids:
item = asset_repository.find_by_id(asset_id)
if item is None:
failed_details[asset_id] = "not_found"
continue
try:
check_project_access(item.project_id, user_id, project_repository)
success_ids.append(asset_id)
except HTTPException:
failed_details[asset_id] = "access_denied"
if success_ids:
asset_repository.batch_update_metadata(success_ids, {"category": request.category})
return BatchOperationResponse(
success_count=len(success_ids),
failed_ids=list(failed_details.keys()),
failed_details=failed_details,
)
@router.post("/batch-mark", response_model=BatchOperationResponse)
def batch_mark_assets(
request: BatchMarkRequest,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
asset_repository: Any = Depends(get_asset_repository),
project_repository: Any = Depends(get_project_repository),
) -> BatchOperationResponse:
"""批量设置智能视图标记(recommended/caution/high_risk),存在metadata.smart_view中。"""
user_id = authenticated_user.user.id
success_ids: list[str] = []
failed_details: dict[str, str] = {}
for asset_id in request.asset_ids:
item = asset_repository.find_by_id(asset_id)
if item is None:
failed_details[asset_id] = "not_found"
continue
try:
check_project_access(item.project_id, user_id, project_repository)
success_ids.append(asset_id)
except HTTPException:
failed_details[asset_id] = "access_denied"
if success_ids:
asset_repository.batch_update_metadata(success_ids, {"smart_view": request.smart_view})
return BatchOperationResponse(
success_count=len(success_ids),
failed_ids=list(failed_details.keys()),
failed_details=failed_details,
)
return BatchDeleteResponse(deleted_count=len(deleted_ids), failed_ids=failed_ids)
@router.get("/{asset_id}", response_model=AssetResponse)
@@ -529,7 +307,7 @@ def get_asset(
item = asset_repository.find_by_id(asset_id)
if item is None:
raise HTTPException(status_code=404, detail=f"Asset {asset_id} not found")
check_project_access(item.project_id, authenticated_user.user.id, project_repository)
_check_project_access(item.project_id, authenticated_user.user.id, project_repository)
return _to_asset_response(item)
@@ -544,7 +322,7 @@ def update_asset(
item = asset_repository.find_by_id(asset_id)
if item is None:
raise HTTPException(status_code=404, detail=f"Asset {asset_id} not found")
check_project_access(item.project_id, authenticated_user.user.id, project_repository)
_check_project_access(item.project_id, authenticated_user.user.id, project_repository)
# 合并可修改字段
if request.name is not None:
@@ -558,7 +336,7 @@ def update_asset(
return _to_asset_response(updated)
@router.delete("/{asset_id}", status_code=204, response_class=Response)
@router.delete("/{asset_id}", status_code=204)
def delete_asset(
asset_id: str,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
@@ -568,7 +346,7 @@ def delete_asset(
item = asset_repository.find_by_id(asset_id)
if item is None:
raise HTTPException(status_code=404, detail=f"Asset {asset_id} not found")
check_project_access(item.project_id, authenticated_user.user.id, project_repository)
_check_project_access(item.project_id, authenticated_user.user.id, project_repository)
asset_repository.delete(asset_id)
@@ -585,7 +363,7 @@ def tag_asset(
item = asset_repository.find_by_id(asset_id)
if item is None:
raise HTTPException(status_code=404, detail=f"Asset {asset_id} not found")
check_project_access(item.project_id, authenticated_user.user.id, project_repository)
_check_project_access(item.project_id, authenticated_user.user.id, project_repository)
for tag_id in request.tag_ids:
tag = tag_repository.get(tag_id)
if tag is None:
@@ -597,7 +375,7 @@ def tag_asset(
return _to_asset_response(updated)
@router.delete("/{asset_id}/tags/{tag_id}", status_code=204, response_class=Response)
@router.delete("/{asset_id}/tags/{tag_id}", status_code=204)
def untag_asset(
asset_id: str,
tag_id: str,
@@ -609,7 +387,7 @@ def untag_asset(
item = asset_repository.find_by_id(asset_id)
if item is None:
raise HTTPException(status_code=404, detail=f"Asset {asset_id} not found")
check_project_access(item.project_id, authenticated_user.user.id, project_repository)
_check_project_access(item.project_id, authenticated_user.user.id, project_repository)
item.remove_tag(tag_id)
asset_repository.update(item)
+14 -13
View File
@@ -105,7 +105,7 @@ async def register(
request: RegisterRequest,
user_repository: UserRepository = Depends(get_user_repository),
email_service=Depends(get_auth_email_service),
) -> RegisterResponse:
):
use_case = RegisterUserUseCase(
user_repository=user_repository,
base_url=settings.APP_BASE_URL,
@@ -136,7 +136,7 @@ async def login(
request: LoginRequest,
user_repository: UserRepository = Depends(get_user_repository),
session_store=Depends(get_auth_session_store),
) -> LoginResponse:
):
use_case = LoginUseCase(
user_repository=user_repository,
session_store=session_store,
@@ -162,7 +162,7 @@ async def refresh(
request: RefreshRequest,
user_repository: UserRepository = Depends(get_user_repository),
session_store=Depends(get_auth_session_store),
) -> LoginResponse:
):
use_case = RefreshTokenUseCase(
user_repository=user_repository,
session_store=session_store,
@@ -194,7 +194,7 @@ def _verify_email_token(token: str, user_repository: UserRepository) -> MessageR
async def verify_email(
token: str,
user_repository: UserRepository = Depends(get_user_repository),
) -> MessageResponse:
):
return _verify_email_token(token, user_repository)
@@ -202,16 +202,16 @@ async def verify_email(
async def verify_email_post(
request: VerifyEmailRequestModel,
user_repository: UserRepository = Depends(get_user_repository),
) -> MessageResponse:
):
return _verify_email_token(request.token, user_repository)
@router.post("/forgot-password", response_model=MessageResponse, status_code=status.HTTP_202_ACCEPTED)
@router.post("/password/forgot", response_model=MessageResponse, status_code=status.HTTP_202_ACCEPTED)
async def forgot_password(
request: PasswordResetRequestModel,
user_repository: UserRepository = Depends(get_user_repository),
email_service=Depends(get_auth_email_service),
) -> MessageResponse:
):
success, error = RequestPasswordResetUseCase(
user_repository=user_repository,
base_url=settings.APP_BASE_URL,
@@ -223,11 +223,11 @@ async def forgot_password(
return MessageResponse(message="如果账户存在,密码重置邮件已发送")
@router.post("/reset-password", response_model=MessageResponse)
@router.post("/password/reset", response_model=MessageResponse)
async def reset_password(
request: ResetPasswordModel,
user_repository: UserRepository = Depends(get_user_repository),
) -> MessageResponse:
):
success, error = ResetPasswordUseCase(user_repository=user_repository).execute(
ResetPasswordRequest(token=request.token, new_password=request.new_password)
)
@@ -241,8 +241,9 @@ async def reset_password(
async def logout(
credentials: HTTPAuthorizationCredentials | None = Depends(bearer_scheme),
current_user: AuthenticatedUser = Depends(get_current_user),
) -> MessageResponse:
):
"""登出 - 将当前 token 加入黑名单"""
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
if credentials:
try:
@@ -257,7 +258,7 @@ async def logout(
@router.get("/me", response_model=CurrentUserResponse)
async def get_current_user_info(
authenticated_user: AuthenticatedUser = Depends(get_current_user),
) -> CurrentUserResponse:
):
user = authenticated_user.user
return CurrentUserResponse(
user_id=user.id,
@@ -323,7 +324,7 @@ def _get_internal_api_keys() -> list[str]:
if content:
return [k.strip() for k in content.split(",") if k.strip()]
except Exception:
logger.warning("无法读取内部 API 密钥文件,仅依赖环境变量配置", exc_info=True)
logger.debug("Failed to read internal API keys from file", exc_info=True)
return []
@@ -354,7 +355,7 @@ async def wechat_sync(
request: WechatSyncRequest,
user_repository: UserRepository = Depends(get_user_repository),
_: bool = Depends(_verify_internal_api_key),
) -> WechatSyncResponse:
):
"""
微信同步登录/注册(系统级内部接口)
+92 -75
View File
@@ -13,8 +13,8 @@ from pathlib import Path
from typing import Any
from uuid import uuid4
from app.api.routes._helpers import require_project_and_library
from app.auth import AuthenticatedUser, get_current_user
from app.config import get_settings
from app.core.celery_app import celery_app
from app.core.storage import OSSStorageService, get_storage_service
from app.dependencies import (
@@ -113,6 +113,22 @@ def _atomic_check_and_record(upload_id: str, chunk_index: int) -> bool:
fcntl.flock(f.fileno(), fcntl.LOCK_UN)
def _require_project_and_library(
project_id: str,
library_id: str,
project_repository: Any,
asset_library_repository: Any,
) -> None:
"""Verify project and asset library exist"""
project = GetProjectUseCase(project_repository).execute(project_id)
if project is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Project not found")
libraries = asset_library_repository.find_by_project(project_id)
if not any(item.id == library_id for item in libraries):
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Asset library not found")
def _load_upload_meta(upload_id: str) -> dict[str, Any]:
"""Load upload metadata"""
meta_path = _get_upload_meta_path(upload_id)
@@ -190,6 +206,7 @@ async def init_chunked_upload(
asset_library_repository: Any = Depends(get_asset_library_repository),
) -> ChunkedUploadInitResponse:
"""Initialize chunked upload"""
settings = get_settings()
# Validate file size
if request.file_size > MAX_FILE_SIZE:
@@ -204,7 +221,7 @@ async def init_chunked_upload(
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Project not found")
# Verify asset library
require_project_and_library(
_require_project_and_library(
request.project_id,
request.library_id,
project_repository,
@@ -257,6 +274,79 @@ async def init_chunked_upload(
)
@router.post("/{upload_id}/{chunk_index}")
async def upload_chunk(
upload_id: str,
chunk_index: int,
chunk: UploadFile = File(..., description="Chunk data"),
authenticated_user: AuthenticatedUser = Depends(get_current_user),
) -> dict[str, Any]:
"""Upload a single chunk"""
# Load metadata
meta = _load_upload_meta(upload_id)
# Check expiry
expires_at = datetime.fromisoformat(meta["expires_at"])
if expires_at.tzinfo is None:
expires_at = expires_at.replace(tzinfo=timezone.utc)
if expires_at < datetime.now(timezone.utc):
raise HTTPException(status_code=status.HTTP_410_GONE, detail="Upload has expired")
# Validate chunk index
if chunk_index < 0 or chunk_index >= meta["total_chunks"]:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Invalid chunk index. Must be between 0 and {meta['total_chunks'] - 1}",
)
# Atomic check and record to prevent race conditions
if not _atomic_check_and_record(upload_id, chunk_index):
return {"message": "Chunk already uploaded", "chunk_index": chunk_index}
# Read chunk data
chunk_data = await chunk.read()
# Validate chunk size (last chunk can be smaller than chunk_size)
expected_size = DEFAULT_CHUNK_SIZE
if chunk_index == meta["total_chunks"] - 1:
expected_size = meta["file_size"] - (chunk_index * DEFAULT_CHUNK_SIZE)
if len(chunk_data) != expected_size:
# Rollback the recorded chunk
meta_path = _get_upload_meta_path(upload_id)
with open(meta_path, "r+", encoding="utf-8") as f:
fcntl.flock(f.fileno(), fcntl.LOCK_EX)
try:
meta = json.load(f)
if chunk_index in meta["uploaded_chunks"]:
meta["uploaded_chunks"].remove(chunk_index)
f.seek(0)
json.dump(meta, f, ensure_ascii=False, indent=2)
f.truncate()
finally:
fcntl.flock(f.fileno(), fcntl.LOCK_UN)
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Chunk size mismatch. Expected {expected_size}, got {len(chunk_data)}",
)
# Save chunk
chunk_path = _get_chunk_dir(upload_id) / f"chunk_{chunk_index:06d}"
with open(chunk_path, "wb") as f:
f.write(chunk_data)
# Reload metadata for response
meta = _load_upload_meta(upload_id)
return {
"message": "Chunk uploaded successfully",
"chunk_index": chunk_index,
"uploaded_chunks": len(meta["uploaded_chunks"]),
"total_chunks": meta["total_chunks"],
}
@router.get("/{upload_id}/status", response_model=ChunkedUploadStatusResponse)
async def get_upload_status(
upload_id: str,
@@ -403,76 +493,3 @@ async def complete_chunked_upload(
meta_path = _get_upload_meta_path(upload_id)
if meta_path.exists():
meta_path.unlink()
@router.post("/{upload_id}/{chunk_index}")
async def upload_chunk(
upload_id: str,
chunk_index: int,
chunk: UploadFile = File(..., description="Chunk data"),
authenticated_user: AuthenticatedUser = Depends(get_current_user),
) -> dict[str, Any]:
"""Upload a single chunk"""
# Load metadata
meta = _load_upload_meta(upload_id)
# Check expiry
expires_at = datetime.fromisoformat(meta["expires_at"])
if expires_at.tzinfo is None:
expires_at = expires_at.replace(tzinfo=timezone.utc)
if expires_at < datetime.now(timezone.utc):
raise HTTPException(status_code=status.HTTP_410_GONE, detail="Upload has expired")
# Validate chunk index
if chunk_index < 0 or chunk_index >= meta["total_chunks"]:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Invalid chunk index. Must be between 0 and {meta['total_chunks'] - 1}",
)
# Atomic check and record to prevent race conditions
if not _atomic_check_and_record(upload_id, chunk_index):
return {"message": "Chunk already uploaded", "chunk_index": chunk_index}
# Read chunk data
chunk_data = await chunk.read()
# Validate chunk size (last chunk can be smaller than chunk_size)
expected_size = DEFAULT_CHUNK_SIZE
if chunk_index == meta["total_chunks"] - 1:
expected_size = meta["file_size"] - (chunk_index * DEFAULT_CHUNK_SIZE)
if len(chunk_data) != expected_size:
# Rollback the recorded chunk
meta_path = _get_upload_meta_path(upload_id)
with open(meta_path, "r+", encoding="utf-8") as f:
fcntl.flock(f.fileno(), fcntl.LOCK_EX)
try:
meta = json.load(f)
if chunk_index in meta["uploaded_chunks"]:
meta["uploaded_chunks"].remove(chunk_index)
f.seek(0)
json.dump(meta, f, ensure_ascii=False, indent=2)
f.truncate()
finally:
fcntl.flock(f.fileno(), fcntl.LOCK_UN)
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Chunk size mismatch. Expected {expected_size}, got {len(chunk_data)}",
)
# Save chunk
chunk_path = _get_chunk_dir(upload_id) / f"chunk_{chunk_index:06d}"
with open(chunk_path, "wb") as f:
f.write(chunk_data)
# Reload metadata for response
meta = _load_upload_meta(upload_id)
return {
"message": "Chunk uploaded successfully",
"chunk_index": chunk_index,
"uploaded_chunks": len(meta["uploaded_chunks"]),
"total_chunks": meta["total_chunks"],
}
+92
View File
@@ -0,0 +1,92 @@
from typing import Any
from app.auth import AuthenticatedUser, get_current_user
from app.dependencies import (
get_asset_repository,
get_generation_task_repository,
get_project_repository,
get_title_library_repository,
get_voice_library_repository,
)
from app.schemas.dashboard import DashboardOverviewResponse, RecentTaskItem, SubscriptionInfo
from fastapi import APIRouter, Depends
router = APIRouter()
def _status_value(status) -> str:
return status.value if hasattr(status, "value") else str(status)
def _generation_step(status: str) -> str:
if status == "pending":
return "等待 Worker 执行"
if status == "running":
return "正在生成成片"
if status == "completed":
return "生成完成"
if status == "failed":
return "生成失败"
return status
@router.get("/overview", response_model=DashboardOverviewResponse)
def get_dashboard_overview(
authenticated_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
asset_repository: Any = Depends(get_asset_repository),
generation_task_repository: Any = Depends(get_generation_task_repository),
title_library_repository: Any = Depends(get_title_library_repository),
voice_library_repository: Any = Depends(get_voice_library_repository),
) -> DashboardOverviewResponse:
"""Dashboard 概览:用户级汇总数据。"""
user_id = authenticated_user.user.id
# 获取用户可访问的所有 project
projects = project_repository.find_accessible_projects(user_id)
project_ids = [p.id for p in projects]
# 素材统计
total_assets = asset_repository.count_by_project_ids(project_ids)
used_storage_bytes = asset_repository.sum_storage_by_project_ids(project_ids)
# 标题库 / 配音库统计
total_titles = title_library_repository.count_by_user(user_id)
total_voices = voice_library_repository.count_by_user(user_id)
# 生成任务统计
total_tasks = generation_task_repository.count_by_user(user_id)
# 最近任务(SQL 层 LIMIT 5
recent = generation_task_repository.list_recent_by_user(user_id, limit=5)
recent_tasks = []
for task in recent:
s = _status_value(task.status)
recent_tasks.append(
RecentTaskItem(
id=task.id,
task_type="generation",
status=s,
current_step=_generation_step(s),
error_message=task.error_message or "",
updated_at=task.completed_at or task.started_at or task.created_at,
)
)
# 订阅信息
user = authenticated_user.user
subscription = SubscriptionInfo(
plan=getattr(user, "subscription_plan", "free") or "free",
is_active=getattr(user, "subscription_status", "") == "active",
)
return DashboardOverviewResponse(
total_assets=total_assets,
used_storage_bytes=used_storage_bytes,
total_titles=total_titles,
total_voices=total_voices,
total_tasks=total_tasks,
total_products=len(projects),
subscription=subscription,
recent_tasks=recent_tasks,
)
+3 -5
View File
@@ -239,9 +239,7 @@ def get_duplication_detail(
return _to_detail_response(record)
@router.delete(
"/records/{record_id}", status_code=status.HTTP_204_NO_CONTENT, response_model=None, response_class=Response
)
@router.delete("/records/{record_id}", status_code=status.HTTP_204_NO_CONTENT, response_model=None)
def delete_duplication_record(
record_id: str,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
@@ -259,7 +257,7 @@ def delete_duplication_record(
use_case = DeleteDuplicationRecordUseCase(duplication_repository)
use_case.execute(record_id)
return
return Response(status_code=204)
@router.post("/records/{record_id}/retry", response_model=DuplicationUploadResponse)
@@ -289,7 +287,7 @@ def retry_duplication(
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=str(e),
) from e
)
if updated is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
File diff suppressed because it is too large Load Diff
@@ -1,311 +0,0 @@
"""片段调整 API.
- PUT /clips/{clip_id}/speed 调速
- PUT /clips/{clip_id}/volume 音量调节
- PUT /clips/{clip_id}/trim 裁剪(trim in/out
- PUT /clips/{clip_id}/adjustments 统一调整(speed+volume+trim
- POST /{plan_id}/clips/batch-speed 批量调速
"""
from __future__ import annotations
import logging
from typing import Any, Optional
from app.auth import AuthenticatedUser, get_current_user
from app.dependencies import get_db_session, get_project_repository
from app.services import EditPlanService
from fastapi import APIRouter, Depends, HTTPException, status
from pydantic import BaseModel, Field
from sqlalchemy.orm import Session
from ._helpers import check_project_access
logger = logging.getLogger(__name__)
router = APIRouter()
# ── Schemas ──────────────────────────────────────────────────────────────────
class SpeedAdjustRequest(BaseModel):
"""调速请求"""
speed: float = Field(..., ge=0.25, le=4.0, description="播放速度 0.25~4.0")
class VolumeAdjustRequest(BaseModel):
"""音量调节请求"""
volume: float = Field(..., ge=0.0, le=2.0, description="音量倍率 0~2.01.0=原音量)")
class TrimAdjustRequest(BaseModel):
"""裁剪请求"""
trim_start: float = Field(0.0, ge=0.0, description="开头裁剪秒数")
trim_end: float = Field(0.0, ge=0.0, description="结尾裁剪秒数")
class ClipAdjustmentsRequest(BaseModel):
"""统一调整请求"""
speed: Optional[float] = Field(default=None, ge=0.25, le=4.0)
volume: Optional[float] = Field(default=None, ge=0.0, le=2.0)
trim_start: Optional[float] = Field(default=None, ge=0.0)
trim_end: Optional[float] = Field(default=None, ge=0.0)
class BatchSpeedRequest(BaseModel):
"""批量调速请求"""
speed: float = Field(..., ge=0.25, le=4.0, description="播放速度")
class ClipAdjustResponse(BaseModel):
"""片段调整响应"""
clip_id: str
speed: float
volume: float
trim_start: float
trim_end: float
duration: float
class BatchSpeedResponse(BaseModel):
"""批量调速响应"""
updated_count: int
plan_id: str
# ── Helpers ──────────────────────────────────────────────────────────────────
def _get_clip_config(clip) -> dict:
config = getattr(clip, "config", {}) or {}
if not isinstance(config, dict):
config = {}
return config
def _get_volume(clip) -> float:
config = _get_clip_config(clip)
return float(config.get("volume", 1.0))
def _get_trim(clip) -> tuple[float, float]:
config = _get_clip_config(clip)
trim_start = float(config.get("trim_start", 0.0))
trim_end = float(config.get("trim_end", 0.0))
return trim_start, trim_end
def _build_response(clip) -> ClipAdjustResponse:
trim_start, trim_end = _get_trim(clip)
return ClipAdjustResponse(
clip_id=clip.id,
speed=clip.playback_speed,
volume=_get_volume(clip),
trim_start=trim_start,
trim_end=trim_end,
duration=clip.duration,
)
def _validate_trim(trim_start: float, trim_end: float, total_duration: float) -> None:
"""验证裁剪时长不超过总时长"""
if trim_start + trim_end >= total_duration:
raise ValueError(f"裁剪总时长({trim_start + trim_end:.2f}s)不能大于等于片段总时长({total_duration:.2f}s")
# ── Routes ───────────────────────────────────────────────────────────────────
def _get_svc_and_plan_and_clip(db, clip_id, current_user, project_repository):
svc = EditPlanService(db)
clip = svc.get_clip(clip_id)
if not clip:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"片段不存在: {clip_id}",
)
plan = svc.get_plan(clip.plan_id)
if plan and plan.project_id:
check_project_access(plan.project_id, current_user.user.id, project_repository)
return svc, plan, clip
@router.put("/clips/{clip_id}/speed", response_model=ClipAdjustResponse)
def adjust_speed(
clip_id: str,
body: SpeedAdjustRequest,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> ClipAdjustResponse:
"""调整片段播放速度"""
svc, _, clip = _get_svc_and_plan_and_clip(db, clip_id, current_user, project_repository)
updated = svc.update_clip(clip_id, playback_speed=body.speed)
logger.info(
"调整片段速度: clip_id=%s speed=%.2f by user=%s",
clip_id,
body.speed,
current_user.user.id,
)
return _build_response(updated)
@router.put("/clips/{clip_id}/volume", response_model=ClipAdjustResponse)
def adjust_volume(
clip_id: str,
body: VolumeAdjustRequest,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> ClipAdjustResponse:
"""调整片段音量"""
svc, _, clip = _get_svc_and_plan_and_clip(db, clip_id, current_user, project_repository)
# 更新 config.volume
config = dict(_get_clip_config(clip))
config["volume"] = body.volume
updated = svc.update_clip(clip_id, config=config)
logger.info(
"调整片段音量: clip_id=%s volume=%.2f by user=%s",
clip_id,
body.volume,
current_user.user.id,
)
return _build_response(updated)
@router.put("/clips/{clip_id}/trim", response_model=ClipAdjustResponse)
def adjust_trim(
clip_id: str,
body: TrimAdjustRequest,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> ClipAdjustResponse:
"""裁剪片段(trim in/out"""
svc, _, clip = _get_svc_and_plan_and_clip(db, clip_id, current_user, project_repository)
# 验证裁剪时长
try:
_validate_trim(body.trim_start, body.trim_end, clip.duration)
except ValueError as e:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(e)) from e
# 更新 config
config = dict(_get_clip_config(clip))
config["trim_start"] = body.trim_start
config["trim_end"] = body.trim_end
updated = svc.update_clip(clip_id, config=config)
logger.info(
"裁剪片段: clip_id=%s trim_start=%.2f trim_end=%.2f by user=%s",
clip_id,
body.trim_start,
body.trim_end,
current_user.user.id,
)
return _build_response(updated)
@router.put("/clips/{clip_id}/adjustments", response_model=ClipAdjustResponse)
def adjust_all(
clip_id: str,
body: ClipAdjustmentsRequest,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> ClipAdjustResponse:
"""统一调整片段的 speed / volume / trim"""
svc, _, clip = _get_svc_and_plan_and_clip(db, clip_id, current_user, project_repository)
update_kwargs = {}
config_updates = {}
if body.speed is not None:
update_kwargs["playback_speed"] = body.speed
if body.volume is not None:
config_updates["volume"] = body.volume
if body.trim_start is not None:
config_updates["trim_start"] = body.trim_start
if body.trim_end is not None:
config_updates["trim_end"] = body.trim_end
# 验证 trim
current_trim_start, current_trim_end = _get_trim(clip)
new_trim_start = body.trim_start if body.trim_start is not None else current_trim_start
new_trim_end = body.trim_end if body.trim_end is not None else current_trim_end
if body.trim_start is not None or body.trim_end is not None:
try:
_validate_trim(new_trim_start, new_trim_end, clip.duration)
except ValueError as e:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(e)) from e
if config_updates:
config = dict(_get_clip_config(clip))
config.update(config_updates)
update_kwargs["config"] = config
if not update_kwargs:
return _build_response(clip)
updated = svc.update_clip(clip_id, **update_kwargs)
logger.info(
"统一调整片段: clip_id=%s speed=%s volume=%s by user=%s",
clip_id,
body.speed,
body.volume,
current_user.user.id,
)
return _build_response(updated)
@router.post("/{plan_id}/clips/batch-speed", response_model=BatchSpeedResponse)
def batch_adjust_speed(
plan_id: str,
body: BatchSpeedRequest,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> BatchSpeedResponse:
"""批量调整计划内所有片段的播放速度"""
svc = EditPlanService(db)
plan = svc.get_plan(plan_id)
if not plan:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"剪辑计划不存在: {plan_id}",
)
if plan.project_id:
check_project_access(plan.project_id, current_user.user.id, project_repository)
clips = svc.list_clips(plan_id, limit=500, skip=0)
count = 0
for clip in clips:
svc.update_clip(clip.id, playback_speed=body.speed)
count += 1
logger.info(
"批量调速: plan_id=%s count=%d speed=%.2f by user=%s",
plan_id,
count,
body.speed,
current_user.user.id,
)
return BatchSpeedResponse(updated_count=count, plan_id=plan_id)
-199
View File
@@ -1,199 +0,0 @@
"""剪辑计划 AI 推荐 & 封面生成 API 端点。
从 edit_plans.py 拆分,包含:
- POST /{plan_id}/ai-recommend AI 推荐片段方案
- POST /{plan_id}/generate-cover AI 生成封面
"""
from __future__ import annotations
import logging
from typing import Any
from app.api.routes._helpers import check_project_access
from app.api.routes.edit_plans import (
AIRecommendClipItem,
AIRecommendRequest,
AIRecommendResponse,
GenerateCoverRequest,
GenerateCoverResponse,
)
from app.auth import AuthenticatedUser, get_current_user
from app.dependencies import get_db_session, get_project_repository
from app.services import EditPlanService
from fastapi import APIRouter, Depends, HTTPException, status
from sqlalchemy.orm import Session
from packages.domain.config_schemas import normalize_plan_config
logger = logging.getLogger(__name__)
router = APIRouter()
@router.post(
"/{plan_id}/ai-recommend",
response_model=AIRecommendResponse,
)
def ai_recommend_clips(
plan_id: str,
body: AIRecommendRequest,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> AIRecommendResponse:
"""AI 推荐片段方案
调用 AI 服务分析素材,自动生成片段编排方案并写入剪辑计划。
流程:
1. 验证计划存在且状态为 draft/editing
2. 调用 AI 推荐服务(当前为 stub,后续接入真实 AI)
3. 清除计划现有片段,按推荐方案重新创建
4. 更新计划 configcover/title/subtitle/bgm)和 total_duration
5. 返回推荐方案详情
"""
svc = EditPlanService(db)
try:
plan = svc.get_plan_or_raise(plan_id)
except ValueError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc
if plan.project_id:
check_project_access(plan.project_id, current_user.user.id, project_repository)
plan_status = plan.status.value if hasattr(plan.status, "value") else plan.status
if plan_status not in ("draft", "editing"):
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="当前计划状态不支持AI推荐,请先创建或编辑计划后再试",
)
from apps.worker.worker_app.tasks.ai_tasks import run_ai_recommend
result = run_ai_recommend(
plan_id=plan_id,
template_id=plan.template_id,
asset_ids=body.asset_ids,
editing_mode=body.editing_mode,
target_duration=body.target_duration,
)
# 事务保护:清除 → 重建 → 更新 必须在同一逻辑事务中
try:
svc.delete_all_clips(plan_id)
for clip_data in result["clips"]:
svc.create_clip(
plan_id=plan_id,
clip_type=clip_data["clip_type"],
order=clip_data["order"],
text_content=clip_data.get("text_content", ""),
duration=clip_data["duration"],
transition_effect=clip_data.get("transition_effect", "cut"),
asset_id=clip_data.get("asset_id", ""),
start_time=clip_data.get("start_time", 0.0),
config=clip_data.get("config", {}),
)
normalized_config = normalize_plan_config(result.get("config", {}))
svc.update_plan(
plan_id,
config=normalized_config,
total_duration=result["total_duration"],
)
except Exception as _e:
logger.exception("AI 推荐写入失败,plan_id=%s 数据可能不一致", plan_id)
try:
db.rollback()
except Exception as rollback_err:
logger.error(
"AI 推荐回滚失败,数据库会话可能处于不一致状态: plan_id=%s error=%s",
plan_id,
rollback_err,
)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="AI推荐结果保存失败,请稍后重试",
) from _e
logger.info(
"AI 推荐片段方案: plan_id=%s clips=%d duration=%.1f by user=%s",
plan_id,
len(result["clips"]),
result["total_duration"],
current_user.user.id,
)
return AIRecommendResponse(
plan_id=plan_id,
clips=[
AIRecommendClipItem(
clip_type=c["clip_type"],
order=c["order"],
text_content=c.get("text_content", ""),
duration=c["duration"],
transition_effect=c.get("transition_effect", "cut"),
asset_id=c.get("asset_id", ""),
start_time=c.get("start_time", 0.0),
config=c.get("config", {}),
)
for c in result["clips"]
],
config=normalized_config,
total_duration=result["total_duration"],
confidence=result["confidence"],
)
@router.post(
"/{plan_id}/generate-cover",
response_model=GenerateCoverResponse,
)
def generate_cover(
plan_id: str,
body: GenerateCoverRequest,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> GenerateCoverResponse:
"""AI 生成封面
调用 AI 服务从视频中选帧或生成封面图,并更新计划 config.cover。
"""
svc = EditPlanService(db)
try:
plan = svc.get_plan_or_raise(plan_id)
except ValueError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc
if plan.project_id:
check_project_access(plan.project_id, current_user.user.id, project_repository)
from apps.worker.worker_app.tasks.ai_tasks import run_generate_cover
cover_data = run_generate_cover(
plan_id=plan_id,
asset_ids=body.asset_ids,
cover_type=body.cover_type,
frame_time=body.frame_time,
)
current_config = dict(plan.config)
current_config["cover"] = cover_data
normalized = normalize_plan_config(current_config)
svc.update_plan_config(plan_id, {"cover": normalized["cover"]})
logger.info(
"AI 封面生成: plan_id=%s type=%s by user=%s",
plan_id,
body.cover_type,
current_user.user.id,
)
return GenerateCoverResponse(
plan_id=plan_id,
cover=cover_data,
)
-415
View File
@@ -1,415 +0,0 @@
"""剪辑计划片段(Clip)CRUD 路由。"""
from __future__ import annotations
import logging
from typing import Any, List, Optional
from app.auth import AuthenticatedUser, get_current_user
from app.dependencies import get_db_session, get_project_repository
from fastapi import APIRouter, Depends, HTTPException, Query, Response, status
from sqlalchemy.orm import Session
from packages.domain.edit_plan_clip import EditPlanClipStatus
logger = logging.getLogger(__name__)
router = APIRouter()
# ── Schemas ──────────────────────────────────────────────────────────────────
from pydantic import BaseModel, Field
class EditPlanClipResponse(BaseModel):
"""剪辑片段响应体"""
id: str
plan_id: str
clip_type: str
order: int
asset_id: str = ""
text_content: str = ""
start_time: float = 0.0
duration: float = 0.0
transition_effect: str = "cut"
transition_duration: float = 0.0
playback_speed: float = 1.0
status: str
config: dict[str, Any] = Field(default_factory=dict)
created_at: Optional[str] = None
updated_at: Optional[str] = None
class EditPlanClipListResponse(BaseModel):
"""剪辑片段列表响应体"""
items: List[EditPlanClipResponse]
total: int
class EditPlanClipCreateRequest(BaseModel):
"""创建剪辑片段请求体"""
clip_type: str = Field(
..., min_length=1, max_length=50, description="片段类型: main/intro/outro/overlay/background/b_roll 等"
)
order: int = Field(..., ge=0, description="排序序号")
asset_id: str = Field(default="", max_length=64, description="关联素材 ID")
text_content: str = Field(default="", max_length=5000, description="文本内容(字幕/配音等)")
start_time: float = Field(default=0.0, ge=0.0, description="起始时间 (秒)")
duration: float = Field(default=0.0, ge=0.0, description="时长 (秒)")
transition_effect: str = Field(default="cut", max_length=50, description="转场效果")
transition_duration: float = Field(default=0.0, ge=0.0, description="转场时长 (秒)")
playback_speed: float = Field(default=1.0, gt=0.0, le=10.0, description="播放速度倍率")
config: dict[str, Any] = Field(default_factory=dict, description="扩展配置 (JSON)")
class EditPlanClipUpdateRequest(BaseModel):
"""更新剪辑片段请求体"""
clip_type: Optional[str] = Field(default=None, min_length=1, max_length=50, description="片段类型")
order: Optional[int] = Field(default=None, ge=0, description="排序序号")
asset_id: Optional[str] = Field(default=None, max_length=64, description="关联素材 ID")
text_content: Optional[str] = Field(default=None, max_length=5000, description="文本内容")
start_time: Optional[float] = Field(default=None, ge=0.0, description="起始时间 (秒)")
duration: Optional[float] = Field(default=None, ge=0.0, description="时长 (秒)")
transition_effect: Optional[str] = Field(default=None, max_length=50, description="转场效果")
transition_duration: Optional[float] = Field(default=None, ge=0.0, description="转场时长 (秒)")
playback_speed: Optional[float] = Field(default=None, gt=0.0, le=10.0, description="播放速度倍率")
config: Optional[dict[str, Any]] = Field(default=None, description="扩展配置 (JSON)")
# ── Helpers ──────────────────────────────────────────────────────────────────
def _check_plan_access(plan_id: str, user_id: str, project_repository: Any, db: Session) -> Any:
"""验证用户是否有权限访问该剪辑计划(通过项目关联)。
返回 plan 对象供后续使用,避免重复查询。
"""
from app.services.edit_plan_service import EditPlanService
from ._helpers import check_project_access
svc = EditPlanService(db)
plan = svc.get_plan(plan_id)
if plan is None:
raise HTTPException(status_code=404, detail=f"剪辑计划不存在: {plan_id}")
if plan.project_id:
check_project_access(plan.project_id, user_id, project_repository)
return plan
def _clip_to_response(clip) -> EditPlanClipResponse:
"""将领域对象转换为响应体"""
return EditPlanClipResponse(
id=clip.id,
plan_id=clip.plan_id,
clip_type=clip.clip_type,
order=clip.order,
asset_id=clip.asset_id or "",
text_content=clip.text_content or "",
start_time=clip.start_time,
duration=clip.duration,
transition_effect=clip.transition_effect or "cut",
transition_duration=clip.transition_duration or 0.0,
playback_speed=clip.playback_speed or 1.0,
status=clip.status.value if hasattr(clip.status, "value") else str(clip.status),
config=clip.config or {},
created_at=clip.created_at.isoformat() if clip.created_at else None,
updated_at=clip.updated_at.isoformat() if clip.updated_at else None,
)
def _get_svc(db: Session):
"""获取 EditPlanService 实例"""
from app.services.edit_plan_service import EditPlanService
return EditPlanService(db)
# ── Routes ───────────────────────────────────────────────────────────────────
@router.get("", response_model=EditPlanClipListResponse)
def list_clips(
plan_id: str,
status_filter: Optional[str] = Query(None, alias="status", description="按状态过滤"),
skip: int = Query(0, ge=0, description="分页偏移"),
limit: int = Query(100, ge=1, le=500, description="每页数量"),
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> EditPlanClipListResponse:
"""获取剪辑计划的片段列表"""
_check_plan_access(plan_id, current_user.user.id, project_repository, db)
svc = _get_svc(db)
status_enum = EditPlanClipStatus(status_filter) if status_filter else None
clips = svc.list_clips(plan_id, status=status_enum, skip=skip, limit=limit)
total = svc.count_clips(plan_id, status=status_enum)
return EditPlanClipListResponse(
items=[_clip_to_response(c) for c in clips],
total=total,
)
@router.post("", response_model=EditPlanClipResponse, status_code=status.HTTP_201_CREATED)
def create_clip(
plan_id: str,
body: EditPlanClipCreateRequest,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> EditPlanClipResponse:
"""创建剪辑片段"""
_check_plan_access(plan_id, current_user.user.id, project_repository, db)
svc = _get_svc(db)
try:
clip = svc.create_clip(
plan_id=plan_id,
clip_type=body.clip_type,
order=body.order,
asset_id=body.asset_id,
text_content=body.text_content,
start_time=body.start_time,
duration=body.duration,
transition_effect=body.transition_effect,
transition_duration=body.transition_duration,
playback_speed=body.playback_speed,
config=body.config,
)
except ValueError as e:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(e)) from e
logger.info("创建剪辑片段: plan_id=%s clip_id=%s by user=%s", plan_id, clip.id, current_user.user.id)
return _clip_to_response(clip)
@router.get("/{clip_id}", response_model=EditPlanClipResponse)
def get_clip(
plan_id: str,
clip_id: str,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> EditPlanClipResponse:
"""获取剪辑片段详情"""
_check_plan_access(plan_id, current_user.user.id, project_repository, db)
svc = _get_svc(db)
clip = svc.get_clip(clip_id)
if clip is None:
raise HTTPException(status_code=404, detail=f"片段不存在: {clip_id}")
if clip.plan_id != plan_id:
raise HTTPException(status_code=404, detail=f"片段不存在: {clip_id}")
return _clip_to_response(clip)
@router.put("/{clip_id}", response_model=EditPlanClipResponse)
def update_clip(
plan_id: str,
clip_id: str,
body: EditPlanClipUpdateRequest,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> EditPlanClipResponse:
"""更新剪辑片段"""
_check_plan_access(plan_id, current_user.user.id, project_repository, db)
svc = _get_svc(db)
# 验证 clip 属于该 plan
clip = svc.get_clip(clip_id)
if clip is None:
raise HTTPException(status_code=404, detail=f"片段不存在: {clip_id}")
if clip.plan_id != plan_id:
raise HTTPException(status_code=404, detail=f"片段不存在: {clip_id}")
try:
updated = svc.update_clip(
clip_id,
clip_type=body.clip_type,
order=body.order,
asset_id=body.asset_id,
text_content=body.text_content,
start_time=body.start_time,
duration=body.duration,
transition_effect=body.transition_effect,
transition_duration=body.transition_duration,
playback_speed=body.playback_speed,
config=body.config,
)
except ValueError as e:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(e)) from e
logger.info("更新剪辑片段: plan_id=%s clip_id=%s by user=%s", plan_id, clip_id, current_user.user.id)
return _clip_to_response(updated)
@router.delete("/{clip_id}", status_code=status.HTTP_204_NO_CONTENT, response_model=None, response_class=Response)
def delete_clip(
plan_id: str,
clip_id: str,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> None:
"""删除剪辑片段"""
_check_plan_access(plan_id, current_user.user.id, project_repository, db)
svc = _get_svc(db)
# 验证 clip 属于该 plan
clip = svc.get_clip(clip_id)
if clip is None:
raise HTTPException(status_code=404, detail=f"片段不存在: {clip_id}")
if clip.plan_id != plan_id:
raise HTTPException(status_code=404, detail=f"片段不存在: {clip_id}")
deleted = svc.delete_clip(clip_id)
if not deleted:
raise HTTPException(status_code=404, detail=f"片段不存在: {clip_id}")
logger.info("删除剪辑片段: plan_id=%s clip_id=%s by user=%s", plan_id, clip_id, current_user.user.id)
return None
# ── 片段分割与合并 ──────────────────────────────────────────────────────────
class SplitClipRequest(BaseModel):
"""分割片段请求体"""
split_time: float = Field(..., gt=0, description="分割点(秒,相对于片段起始)")
class MergeClipsRequest(BaseModel):
"""合并片段请求体"""
clip_ids: list[str] = Field(..., min_length=2, description="要合并的片段 ID 列表")
@router.post(
"/{clip_id}/split",
response_model=dict[str, Any],
summary="分割片段",
status_code=status.HTTP_200_OK,
)
def split_clip(
plan_id: str,
clip_id: str,
body: SplitClipRequest,
current_user: AuthenticatedUser = Depends(get_current_user),
db: Session = Depends(get_db_session),
project_repository: Any = Depends(get_project_repository),
) -> dict[str, Any]:
"""将一个片段从指定时间点分割为两个片段。
分割后原片段变为左半部分,新增右半部分片段,后续片段顺序自动后移。
若片段有关联素材,会自动设置 trim_start/trim_end 标记裁剪范围。
"""
_check_plan_access(plan_id, current_user.user.id, project_repository, db)
svc = _get_svc(db)
clip = svc.get_clip(clip_id)
if clip is None or clip.plan_id != plan_id:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"片段不存在: {clip_id}",
)
try:
result = svc.split_clip(clip_id, body.split_time)
except ValueError as e:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=str(e),
) from e
left = result["left_clip"]
right = result["right_clip"]
logger.info("分割片段: plan_id=%s clip_id=%s by user=%s", plan_id, clip_id, current_user.user.id)
return {
"left_clip": {
"id": left.id,
"plan_id": left.plan_id,
"clip_type": left.clip_type,
"order": left.order,
"duration": left.duration,
"start_time": left.start_time,
},
"right_clip": {
"id": right.id,
"plan_id": right.plan_id,
"clip_type": right.clip_type,
"order": right.order,
"duration": right.duration,
"start_time": right.start_time,
},
}
@router.post(
"/merge",
response_model=dict[str, Any],
summary="合并多个连续片段",
status_code=status.HTTP_200_OK,
)
def merge_clips(
plan_id: str,
body: MergeClipsRequest,
current_user: AuthenticatedUser = Depends(get_current_user),
db: Session = Depends(get_db_session),
project_repository: Any = Depends(get_project_repository),
) -> dict[str, Any]:
"""将多个连续的同类型片段合并为一个片段。
合并要求:
- 至少 2 个片段
- 属于同一剪辑计划
- order 连续
- 类型相同
合并后保留第一个片段,其余删除,后续片段顺序自动前移。
"""
_check_plan_access(plan_id, current_user.user.id, project_repository, db)
svc = _get_svc(db)
# 校验所有片段都属于该 plan
for cid in body.clip_ids:
clip = svc.get_clip(cid)
if clip is None or clip.plan_id != plan_id:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"片段不存在: {cid}",
)
try:
merged = svc.merge_clips(body.clip_ids)
except ValueError as e:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=str(e),
) from e
logger.info(
"合并片段: plan_id=%s clip_count=%d by user=%s",
plan_id,
len(body.clip_ids),
current_user.user.id,
)
return {
"id": merged.id,
"plan_id": merged.plan_id,
"clip_type": merged.clip_type,
"order": merged.order,
"duration": merged.duration,
"text_content": merged.text_content,
}
@@ -1,241 +0,0 @@
"""剪辑计划片段批量操作 API。"""
from __future__ import annotations
import logging
from typing import Any, List
from app.auth import AuthenticatedUser, get_current_user
from app.dependencies import get_db_session, get_project_repository
from fastapi import APIRouter, Depends, HTTPException, status
from pydantic import BaseModel, Field
from sqlalchemy.orm import Session
logger = logging.getLogger(__name__)
router = APIRouter()
# ── Schemas ──────────────────────────────────────────────────────────────────
class ClipReorderItem(BaseModel):
"""重排序条目"""
clip_id: str
new_order: int = Field(..., ge=0, description="新的排序序号")
class ClipReorderRequest(BaseModel):
"""片段重排序请求"""
items: List[ClipReorderItem] = Field(..., min_length=1, max_length=500, description="重排序条目列表")
class ClipReorderResponse(BaseModel):
"""片段重排序响应"""
success: bool
updated_count: int
message: str = ""
class ClipBatchDeleteRequest(BaseModel):
"""批量删除片段请求"""
clip_ids: List[str] = Field(..., min_length=1, max_length=500, description="要删除的片段ID列表")
class ClipBatchDeleteResponse(BaseModel):
"""批量删除片段响应"""
success: bool
deleted_count: int
message: str = ""
class ClipsFromAssetsRequest(BaseModel):
"""从素材批量创建片段请求"""
asset_ids: List[str] = Field(..., min_length=1, max_length=200, description="素材 ID 列表,按顺序追加到时间线末尾")
clip_type: str = Field(default="main", description="片段类型,默认 main")
class ClipsFromAssetsResponse(BaseModel):
"""从素材批量创建片段响应"""
success: bool
created_count: int
message: str = ""
clip_ids: List[str] = Field(default_factory=list, description="创建的片段ID列表")
# ── Helpers ──────────────────────────────────────────────────────────────────
def _check_plan_access(plan_id: str, user_id: str, project_repository: Any, db: Session) -> Any:
"""验证用户是否有权限访问该剪辑计划,返回 plan 对象。"""
from app.services.edit_plan_service import EditPlanService
from ._helpers import check_project_access
svc = EditPlanService(db)
plan = svc.get_plan(plan_id)
if plan is None:
raise HTTPException(status_code=404, detail=f"剪辑计划不存在: {plan_id}")
if plan.project_id:
check_project_access(plan.project_id, user_id, project_repository)
return plan
def _get_svc(db: Session):
"""获取 EditPlanService 实例"""
from app.services.edit_plan_service import EditPlanService
return EditPlanService(db)
# ── Routes ───────────────────────────────────────────────────────────────────
@router.post("/reorder", response_model=ClipReorderResponse)
def reorder_clips(
plan_id: str,
body: ClipReorderRequest,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> ClipReorderResponse:
"""批量重排序片段
前端拖拽调整顺序后,一次性提交所有变更的 order。
自动触发编辑状态回退(从 completed/failed 切回 editing)。
"""
_check_plan_access(plan_id, current_user.user.id, project_repository, db)
svc = _get_svc(db)
# 验证所有 clip 都属于该 plan
clip_ids = [item.clip_id for item in body.items]
existing_clips = svc.list_clips(plan_id, skip=0, limit=10000)
existing_ids = {c.id for c in existing_clips}
invalid_ids = [cid for cid in clip_ids if cid not in existing_ids]
if invalid_ids:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"以下片段不属于该计划: {', '.join(invalid_ids[:5])}",
)
# 执行重排序
updated_count = 0
for item in body.items:
try:
svc.update_clip(item.clip_id, order=item.new_order)
updated_count += 1
except ValueError as e:
logger.warning("重排序片段失败: clip_id=%s error=%s", item.clip_id, e)
logger.info(
"批量重排序片段: plan_id=%s count=%d by user=%s",
plan_id,
updated_count,
current_user.user.id,
)
return ClipReorderResponse(
success=True,
updated_count=updated_count,
message=f"成功更新 {updated_count} 个片段的顺序",
)
@router.post("/batch-delete", response_model=ClipBatchDeleteResponse)
def batch_delete_clips(
plan_id: str,
body: ClipBatchDeleteRequest,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> ClipBatchDeleteResponse:
"""批量删除片段
自动触发编辑状态回退(从 completed/failed 切回 editing)。
"""
_check_plan_access(plan_id, current_user.user.id, project_repository, db)
svc = _get_svc(db)
# 验证所有 clip 都属于该 plan
existing_clips = svc.list_clips(plan_id, skip=0, limit=10000)
existing_ids = {c.id for c in existing_clips}
valid_ids = [cid for cid in body.clip_ids if cid in existing_ids]
skipped = len(body.clip_ids) - len(valid_ids)
# 执行删除
deleted_count = 0
for clip_id in valid_ids:
if svc.delete_clip(clip_id):
deleted_count += 1
message = f"成功删除 {deleted_count} 个片段"
if skipped > 0:
message += f",跳过 {skipped} 个不存在的片段"
logger.info(
"批量删除片段: plan_id=%s deleted=%d skipped=%d by user=%s",
plan_id,
deleted_count,
skipped,
current_user.user.id,
)
return ClipBatchDeleteResponse(
success=True,
deleted_count=deleted_count,
message=message,
)
@router.post("/from-assets", response_model=ClipsFromAssetsResponse)
def create_clips_from_assets(
plan_id: str,
body: ClipsFromAssetsRequest,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> ClipsFromAssetsResponse:
"""从素材批量创建片段(追加到时间线末尾)
一次性将多个素材作为片段添加到剪辑计划,自动读取素材时长。
自动触发编辑状态回退(completed/failed → editing)。
"""
_check_plan_access(plan_id, current_user.user.id, project_repository, db)
svc = _get_svc(db)
try:
clips = svc.create_clips_from_assets(
plan_id=plan_id,
asset_ids=body.asset_ids,
clip_type=body.clip_type,
)
except ValueError as e:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(e)) from e
clip_ids = [c.id for c in clips]
logger.info(
"从素材批量创建片段: plan_id=%s count=%d by user=%s",
plan_id,
len(clips),
current_user.user.id,
)
return ClipsFromAssetsResponse(
success=True,
created_count=len(clips),
message=f"成功创建 {len(clips)} 个片段",
clip_ids=clip_ids,
)
-315
View File
@@ -1,315 +0,0 @@
"""封面管理 API.
- GET /{plan_id}/cover 获取封面配置
- PUT /{plan_id}/cover 更新封面配置
- POST /{plan_id}/cover/extract 从指定片段抽帧生成封面
- POST /{plan_id}/cover/smart 智能选帧生成封面
"""
from __future__ import annotations
import logging
from typing import Any, Optional
from app.auth import AuthenticatedUser, get_current_user
from app.core.storage import get_storage_service
from app.dependencies import (
get_asset_repository,
get_db_session,
get_project_repository,
)
from app.services import EditPlanService
from app.services.cover_service import CoverService
from fastapi import APIRouter, Depends, HTTPException, status
from pydantic import BaseModel, Field
from sqlalchemy.orm import Session
from packages.domain.config_schemas import normalize_plan_config
from ._helpers import check_project_access
logger = logging.getLogger(__name__)
router = APIRouter()
# ── Schemas ──────────────────────────────────────────────────────────────────
class CoverConfigResponse(BaseModel):
"""封面配置响应"""
type: str = Field(..., description="封面类型: ai_frame / manual / upload")
image_url: str = Field(default="", description="封面图片 URL")
frame_time: Optional[float] = Field(default=None, description="抽帧时间点(秒)")
class CoverUpdateRequest(BaseModel):
"""更新封面配置请求"""
type: Optional[str] = Field(default=None, description="封面类型")
image_url: Optional[str] = Field(default=None, description="封面图片 URL")
frame_time: Optional[float] = Field(default=None, ge=0.0, description="抽帧时间点(秒)")
class CoverExtractRequest(BaseModel):
"""从片段抽帧生成封面请求"""
clip_id: str = Field(..., description="片段 ID")
frame_time: float = Field(1.0, ge=0.0, description="抽帧时间点(秒)")
class CoverSmartRequest(BaseModel):
"""智能选帧请求"""
clip_id: Optional[str] = Field(default=None, description="指定片段 ID(不传则用第一个视频片段)")
class CoverGenerateResponse(BaseModel):
"""封面生成响应"""
type: str = Field(..., description="封面类型")
image_url: str = Field(..., description="封面图片 URL")
frame_time: Optional[float] = Field(default=None, description="抽帧时间点(秒)")
# ── Routes ───────────────────────────────────────────────────────────────────
@router.get("/{plan_id}/cover", response_model=CoverConfigResponse)
def get_cover(
plan_id: str,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> CoverConfigResponse:
"""获取封面配置"""
svc = EditPlanService(db)
plan = svc.get_plan(plan_id)
if not plan:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"剪辑计划不存在: {plan_id}",
)
if plan.project_id:
check_project_access(plan.project_id, current_user.user.id, project_repository)
cover = CoverService.get_cover_config(plan.config or {})
return CoverConfigResponse(**cover)
@router.put("/{plan_id}/cover", response_model=CoverConfigResponse)
def update_cover(
plan_id: str,
body: CoverUpdateRequest,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> CoverConfigResponse:
"""更新封面配置
用于:设置上传的封面图片 URL、切换封面类型、调整时间点等。
"""
svc = EditPlanService(db)
plan = svc.get_plan(plan_id)
if not plan:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"剪辑计划不存在: {plan_id}",
)
if plan.project_id:
check_project_access(plan.project_id, current_user.user.id, project_repository)
# 合并更新
current_cover = CoverService.get_cover_config(plan.config or {})
updates = body.model_dump(exclude_none=True)
new_cover = {**current_cover, **updates}
# 验证 type 值
valid_types = {"ai_frame", "manual", "upload", "ai_regenerate"}
if "type" in updates and updates["type"] not in valid_types:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"无效的封面类型: {updates['type']},有效值: {valid_types}",
)
# 更新到 plan.config
current_config = dict(plan.config or {})
current_config["cover"] = new_cover
normalized = normalize_plan_config(current_config)
updated_plan = svc.update_plan_config(plan_id, {"cover": normalized["cover"]})
result = CoverService.get_cover_config(updated_plan.config or {})
logger.info("更新封面配置: plan_id=%s type=%s by user=%s", plan_id, result["type"], current_user.user.id)
return CoverConfigResponse(**result)
@router.post("/{plan_id}/cover/extract", response_model=CoverGenerateResponse)
def extract_cover(
plan_id: str,
body: CoverExtractRequest,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
storage_service: Any = Depends(get_storage_service),
asset_repository: Any = Depends(get_asset_repository),
) -> CoverGenerateResponse:
"""从指定片段的指定时间点抽帧生成封面"""
svc = EditPlanService(db)
plan = svc.get_plan(plan_id)
if not plan:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"剪辑计划不存在: {plan_id}",
)
if plan.project_id:
check_project_access(plan.project_id, current_user.user.id, project_repository)
# 获取片段对应的素材
clip = svc.get_clip(body.clip_id)
if not clip:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"片段不存在: {body.clip_id}",
)
if clip.plan_id != plan_id:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="片段不属于该剪辑计划",
)
if not clip.asset_id:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="片段没有关联素材,无法抽帧",
)
# 抽帧生成封面
cover_svc = CoverService(storage_service, asset_repository)
try:
cover_data = cover_svc.extract_cover_from_clip(
plan_id=plan_id,
asset_id=clip.asset_id,
frame_time=body.frame_time,
)
except ValueError as e:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(e)) from e
except RuntimeError as e:
logger.error("封面抽帧失败: plan_id=%s clip_id=%s error=%s", plan_id, body.clip_id, e)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"封面抽帧失败: {e}",
) from e
# 更新到 plan.config.cover
current_config = dict(plan.config or {})
current_config["cover"] = cover_data
normalized = normalize_plan_config(current_config)
svc.update_plan_config(plan_id, {"cover": normalized["cover"]})
logger.info(
"封面抽帧完成: plan_id=%s clip_id=%s time=%.2fs by user=%s",
plan_id,
body.clip_id,
body.frame_time,
current_user.user.id,
)
return CoverGenerateResponse(**cover_data)
@router.post("/{plan_id}/cover/smart", response_model=CoverGenerateResponse)
def smart_cover(
plan_id: str,
body: CoverSmartRequest,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
storage_service: Any = Depends(get_storage_service),
asset_repository: Any = Depends(get_asset_repository),
) -> CoverGenerateResponse:
"""智能选帧生成封面
从指定片段(或第一个视频片段)中智能选取一帧作为封面。
当前实现:取片段第3秒帧(后续可优化为多帧选最清晰)。
"""
svc = EditPlanService(db)
plan = svc.get_plan(plan_id)
if not plan:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"剪辑计划不存在: {plan_id}",
)
if plan.project_id:
check_project_access(plan.project_id, current_user.user.id, project_repository)
# 确定使用哪个片段
clip_id = body.clip_id
asset_id = ""
if clip_id:
clip = svc.get_clip(clip_id)
if not clip:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"片段不存在: {clip_id}",
)
if clip.plan_id != plan_id:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="片段不属于该剪辑计划",
)
if not clip.asset_id:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="片段没有关联素材",
)
asset_id = clip.asset_id
else:
# 找第一个有素材的视频片段
clips = svc.list_clips(plan_id, limit=50, skip=0)
for c in clips:
if c.asset_id and c.clip_type == "video":
asset_id = c.asset_id
clip_id = c.id
break
if not asset_id:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="没有找到可用的视频片段",
)
# 智能选帧
cover_svc = CoverService(storage_service, asset_repository)
try:
cover_data = cover_svc.generate_smart_cover(
plan_id=plan_id,
asset_id=asset_id,
)
except ValueError as e:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(e)) from e
except RuntimeError as e:
logger.error("智能封面生成失败: plan_id=%s error=%s", plan_id, e)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail=f"智能封面生成失败: {e}",
) from e
# 更新到 plan.config.cover
current_config = dict(plan.config or {})
current_config["cover"] = cover_data
normalized = normalize_plan_config(current_config)
svc.update_plan_config(plan_id, {"cover": normalized["cover"]})
logger.info(
"智能封面生成完成: plan_id=%s clip_id=%s by user=%s",
plan_id,
clip_id,
current_user.user.id,
)
return CoverGenerateResponse(**cover_data)
@@ -1,274 +0,0 @@
"""导出设置 API.
- GET /{plan_id}/export 获取导出配置
- PUT /{plan_id}/export 更新导出配置
- GET /export-presets 导出预设列表
"""
from __future__ import annotations
import logging
import re
from typing import Any, List, Optional
from app.auth import AuthenticatedUser, get_current_user
from app.dependencies import get_db_session, get_project_repository
from app.services import EditPlanService
from fastapi import APIRouter, Depends, HTTPException, status
from pydantic import BaseModel, Field, validator
from sqlalchemy.orm import Session
from packages.domain.config_schemas import normalize_plan_config
from ._helpers import check_project_access
logger = logging.getLogger(__name__)
router = APIRouter()
# ── 导出预设 ──────────────────────────────────────────────────────────────────
EXPORT_PRESETS = [
{
"id": "export_1080p_30",
"name": "1080P 高清",
"resolution": "1080x1920",
"fps": 30,
"video_bitrate": 8000,
"audio_bitrate": 128,
"format": "mp4",
"quality_preset": "balanced",
"description": "竖屏高清,适合短视频平台",
"size_hint": "约 10MB/分钟",
},
{
"id": "export_1080p_60",
"name": "1080P 高帧率",
"resolution": "1080x1920",
"fps": 60,
"video_bitrate": 12000,
"audio_bitrate": 128,
"format": "mp4",
"quality_preset": "high",
"description": "60帧高帧率,流畅运动画面",
"size_hint": "约 18MB/分钟",
},
{
"id": "export_720p_30",
"name": "720P 流畅",
"resolution": "720x1280",
"fps": 30,
"video_bitrate": 4000,
"audio_bitrate": 128,
"format": "mp4",
"quality_preset": "fast",
"description": "快速导出,文件较小",
"size_hint": "约 5MB/分钟",
},
{
"id": "export_4k_30",
"name": "4K 超清",
"resolution": "2160x3840",
"fps": 30,
"video_bitrate": 20000,
"audio_bitrate": 192,
"format": "mp4",
"quality_preset": "best",
"description": "4K超清画质,专业品质",
"size_hint": "约 30MB/分钟",
},
{
"id": "export_1080p_30_mov",
"name": "1080P ProRes",
"resolution": "1080x1920",
"fps": 30,
"video_bitrate": 15000,
"audio_bitrate": 256,
"format": "mov",
"quality_preset": "high",
"description": "MOV格式,适合后期剪辑",
"size_hint": "约 25MB/分钟",
},
]
VALID_QUALITY_PRESETS = {"ultra_fast", "fast", "balanced", "high", "best"}
VALID_FORMATS = {"mp4", "mov"}
RESOLUTION_PATTERN = re.compile(r"^\d+x\d+$")
# ── Schemas ──────────────────────────────────────────────────────────────────
class ExportConfigResponse(BaseModel):
"""导出配置响应"""
resolution: str
fps: int
video_bitrate: int
audio_bitrate: int
format: str
quality_preset: str
watermark_enabled: bool
watermark_text: str
class ExportUpdateRequest(BaseModel):
"""更新导出配置请求"""
resolution: Optional[str] = None
fps: Optional[int] = Field(default=None, ge=15, le=60)
video_bitrate: Optional[int] = Field(default=None, ge=1000, le=20000)
audio_bitrate: Optional[int] = Field(default=None, ge=64, le=320)
format: Optional[str] = None
quality_preset: Optional[str] = None
watermark_enabled: Optional[bool] = None
watermark_text: Optional[str] = None
@validator("resolution")
def validate_resolution(cls, v):
if v is None:
return v
if not RESOLUTION_PATTERN.match(v):
raise ValueError("分辨率格式错误,应为 宽x高,如 1080x1920")
w, h = v.split("x")
if int(w) < 100 or int(h) < 100:
raise ValueError("分辨率数值过小")
if int(w) > 4096 or int(h) > 4096:
raise ValueError("分辨率数值过大,最大 4096x4096")
return v
@validator("format")
def validate_format(cls, v):
if v is None:
return v
if v not in VALID_FORMATS:
raise ValueError(f"无效格式: {v},支持: {VALID_FORMATS}")
return v
@validator("quality_preset")
def validate_quality_preset(cls, v):
if v is None:
return v
if v not in VALID_QUALITY_PRESETS:
raise ValueError(f"无效质量预设: {v},支持: {VALID_QUALITY_PRESETS}")
return v
class ExportPresetItem(BaseModel):
"""导出预设条目"""
id: str
name: str
resolution: str
fps: int
video_bitrate: int
audio_bitrate: int
format: str
quality_preset: str
description: str
size_hint: str
class ExportPresetListResponse(BaseModel):
"""导出预设列表响应"""
items: List[ExportPresetItem]
total: int
# ── Helpers ──────────────────────────────────────────────────────────────────
def _get_export_config(plan_config: dict) -> dict:
e = plan_config.get("export", {})
if not isinstance(e, dict):
e = {}
return {
"resolution": e.get("resolution", "1080x1920"),
"fps": e.get("fps", 30),
"video_bitrate": e.get("video_bitrate", 8000),
"audio_bitrate": e.get("audio_bitrate", 128),
"format": e.get("format", "mp4"),
"quality_preset": e.get("quality_preset", "balanced"),
"watermark_enabled": e.get("watermark_enabled", False),
"watermark_text": e.get("watermark_text", ""),
}
# ── Routes ───────────────────────────────────────────────────────────────────
@router.get("/export-presets", response_model=ExportPresetListResponse)
def list_export_presets(
current_user: AuthenticatedUser = Depends(get_current_user),
) -> ExportPresetListResponse:
"""获取导出预设列表"""
items = [ExportPresetItem(**p) for p in EXPORT_PRESETS]
return ExportPresetListResponse(items=items, total=len(items))
@router.get("/{plan_id}/export", response_model=ExportConfigResponse)
def get_export_config(
plan_id: str,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> ExportConfigResponse:
"""获取导出配置"""
svc = EditPlanService(db)
plan = svc.get_plan(plan_id)
if not plan:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"剪辑计划不存在: {plan_id}",
)
if plan.project_id:
check_project_access(plan.project_id, current_user.user.id, project_repository)
config = _get_export_config(plan.config or {})
return ExportConfigResponse(**config)
@router.put("/{plan_id}/export", response_model=ExportConfigResponse)
def update_export_config(
plan_id: str,
body: ExportUpdateRequest,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> ExportConfigResponse:
"""更新导出配置"""
svc = EditPlanService(db)
plan = svc.get_plan(plan_id)
if not plan:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"剪辑计划不存在: {plan_id}",
)
if plan.project_id:
check_project_access(plan.project_id, current_user.user.id, project_repository)
# 合并更新
current = _get_export_config(plan.config or {})
updates = body.model_dump(exclude_none=True)
new_export = {**current, **updates}
# 更新到 plan.config
current_config = dict(plan.config or {})
current_config["export"] = new_export
normalized = normalize_plan_config(current_config)
updated_plan = svc.update_plan_config(plan_id, {"export": normalized["export"]})
result = _get_export_config(updated_plan.config or {})
logger.info(
"更新导出配置: plan_id=%s resolution=%s fps=%d by user=%s",
plan_id,
result["resolution"],
result["fps"],
current_user.user.id,
)
return ExportConfigResponse(**result)
@@ -1,197 +0,0 @@
"""滤镜调色 API.
- GET /filter-presets 滤镜预设列表
- GET /{plan_id}/filter 获取全局滤镜配置
- PUT /{plan_id}/filter 更新全局滤镜配置
"""
from __future__ import annotations
import logging
from typing import Any, List, Optional
from app.auth import AuthenticatedUser, get_current_user
from app.dependencies import get_db_session, get_project_repository
from app.services import EditPlanService
from fastapi import APIRouter, Depends, HTTPException, Query, status
from pydantic import BaseModel, Field
from sqlalchemy.orm import Session
from packages.domain.config_schemas import normalize_plan_config
from packages.domain.filter_presets import (
FilterPreset,
get_filter_preset,
list_filter_presets,
)
from ._helpers import check_project_access
logger = logging.getLogger(__name__)
router = APIRouter()
# ── Schemas ──────────────────────────────────────────────────────────────────
class FilterPresetResponse(BaseModel):
"""滤镜预设响应"""
id: str
name: str
category: str
description: str
tags: List[str] = Field(default_factory=list)
class FilterConfigResponse(BaseModel):
"""滤镜配置响应"""
enabled: bool
preset_id: str
intensity: int
brightness: float
contrast: float
saturation: float
warmth: float
class FilterUpdateRequest(BaseModel):
"""更新滤镜配置请求"""
enabled: Optional[bool] = None
preset_id: Optional[str] = None
intensity: Optional[int] = Field(default=None, ge=0, le=100)
brightness: Optional[float] = Field(default=None, ge=-1.0, le=1.0)
contrast: Optional[float] = Field(default=None, ge=0.0, le=2.0)
saturation: Optional[float] = Field(default=None, ge=0.0, le=3.0)
warmth: Optional[float] = Field(default=None, ge=-1.0, le=1.0)
class FilterPresetListResponse(BaseModel):
"""滤镜预设列表响应"""
items: List[FilterPresetResponse]
total: int
# ── Helpers ──────────────────────────────────────────────────────────────────
def _preset_to_response(p: FilterPreset) -> FilterPresetResponse:
return FilterPresetResponse(
id=p.id,
name=p.name,
category=p.category,
description=p.description,
tags=list(p.tags),
)
def _get_filter_config(plan_config: dict) -> dict:
"""从 plan.config 中提取滤镜配置"""
f = plan_config.get("filter", {})
if not isinstance(f, dict):
f = {}
return {
"enabled": f.get("enabled", False),
"preset_id": f.get("preset_id", "filter_none"),
"intensity": f.get("intensity", 100),
"brightness": f.get("brightness", 0.0),
"contrast": f.get("contrast", 1.0),
"saturation": f.get("saturation", 1.0),
"warmth": f.get("warmth", 0.0),
}
# ── Routes ───────────────────────────────────────────────────────────────────
@router.get("/filter-presets", response_model=FilterPresetListResponse)
def list_presets(
category: Optional[str] = Query(default=None, description="按分类筛选"),
keyword: Optional[str] = Query(default=None, description="关键词搜索"),
current_user: AuthenticatedUser = Depends(get_current_user),
) -> FilterPresetListResponse:
"""获取滤镜预设列表"""
presets = list_filter_presets(category=category, keyword=keyword)
items = [_preset_to_response(p) for p in presets]
return FilterPresetListResponse(items=items, total=len(items))
@router.get("/{plan_id}/filter", response_model=FilterConfigResponse)
def get_filter(
plan_id: str,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> FilterConfigResponse:
"""获取剪辑计划的全局滤镜配置"""
svc = EditPlanService(db)
plan = svc.get_plan(plan_id)
if not plan:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"剪辑计划不存在: {plan_id}",
)
if plan.project_id:
check_project_access(plan.project_id, current_user.user.id, project_repository)
config = _get_filter_config(plan.config or {})
return FilterConfigResponse(**config)
@router.put("/{plan_id}/filter", response_model=FilterConfigResponse)
def update_filter(
plan_id: str,
body: FilterUpdateRequest,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> FilterConfigResponse:
"""更新全局滤镜配置"""
svc = EditPlanService(db)
plan = svc.get_plan(plan_id)
if not plan:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"剪辑计划不存在: {plan_id}",
)
if plan.project_id:
check_project_access(plan.project_id, current_user.user.id, project_repository)
# 验证 preset_id
updates = body.model_dump(exclude_none=True)
if "preset_id" in updates:
preset = get_filter_preset(updates["preset_id"])
if preset is None:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"无效的滤镜预设: {updates['preset_id']}",
)
# 合并更新
current = _get_filter_config(plan.config or {})
new_filter = {**current, **updates}
# 如果设为原图 preset,自动关闭
if new_filter["preset_id"] == "filter_none":
new_filter["enabled"] = False
# 更新到 plan.config
current_config = dict(plan.config or {})
current_config["filter"] = new_filter
normalized = normalize_plan_config(current_config)
updated_plan = svc.update_plan_config(plan_id, {"filter": normalized["filter"]})
result = _get_filter_config(updated_plan.config or {})
logger.info(
"更新滤镜配置: plan_id=%s preset=%s intensity=%d by user=%s",
plan_id,
result["preset_id"],
result["intensity"],
current_user.user.id,
)
return FilterConfigResponse(**result)
@@ -1,412 +0,0 @@
"""剪辑计划生成相关 API 端点。
从 edit_plans.py 拆分,包含:
- POST /{plan_id}/generate 触发剪辑渲染生成
- GET /{plan_id}/generation-status 查询生成进度
- GET /{plan_id}/generations 查询关联的生成记录
"""
from __future__ import annotations
import logging
from typing import Any
from app.api.routes._helpers import check_project_access
from app.api.routes.edit_plans import (
ClipStatusItem,
EditPlanGenerateResponse,
EditPlanGenerationsResponse,
EditPlanGenerationStatusResponse,
)
from app.auth import AuthenticatedUser, get_current_user
from app.core.celery_app import celery_app
from app.core.storage import OSSStorageService, get_storage_service
from app.core.task_enqueue import GLOBAL_PENDING_LIMIT, USER_PENDING_LIMIT
from app.dependencies import get_asset_library_repository, get_asset_repository, get_db_session, get_project_repository
from app.services import EditPlanService
from fastapi import APIRouter, Depends, HTTPException, status
from sqlalchemy.orm import Session
from packages.adapters.sqlalchemy_impl.generation_task_repository import (
SQLAlchemyGenerationTaskRepository,
)
from packages.adapters.sqlalchemy_impl.template_clip_config_repository import (
SQLAlchemyTemplateClipConfigRepository,
)
from packages.adapters.sqlalchemy_impl.template_repository import (
SQLAlchemyTemplateRepository,
)
from packages.application.generation_tasks import (
CreateGenerationTaskCommand,
CreateGenerationTaskUseCase,
)
from packages.domain.edit_plan import EditPlanStatus
logger = logging.getLogger(__name__)
router = APIRouter()
def _auto_fallback_draft_to_editing(svc: EditPlanService, plan_id: str, plan_check) -> None:
"""自动兜底 1: draft → editing"""
if plan_check.status == EditPlanStatus.DRAFT:
logger.info("自动兜底: plan=%s draft→editing", plan_id)
svc.transition_status(plan_id, EditPlanStatus.EDITING)
def _auto_fallback_copy_template_clips(svc: EditPlanService, plan_id: str, plan_check, db: Session) -> None:
"""自动兜底 2: 无片段 + 有 template_id → 从模板复制片段配置"""
existing_clips = svc.count_clips(plan_id)
if existing_clips == 0 and plan_check.template_id:
logger.info(
"自动兜底: plan=%s 无片段,从模板 %s 复制片段配置",
plan_id,
plan_check.template_id,
)
clip_config_repo = SQLAlchemyTemplateClipConfigRepository(db)
configs = clip_config_repo.list_by_template(plan_check.template_id)
if configs:
for cfg in configs:
svc.create_clip(
plan_id=plan_id,
clip_type=cfg.clip_type.value if hasattr(cfg.clip_type, "value") else cfg.clip_type,
order=cfg.order,
template_clip_config_id=cfg.id,
duration=cfg.default_duration,
transition_effect=(
cfg.transition_effect.value
if hasattr(cfg.transition_effect, "value")
else cfg.transition_effect
),
)
logger.info("自动兜底: plan=%s 从新模型 template_clip_configs 复制了 %d 个片段", plan_id, len(configs))
else:
tpl_repo = SQLAlchemyTemplateRepository(db)
segments = tpl_repo.list_segments(plan_check.template_id)
for seg in segments:
avg_duration = (seg.duration_min + seg.duration_max) / 2
svc.create_clip(
plan_id=plan_id,
clip_type="main",
order=seg.segment_order,
duration=avg_duration,
config={
"material_type": seg.material_type or "",
"template_segment_id": seg.id,
},
)
logger.info("自动兜底: plan=%s 从旧模型 template_segments 复制了 %d 个片段", plan_id, len(segments))
def _auto_fallback_assign_assets(
svc: EditPlanService,
plan_id: str,
plan_check,
) -> list:
"""自动兜底 3: 为没有素材的片段分配素材。返回剩余无素材片段列表。"""
all_clips = svc.list_clips(plan_id)
clips_without_asset = [c for c in all_clips if not c.asset_id]
config_asset_ids = (plan_check.config or {}).get("asset_ids", [])
if clips_without_asset and config_asset_ids:
logger.info(
"自动兜底3: plan=%s%d 个无素材片段分配 %d 个指定素材",
plan_id,
len(clips_without_asset),
len(config_asset_ids),
)
for i, clip in enumerate(clips_without_asset):
asset_idx = i % len(config_asset_ids)
svc.assign_asset(clip.id, config_asset_ids[asset_idx])
logger.info("自动兜底3: plan=%s 素材分配完成", plan_id)
clips_without_asset = []
return clips_without_asset
def _auto_fallback_auto_material_mode(
svc: EditPlanService,
plan_id: str,
plan_check,
clips_without_asset: list,
asset_library_repo: Any,
asset_repo: Any,
) -> None:
"""自动兜底 4: 项目有视频素材库时,自动选取 ready 视频素材分配给无素材片段
注:原先需要 material_mode=="auto" 才触发,但全代码库没有任何地方设置为 auto,
导致这道兜底防线永远不生效。现改为:只要有 project_id 且存在无素材片段,
就自动从项目视频素材库选取素材兜底,确保一键生成等场景能正常出片。
"""
if not clips_without_asset:
return
if not plan_check.project_id:
return
import random
logger.info(
"自动兜底4: plan=%s 自动选素材分配给 %d 个无素材片段",
plan_id,
len(clips_without_asset),
)
libs = asset_library_repo.find_by_project(plan_check.project_id)
video_lib = None
for lib in libs:
lib_kind = lib.kind.value if hasattr(lib.kind, "value") else lib.kind
if lib_kind == "video":
video_lib = lib
break
if video_lib:
assets = asset_repo.find_by_library(video_lib.id)
ready_videos = [
a
for a in assets
if (a.status.value if hasattr(a.status, "value") else a.status) == "ready"
and a.mime_type
and a.mime_type.startswith("video")
]
if ready_videos:
random.shuffle(ready_videos)
for i, clip in enumerate(clips_without_asset):
asset = ready_videos[i % len(ready_videos)]
svc.assign_asset(clip.id, asset.id)
logger.info(
"自动兜底4: plan=%s 从素材库 %s 分配了 %d 个素材给 %d 个片段",
plan_id,
video_lib.name,
len(ready_videos),
len(clips_without_asset),
)
else:
logger.warning("自动兜底4: plan=%s 素材库无可用视频素材", plan_id)
else:
logger.warning("自动兜底4: plan=%s 项目无视频素材库", plan_id)
def _check_queue_limits(gen_task_repo, user_id: str) -> None:
"""队列限流预检查"""
try:
has_count = hasattr(gen_task_repo, "count_pending_by_user") and hasattr(gen_task_repo, "count_pending_total")
if has_count:
user_pending = gen_task_repo.count_pending_by_user(user_id)
global_pending = gen_task_repo.count_pending_total()
if user_pending >= USER_PENDING_LIMIT:
raise HTTPException(
status_code=429,
detail=f"您的待处理任务过多(当前 {user_pending}/{USER_PENDING_LIMIT}),请等待完成后再提交",
)
if global_pending >= GLOBAL_PENDING_LIMIT:
raise HTTPException(
status_code=503,
detail="系统繁忙,请稍后再试",
)
except HTTPException:
raise
except Exception as e:
logger.warning("[队列限流] 剪辑计划限流检查失败,跳过: %s", e)
@router.post("/{plan_id}/generate", response_model=EditPlanGenerateResponse)
def generate_plan(
plan_id: str,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
asset_library_repo: Any = Depends(get_asset_library_repository),
asset_repo: Any = Depends(get_asset_repository),
) -> EditPlanGenerateResponse:
"""触发剪辑计划渲染生成
前置条件:计划状态必须为 editing,且至少有一个片段。
流程:
1. 验证计划状态为 editing
2. 将 pending 片段标记为 ready
3. 创建 GenerationTask
4. 调度 Celery 任务 worker.render_edit_plan
5. 将计划状态流转为 rendering
"""
svc = EditPlanService(db)
plan_check = svc.get_plan(plan_id)
if plan_check is None:
raise HTTPException(status_code=404, detail=f"剪辑计划不存在: {plan_id}")
if plan_check.project_id:
check_project_access(plan_check.project_id, current_user.user.id, project_repository)
# 自动兜底流程
_auto_fallback_draft_to_editing(svc, plan_id, plan_check)
_auto_fallback_copy_template_clips(svc, plan_id, plan_check, db)
clips_without_asset = _auto_fallback_assign_assets(svc, plan_id, plan_check)
_auto_fallback_auto_material_mode(svc, plan_id, plan_check, clips_without_asset, asset_library_repo, asset_repo)
# 检查是否可生成
try:
can_gen, reason = svc.can_generate(plan_id)
except ValueError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc
if not can_gen:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=reason)
# 核心生成流程
try:
clip_count = svc.mark_clips_ready(plan_id)
gen_task_repo = SQLAlchemyGenerationTaskRepository(db)
user_id = current_user.user.id
_check_queue_limits(gen_task_repo, user_id)
gen_task_use_case = CreateGenerationTaskUseCase(gen_task_repo)
plan = svc.get_plan_or_raise(plan_id)
# 从 plan.config 中读取 asset_ids 并传递给 GenerationTask
config_asset_ids = (plan.config or {}).get("asset_ids", [])
gen_task = gen_task_use_case.execute(
CreateGenerationTaskCommand(
project_id=plan.project_id or "",
template_id=plan.template_id,
created_by_user_id=current_user.user.id,
source_edit_plan_id=plan_id,
asset_ids=list(config_asset_ids) if config_asset_ids else [],
)
)
svc.update_plan_config(plan_id, {"generation_task_id": gen_task.id})
svc.transition_status(plan_id, EditPlanStatus.RENDERING)
celery_app.send_task("worker.render_edit_plan", args=[plan_id])
updated_plan = svc.get_plan_or_raise(plan_id)
logger.info(
"触发剪辑计划生成: plan_id=%s gen_task_id=%s clips=%d by user=%s",
plan_id,
gen_task.id,
clip_count,
current_user.user.id,
)
return EditPlanGenerateResponse(
plan_id=plan_id,
plan_status=updated_plan.status.value if hasattr(updated_plan.status, "value") else updated_plan.status,
generation_task_id=gen_task.id,
clip_count=clip_count,
)
except HTTPException:
raise
except Exception as _e:
logger.exception("触发剪辑计划生成失败: plan_id=%s", plan_id)
try:
svc.transition_status(plan_id, EditPlanStatus.FAILED)
except Exception:
logger.warning("标记计划失败状态时异常: plan_id=%s", plan_id)
raise HTTPException(
status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
detail="生成失败,请稍后重试",
) from _e
@router.get(
"/{plan_id}/generation-status",
response_model=EditPlanGenerationStatusResponse,
)
def get_generation_status(
plan_id: str,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
storage_service: OSSStorageService = Depends(get_storage_service),
) -> EditPlanGenerationStatusResponse:
"""查询剪辑计划生成进度"""
svc = EditPlanService(db)
try:
gen_status = svc.get_generation_status(plan_id)
except ValueError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc
plan = gen_status["plan"]
if plan.project_id:
check_project_access(plan.project_id, current_user.user.id, project_repository)
clips = gen_status["clips"]
clip_items = [
ClipStatusItem(
clip_id=c.id,
clip_type=c.clip_type,
order=c.order,
status=c.status.value if hasattr(c.status, "value") else c.status,
asset_id=c.asset_id or "",
text_content=c.text_content or "",
duration=c.duration,
)
for c in clips
]
# 从 plan.config 中取渲染结果 URL,转换为签名 URL
raw_video_url = (plan.config or {}).get("rendered_url", "")
video_url = ""
if raw_video_url:
try:
video_url = storage_service.get_download_url(raw_video_url, expires_seconds=86400)
except Exception as e:
logger.warning("生成视频签名URL失败,返回原始URL: plan_id=%s error=%s", plan_id, e)
video_url = raw_video_url
# 从 gen_status 中取进度、错误信息、任务状态
progress = gen_status.get("progress", 0.0)
error_message = gen_status.get("error_message", "")
gen_task_status = gen_status.get("generation_task_status")
# 如果计划已完成但进度还是0,补100
plan_status_val = plan.status.value if hasattr(plan.status, "value") else plan.status
if plan_status_val == "completed" and progress < 100:
progress = 100.0
return EditPlanGenerationStatusResponse(
plan_id=plan_id,
plan_status=plan_status_val,
generation_task_id=gen_status["generation_task_id"],
generation_task_status=gen_task_status,
progress=progress,
video_url=video_url,
error_message=error_message,
clips=clip_items,
)
@router.get(
"/{plan_id}/generations",
response_model=EditPlanGenerationsResponse,
)
def list_plan_generations(
plan_id: str,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> EditPlanGenerationsResponse:
"""查询剪辑计划关联的所有生成记录"""
svc = EditPlanService(db)
plan = svc.get_plan_or_raise(plan_id)
if plan.project_id:
check_project_access(plan.project_id, current_user.user.id, project_repository)
from app.schemas.generation_task import GenerationTaskResponse
gen_task_repo = SQLAlchemyGenerationTaskRepository(db)
tasks = gen_task_repo.list_by_source_edit_plan(plan_id)
items = [
GenerationTaskResponse(
id=t.id,
project_id=t.project_id,
asset_library_id=t.asset_library_id,
strategy_id=t.strategy_id,
voice_library_id=t.voice_library_id,
template_id=t.template_id,
asset_ids=t.asset_ids,
title_ids=t.title_ids,
voice_ids=t.voice_ids,
source_edit_plan_id=t.source_edit_plan_id or "",
status=t.status.value if hasattr(t.status, "value") else t.status,
progress=t.progress,
result_count=t.result_count,
error_message=t.error_message,
)
for t in tasks
]
return EditPlanGenerationsResponse(items=items, total=len(items))
@@ -1,257 +0,0 @@
"""剪辑计划时间线 & 模板生成 API 端点。
从 edit_plans.py 拆分,包含:
- GET /{plan_id}/timeline 时间线场景数据
- POST /generate-from-template 基于模板+素材自动生成剪辑计划
"""
from __future__ import annotations
import logging
from typing import Any, List
from app.api.routes._helpers import auto_select_video_assets, check_project_access
from app.api.routes.edit_plans import (
GenerateFromTemplateRequest,
GenerateFromTemplateResponse,
_PlanClipItem,
_to_response,
)
from app.auth import AuthenticatedUser, get_current_user
from app.dependencies import (
get_asset_library_repository,
get_asset_repository,
get_db_session,
get_project_repository,
)
from app.services import EditPlanService, PlanGeneratorService
from fastapi import APIRouter, Depends, HTTPException, status
from pydantic import BaseModel, Field
from sqlalchemy.orm import Session
logger = logging.getLogger(__name__)
router = APIRouter()
# ── Timeline Schemas ──────────────────────────────────────────────────────────
class TimelineSceneResponse(BaseModel):
"""时间线场景"""
scene: str = Field(..., description="场景描述")
time: str = Field(..., description='时间范围,如 "0:00 - 0:05"')
duration: float = Field(..., ge=0, description="时长(秒)")
color: str = Field(..., description="展示颜色")
clip_id: str = Field(default="", description="关联的片段 ID")
clip_type: str = Field(default="", description="片段类型")
class TimelineResponse(BaseModel):
"""时间线响应"""
plan_id: str
total_duration: float
scenes: List[TimelineSceneResponse]
# clip_type → 颜色映射
_CLIP_TYPE_COLORS = {
"intro": "#6366f1",
"title": "#6366f1",
"product": "#818cf8",
"showcase": "#10b981",
"scene": "#10b981",
"subtitle": "#f59e0b",
"text": "#f59e0b",
"cta": "#ef4444",
"outro": "#ef4444",
"voiceover": "#8b5cf6",
"transition": "#64748b",
}
_DEFAULT_COLOR = "#6366f1"
def _format_time(seconds: float) -> str:
"""将秒数格式化为 M:SS"""
m = int(seconds) // 60
s = int(seconds) % 60
return f"{m}:{s:02d}"
def _clip_type_to_scene_label(clip_type: str, text_content: str) -> str:
"""根据 clip_type 和 text_content 生成场景描述"""
type_labels = {
"intro": "开场",
"title": "标题",
"product": "产品展示",
"showcase": "场景展示",
"scene": "场景",
"subtitle": "字幕",
"text": "文字",
"cta": "结尾 CTA",
"outro": "结尾",
"voiceover": "配音",
"transition": "转场",
}
label = type_labels.get(clip_type, clip_type or "片段")
if text_content:
short = text_content[:20].strip()
if short:
return f"{label} - {short}"
return label
# ── Routes ────────────────────────────────────────────────────────────────────
@router.get(
"/{plan_id}/timeline",
response_model=TimelineResponse,
)
def get_plan_timeline(
plan_id: str,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> TimelineResponse:
"""获取剪辑计划的时间线场景数据"""
svc = EditPlanService(db)
plan = svc.get_plan_or_raise(plan_id)
if plan.project_id:
check_project_access(plan.project_id, current_user.user.id, project_repository)
clips = svc.list_clips(plan_id=plan_id, skip=0, limit=200)
clips.sort(key=lambda c: c.order)
scenes: List[TimelineSceneResponse] = []
current_time = 0.0
for clip in clips:
start = current_time
end = start + clip.duration
color = _CLIP_TYPE_COLORS.get(clip.clip_type, _DEFAULT_COLOR)
scene_label = _clip_type_to_scene_label(clip.clip_type, clip.text_content)
scenes.append(
TimelineSceneResponse(
scene=scene_label,
time=f"{_format_time(start)} - {_format_time(end)}",
duration=clip.duration,
color=color,
clip_id=clip.id,
clip_type=clip.clip_type,
)
)
current_time = end
total_duration = sum(s.duration for s in scenes) or plan.total_duration
return TimelineResponse(
plan_id=plan_id,
total_duration=total_duration,
scenes=scenes,
)
@router.post(
"/generate-from-template",
response_model=GenerateFromTemplateResponse,
status_code=status.HTTP_201_CREATED,
)
def generate_from_template(
body: GenerateFromTemplateRequest,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
asset_repository: Any = Depends(get_asset_repository),
asset_library_repository: Any = Depends(get_asset_library_repository),
) -> GenerateFromTemplateResponse:
"""基于模板 + 素材自动生成剪辑计划"""
from app.services import EditTemplateService
if body.project_id:
check_project_access(body.project_id, current_user.user.id, project_repository)
template_svc = EditTemplateService(db)
try:
template = template_svc.get_template_or_raise(body.template_id)
except ValueError as exc:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail=str(exc)) from exc
clip_configs = template_svc.list_clip_configs(body.template_id, skip=0, limit=200)
# 自动选素材:未传 asset_ids 但有 project_id 时,从项目视频素材库选 ready 的视频素材
resolved_asset_ids = list(body.asset_ids)
if not resolved_asset_ids and body.project_id:
auto_assets = auto_select_video_assets(
project_id=body.project_id,
asset_library_repo=asset_library_repository,
asset_repo=asset_repository,
logger=logger,
)
if auto_assets:
resolved_asset_ids = auto_assets
logger.info(
"generate-from-template 自动选素材: project_id=%s count=%d",
body.project_id,
len(auto_assets),
)
generator = PlanGeneratorService(db)
result = generator.generate_from_template(
template=template,
clip_configs=clip_configs,
asset_ids=resolved_asset_ids,
project_id=body.project_id,
created_by_user_id=current_user.user.id,
name=body.name,
)
plan = result["plan"]
clips = result["clips"]
# 把 asset_ids 写入 plan.config,供生成时兜底分配使用
if resolved_asset_ids:
from app.services import EditPlanService
from packages.domain.config_schemas import normalize_plan_config
svc = EditPlanService(db)
current_config = plan.config or {}
if current_config.get("asset_ids") != resolved_asset_ids:
current_config["asset_ids"] = resolved_asset_ids
plan = svc.update_plan(plan.id, config=normalize_plan_config(current_config))
logger.info(
"基于模板生成剪辑计划: plan_id=%s template_id=%s clips=%d by user=%d",
plan.id,
body.template_id,
len(clips),
current_user.user.id,
)
return GenerateFromTemplateResponse(
plan=_to_response(plan),
clips=[
_PlanClipItem(
id=c.id,
clip_type=c.clip_type,
order=c.order,
asset_id=c.asset_id,
text_content=c.text_content,
start_time=c.start_time,
duration=c.duration,
transition_effect=c.transition_effect,
transition_duration=c.transition_duration,
status=c.status.value if hasattr(c.status, "value") else c.status,
config=c.config,
created_at=c.created_at,
updated_at=c.updated_at,
)
for c in clips
],
)
@@ -1,272 +0,0 @@
"""转场特效 API.
- GET /transition-presets 转场预设列表
- PUT /clips/{clip_id}/transition 设置单个片段转场
- POST /{plan_id}/transitions/batch 批量设置转场(所有片段)
"""
from __future__ import annotations
import logging
from typing import Any, List, Optional
from app.auth import AuthenticatedUser, get_current_user
from app.dependencies import get_db_session, get_project_repository
from app.services import EditPlanService
from fastapi import APIRouter, Depends, HTTPException, Query, status
from pydantic import BaseModel, Field
from sqlalchemy.orm import Session
from packages.domain.transition_presets import (
TransitionPreset,
get_transition_preset,
list_transition_presets,
)
from ._helpers import check_project_access
logger = logging.getLogger(__name__)
router = APIRouter()
# ── Schemas ──────────────────────────────────────────────────────────────────
class TransitionPresetResponse(BaseModel):
"""转场预设响应"""
id: str
name: str
category: str
description: str
tags: List[str] = Field(default_factory=list)
default_duration: float
min_duration: float
max_duration: float
class TransitionUpdateRequest(BaseModel):
"""更新转场请求"""
effect: str = Field(..., description="转场效果 ID")
duration: Optional[float] = Field(default=None, ge=0.0, description="转场时长(秒)")
class BatchTransitionRequest(BaseModel):
"""批量设置转场请求"""
effect: str = Field(..., description="转场效果 ID")
duration: Optional[float] = Field(default=None, ge=0.0, description="转场时长(秒)")
apply_to: str = Field(
default="all",
description="应用范围: all=所有片段, except_first=除第一个外, except_last=除最后一个, middle=中间片段",
)
class ClipTransitionResponse(BaseModel):
"""片段转场信息响应"""
clip_id: str
effect: str
duration: float
class BatchTransitionResponse(BaseModel):
"""批量转场响应"""
updated_count: int
plan_id: str
class TransitionPresetListResponse(BaseModel):
"""转场预设列表响应"""
items: List[TransitionPresetResponse]
total: int
# ── Helpers ──────────────────────────────────────────────────────────────────
def _preset_to_response(p: TransitionPreset) -> TransitionPresetResponse:
return TransitionPresetResponse(
id=p.id,
name=p.name,
category=p.category,
description=p.description,
tags=list(p.tags),
default_duration=p.default_duration,
min_duration=p.min_duration,
max_duration=p.max_duration,
)
def _validate_transition(effect: str, duration: Optional[float] = None) -> tuple[str, float]:
"""验证转场效果和时长,返回 (effect, duration)"""
preset = get_transition_preset(effect)
if preset is None:
raise ValueError(f"无效的转场效果: {effect}")
# 硬切特殊处理,时长强制为0
if effect == "transition_none" or preset.transition == "none":
return "cut", 0.0
final_duration = duration if duration is not None else preset.default_duration
if final_duration < preset.min_duration:
final_duration = preset.min_duration
if final_duration > preset.max_duration:
final_duration = preset.max_duration
return preset.transition, round(final_duration, 3)
# ── Routes ───────────────────────────────────────────────────────────────────
@router.get("/transition-presets", response_model=TransitionPresetListResponse)
def list_presets(
category: Optional[str] = Query(default=None, description="按分类筛选"),
keyword: Optional[str] = Query(default=None, description="关键词搜索"),
current_user: AuthenticatedUser = Depends(get_current_user),
) -> TransitionPresetListResponse:
"""获取转场预设列表"""
presets = list_transition_presets(category=category, keyword=keyword)
items = [_preset_to_response(p) for p in presets]
return TransitionPresetListResponse(items=items, total=len(items))
@router.put("/clips/{clip_id}/transition", response_model=ClipTransitionResponse)
def update_clip_transition(
clip_id: str,
body: TransitionUpdateRequest,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> ClipTransitionResponse:
"""设置单个片段的转场效果"""
svc = EditPlanService(db)
clip = svc.get_clip(clip_id)
if not clip:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"片段不存在: {clip_id}",
)
plan = svc.get_plan(clip.plan_id)
if plan and plan.project_id:
check_project_access(plan.project_id, current_user.user.id, project_repository)
# 验证转场参数
try:
effect, duration = _validate_transition(body.effect, body.duration)
except ValueError as e:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(e)) from e
# 更新片段
updated_clip = svc.update_clip(
clip_id,
transition_effect=effect,
transition_duration=duration,
)
logger.info(
"更新片段转场: clip_id=%s effect=%s duration=%.3f by user=%s",
clip_id,
effect,
duration,
current_user.user.id,
)
return ClipTransitionResponse(
clip_id=clip_id,
effect=updated_clip.transition_effect,
duration=updated_clip.transition_duration,
)
@router.post("/{plan_id}/transitions/batch", response_model=BatchTransitionResponse)
def batch_update_transitions(
plan_id: str,
body: BatchTransitionRequest,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> BatchTransitionResponse:
"""批量设置计划内所有片段的转场效果
apply_to 说明:
- all: 所有片段
- except_first: 除第一个片段外(第一个片段不需要前转场)
- except_last: 除最后一个片段外
- middle: 只设置中间片段(除首尾)
"""
svc = EditPlanService(db)
plan = svc.get_plan(plan_id)
if not plan:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=f"剪辑计划不存在: {plan_id}",
)
if plan.project_id:
check_project_access(plan.project_id, current_user.user.id, project_repository)
# 验证转场参数
try:
effect, duration = _validate_transition(body.effect, body.duration)
except ValueError as e:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=str(e)) from e
# 获取所有片段
clips = svc.list_clips(plan_id, limit=500, skip=0)
if not clips:
return BatchTransitionResponse(updated_count=0, plan_id=plan_id)
# 确定应用范围
total = len(clips)
if total <= 1:
# 只有一个片段时,只有 all 模式才应用
if body.apply_to != "all":
return BatchTransitionResponse(updated_count=0, plan_id=plan_id)
# 按 order 排序
clips_sorted = sorted(clips, key=lambda c: c.order)
indices_to_update = []
if body.apply_to == "all":
indices_to_update = list(range(total))
elif body.apply_to == "except_first":
indices_to_update = list(range(1, total))
elif body.apply_to == "except_last":
indices_to_update = list(range(total - 1))
elif body.apply_to == "middle":
if total <= 2:
indices_to_update = []
else:
indices_to_update = list(range(1, total - 1))
else:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"无效的 apply_to: {body.apply_to}",
)
# 批量更新
count = 0
for idx in indices_to_update:
clip = clips_sorted[idx]
svc.update_clip(
clip.id,
transition_effect=effect,
transition_duration=duration,
)
count += 1
logger.info(
"批量更新转场: plan_id=%s count=%d effect=%s apply_to=%s by user=%s",
plan_id,
count,
effect,
body.apply_to,
current_user.user.id,
)
return BatchTransitionResponse(updated_count=count, plan_id=plan_id)
+279
View File
@@ -0,0 +1,279 @@
"""模板管理 API — Phase 8 模板编排引擎.
RESTful CRUD for EditTemplate:
- GET /api/v1/edit-templates 列表(分页 + 类型筛选)
- GET /api/v1/edit-templates/{id} 详情
- POST /api/v1/edit-templates 创建(管理员)
- PUT /api/v1/edit-templates/{id} 更新
- DELETE /api/v1/edit-templates/{id} 删除(软删除 → inactive
业务逻辑委托给 EditTemplateService 服务层。
"""
from __future__ import annotations
import logging
from datetime import datetime
from typing import Any, List, Optional
from app.auth import AuthenticatedUser, get_current_user
from app.dependencies import get_db_session
from app.services import EditTemplateService
from fastapi import APIRouter, Depends, HTTPException, Query, status
from fastapi.responses import Response
from pydantic import BaseModel, Field
from sqlalchemy.orm import Session
from packages.domain.config_schemas import normalize_template_config
from packages.domain.edit_template import EditTemplate, EditTemplateStatus
logger = logging.getLogger(__name__)
router = APIRouter()
# ── Pydantic Schemas ─────────────────────────────────────────────────────────
class EditTemplateCreateRequest(BaseModel):
"""创建模板请求体"""
name: str = Field(..., min_length=1, max_length=200, description="模板名称")
description: str = Field(default="", max_length=2000, description="模板描述")
template_type: str = Field(default="default", max_length=50, description="模板类型")
config: dict[str, Any] = Field(default_factory=dict, description="模板配置 (JSON)")
preview_url: str = Field(default="", max_length=500, description="预览地址")
sort_weight: int = Field(default=0, ge=0, le=9999, description="排序权重")
class EditTemplateUpdateRequest(BaseModel):
"""更新模板请求体"""
name: Optional[str] = Field(default=None, min_length=1, max_length=200, description="模板名称")
description: Optional[str] = Field(default=None, max_length=2000, description="模板描述")
template_type: Optional[str] = Field(default=None, max_length=50, description="模板类型")
config: Optional[dict[str, Any]] = Field(default=None, description="模板配置 (JSON)")
preview_url: Optional[str] = Field(default=None, max_length=500, description="预览地址")
sort_weight: Optional[int] = Field(default=None, ge=0, le=9999, description="排序权重")
status: Optional[str] = Field(default=None, description="状态: active / inactive")
class EditTemplateResponse(BaseModel):
"""模板响应体"""
id: str
name: str
description: str
template_type: str
config: dict[str, Any]
preview_url: str
sort_weight: int
status: str
created_at: datetime
updated_at: datetime
model_config = {"from_attributes": True}
class EditTemplateListResponse(BaseModel):
"""模板列表响应体"""
items: List[EditTemplateResponse]
total: int
page: int
page_size: int
# ── Helpers ───────────────────────────────────────────────────────────────────
def _require_admin(current_user: AuthenticatedUser) -> None:
"""校验当前用户是否为管理员,非管理员返回 403"""
if not getattr(current_user.user, "is_admin", False):
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="仅管理员可执行此操作",
)
def _to_response(t: EditTemplate) -> EditTemplateResponse:
return EditTemplateResponse(
id=t.id,
name=t.name,
description=t.description,
template_type=t.template_type,
config=t.config,
preview_url=t.preview_url,
sort_weight=t.sort_weight,
status=t.status.value if hasattr(t.status, "value") else t.status,
created_at=t.created_at,
updated_at=t.updated_at,
)
# ── Routes ────────────────────────────────────────────────────────────────────
@router.get("", response_model=EditTemplateListResponse)
def list_templates(
page: int = Query(default=1, ge=1, description="页码"),
page_size: int = Query(default=20, ge=1, le=100, description="每页数量"),
template_type: Optional[str] = Query(default=None, description="按类型筛选"),
status_filter: Optional[str] = Query(
default=None,
alias="status",
description="按状态筛选: active / inactive",
),
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
) -> EditTemplateListResponse:
"""获取模板列表(支持分页、按类型/状态筛选)"""
svc = EditTemplateService(db)
# 解析状态筛选
status_enum: Optional[EditTemplateStatus] = None
if status_filter:
try:
status_enum = EditTemplateStatus(status_filter)
except ValueError:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"无效的状态值: {status_filter},可选值: active, inactive",
)
skip = (page - 1) * page_size
templates = svc.list_templates(
template_type=template_type,
status=status_enum,
skip=skip,
limit=page_size,
)
total = svc.count_templates(
template_type=template_type,
status=status_enum,
)
return EditTemplateListResponse(
items=[_to_response(t) for t in templates],
total=total,
page=page,
page_size=page_size,
)
@router.get("/{template_id}", response_model=EditTemplateResponse)
def get_template(
template_id: str,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
) -> EditTemplateResponse:
"""获取单个模板详情"""
svc = EditTemplateService(db)
try:
template = svc.get_template_or_raise(template_id)
except ValueError as exc:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=str(exc),
)
return _to_response(template)
@router.post("", response_model=EditTemplateResponse, status_code=status.HTTP_201_CREATED)
def create_template(
body: EditTemplateCreateRequest,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
) -> EditTemplateResponse:
"""创建模板(管理员)"""
_require_admin(current_user)
svc = EditTemplateService(db)
# 标准化 config,填充 cover/title/subtitle/bgm 默认值
normalized_config = normalize_template_config(body.config)
try:
created = svc.create_template(
name=body.name,
description=body.description,
template_type=body.template_type,
config=normalized_config,
preview_url=body.preview_url,
sort_weight=body.sort_weight,
)
except ValueError as exc:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=str(exc),
)
logger.info("创建模板: id=%s name=%s by user=%s", created.id, created.name, current_user.user.id)
return _to_response(created)
@router.put("/{template_id}", response_model=EditTemplateResponse)
def update_template(
template_id: str,
body: EditTemplateUpdateRequest,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
) -> EditTemplateResponse:
"""更新模板"""
_require_admin(current_user)
svc = EditTemplateService(db)
# 解析状态
status_enum: Optional[EditTemplateStatus] = None
if body.status is not None:
try:
status_enum = EditTemplateStatus(body.status)
except ValueError:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"无效的状态值: {body.status},可选值: active, inactive",
)
# 标准化 config(如果提供了)
config_to_update = normalize_template_config(body.config) if body.config is not None else None
try:
result = svc.update_template(
template_id,
name=body.name,
description=body.description,
template_type=body.template_type,
config=config_to_update,
preview_url=body.preview_url,
sort_weight=body.sort_weight,
status=status_enum,
)
except ValueError as exc:
err_msg = str(exc)
if "不存在" in err_msg:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=err_msg,
)
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=err_msg,
)
logger.info("更新模板: id=%s by user=%s", template_id, current_user.user.id)
return _to_response(result)
@router.delete("/{template_id}", status_code=status.HTTP_204_NO_CONTENT, response_model=None)
def delete_template(
template_id: str,
db: Session = Depends(get_db_session),
current_user: AuthenticatedUser = Depends(get_current_user),
) -> Response:
"""删除模板(软删除 → 设为 inactive)"""
_require_admin(current_user)
svc = EditTemplateService(db)
try:
svc.deactivate_template(template_id)
except ValueError as exc:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail=str(exc),
)
logger.info("删除模板(软删除): id=%s by user=%s", template_id, current_user.user.id)
return Response(status_code=204)
-194
View File
@@ -1,194 +0,0 @@
"""Feature Flag 内部管理接口。
通过内部 API Key 鉴权,支持查看和修改 Feature Flag 配置。
主要用于灰度发布期间的动态开关控制。
API:
GET /api/v1/internal/feature-flags - 列出所有 flag
GET /api/v1/internal/feature-flags/{name} - 查看单个 flag
PUT /api/v1/internal/feature-flags/{name} - 设置 flag 配置
DELETE /api/v1/internal/feature-flags/{name} - 删除 flag
鉴权:X-API-Key header,走内部 API Key 验证
"""
from __future__ import annotations
import logging
from typing import Optional
from app.api.routes.auth import _verify_internal_api_key
from app.config import settings
from fastapi import APIRouter, Depends, HTTPException, Query, Response, status
from pydantic import BaseModel, Field
from packages.adapters.redis.feature_flag_store import (
FeatureFlagConfig,
RedisFeatureFlagStore,
)
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/internal/feature-flags", tags=["Internal"])
# 允许管理的 flag 白名单(防止误操作其他系统 flag)
ALLOWED_FLAGS = {
"render_engine",
}
def _get_feature_flag_store() -> RedisFeatureFlagStore:
"""获取 Feature Flag 存储实例。"""
return RedisFeatureFlagStore(redis_url=settings.REDIS_URL)
class FeatureFlagUpdateRequest(BaseModel):
"""Feature Flag 更新请求体。"""
enabled: bool = Field(..., description="是否启用")
percentage: int = Field(0, ge=0, le=100, description="灰度百分比 (0-100)")
whitelist: list[str] = Field(default_factory=list, description="白名单列表(如 user_id")
class FeatureFlagResponse(BaseModel):
"""Feature Flag 响应。"""
name: str
enabled: bool
percentage: int
whitelist: list[str]
@classmethod
def from_config(cls, config: FeatureFlagConfig) -> "FeatureFlagResponse":
return cls(
name=config.name,
enabled=config.enabled,
percentage=config.percentage,
whitelist=sorted(config.whitelist),
)
class FeatureFlagCheckResponse(BaseModel):
"""Flag 激活检查响应。"""
name: str
active: bool
identifier: Optional[str] = None
def _validate_flag_name(name: str) -> None:
"""校验 flag 名称是否在允许列表中。"""
if name not in ALLOWED_FLAGS:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail=f"Unsupported flag: {name}. Allowed: {sorted(ALLOWED_FLAGS)}",
)
@router.get("", response_model=list[FeatureFlagResponse])
async def list_feature_flags(
_: bool = Depends(_verify_internal_api_key),
store: RedisFeatureFlagStore = Depends(_get_feature_flag_store),
) -> list[FeatureFlagResponse]:
"""列出所有 Feature Flag。"""
try:
flags = store.list_all()
# 同时返回预定义的 flag(即使未设置也显示默认值)
result = []
for name in sorted(ALLOWED_FLAGS):
config = flags.get(name) or FeatureFlagConfig(name=name, enabled=False)
result.append(FeatureFlagResponse.from_config(config))
# 加上已存在但不在白名单中的 flag(只读展示)
for name, config in flags.items():
if name not in ALLOWED_FLAGS:
result.append(FeatureFlagResponse.from_config(config))
return sorted(result, key=lambda x: x.name)
except Exception as exc:
logger.error("Failed to list feature flags: %s", exc)
raise HTTPException(status_code=500, detail=f"Failed to list flags: {exc}") from exc
@router.get("/{name}", response_model=FeatureFlagResponse)
async def get_feature_flag(
name: str,
_: bool = Depends(_verify_internal_api_key),
store: RedisFeatureFlagStore = Depends(_get_feature_flag_store),
) -> FeatureFlagResponse:
"""获取单个 Feature Flag 配置。"""
try:
config = store.get(name)
return FeatureFlagResponse.from_config(config)
except Exception as exc:
logger.error("Failed to get feature flag %s: %s", name, exc)
raise HTTPException(status_code=500, detail=f"Failed to get flag: {exc}") from exc
@router.get("/{name}/check", response_model=FeatureFlagCheckResponse)
async def check_feature_flag(
name: str,
identifier: Optional[str] = Query(None, description="标识符,如 user_id"),
_: bool = Depends(_verify_internal_api_key),
store: RedisFeatureFlagStore = Depends(_get_feature_flag_store),
) -> FeatureFlagCheckResponse:
"""检查某个标识符是否命中 Feature Flag。"""
try:
active = store.is_active(name, identifier=identifier)
return FeatureFlagCheckResponse(name=name, active=active, identifier=identifier)
except Exception as exc:
logger.error("Failed to check feature flag %s: %s", name, exc)
raise HTTPException(status_code=500, detail=f"Failed to check flag: {exc}") from exc
@router.put("/{name}", response_model=FeatureFlagResponse)
async def update_feature_flag(
name: str,
request: FeatureFlagUpdateRequest,
_: bool = Depends(_verify_internal_api_key),
store: RedisFeatureFlagStore = Depends(_get_feature_flag_store),
) -> FeatureFlagResponse:
"""更新 Feature Flag 配置。
只允许修改 ALLOWED_FLAGS 列表中的 flag。
"""
_validate_flag_name(name)
try:
config = FeatureFlagConfig(
name=name,
enabled=request.enabled,
percentage=request.percentage,
whitelist=set(request.whitelist),
)
store.set(config)
logger.info(
"Feature flag updated: name=%s enabled=%s percentage=%d whitelist=%d",
name,
config.enabled,
config.percentage,
len(config.whitelist),
)
return FeatureFlagResponse.from_config(config)
except Exception as exc:
logger.error("Failed to update feature flag %s: %s", name, exc)
raise HTTPException(status_code=500, detail=f"Failed to update flag: {exc}") from exc
@router.delete("/{name}", status_code=status.HTTP_204_NO_CONTENT, response_model=None, response_class=Response)
async def delete_feature_flag(
name: str,
_: bool = Depends(_verify_internal_api_key),
store: RedisFeatureFlagStore = Depends(_get_feature_flag_store),
):
"""删除 Feature Flag。
只允许删除 ALLOWED_FLAGS 列表中的 flag。
"""
_validate_flag_name(name)
try:
deleted = store.delete(name)
logger.info("Feature flag deleted: name=%s deleted=%s", name, deleted)
pass
except Exception as exc:
logger.error("Failed to delete feature flag %s: %s", name, exc)
raise HTTPException(status_code=500, detail=f"Failed to delete flag: {exc}") from exc
+123
View File
@@ -0,0 +1,123 @@
from typing import Any
from app.auth import AuthenticatedUser, get_current_user
from app.core.storage import OSSStorageService, get_storage_service
from app.dependencies import get_generated_video_repository, get_project_repository
from app.schemas.generated_video import (
GeneratedVideoDownloadUrlResponse,
GeneratedVideoResponse,
ListGeneratedVideosResponse,
UpdateGeneratedVideoReviewRequest,
)
from fastapi import APIRouter, Depends, HTTPException, Query
from packages.application import (
GetGeneratedVideoDownloadUrlUseCase,
GetGeneratedVideoUseCase,
ListGeneratedVideosUseCase,
)
router = APIRouter()
def _to_generated_video_response(item, download_url: str | None = None) -> GeneratedVideoResponse:
return GeneratedVideoResponse(
id=item.id,
project_id=item.project_id,
generation_task_id=item.generation_task_id,
name=item.name,
file_url=item.file_url,
file_size=item.file_size,
duration=item.duration,
thumbnail_url=item.thumbnail_url,
width=item.width,
height=item.height,
fps=item.fps,
status=item.status,
review_status=item.review_status,
generation_params=item.generation_params,
download_url=download_url,
)
@router.get("", response_model=ListGeneratedVideosResponse)
def list_generated_videos(
project_id: str | None = Query(None),
authenticated_user: AuthenticatedUser = Depends(get_current_user),
generated_video_repository: Any = Depends(get_generated_video_repository),
project_repository: Any = Depends(get_project_repository),
storage_service: OSSStorageService = Depends(get_storage_service),
) -> ListGeneratedVideosResponse:
user_id = authenticated_user.user.id
use_case = ListGeneratedVideosUseCase(generated_video_repository)
if project_id:
# If project_id provided, check access and filter by project
project = project_repository.find_by_id(project_id)
if project is None:
raise HTTPException(status_code=404, detail=f"Project {project_id} not found")
items = use_case.execute(project_id)
else:
# If no project_id, list all videos from accessible projects
accessible_projects = project_repository.find_accessible_projects(user_id)
all_items = []
for proj in accessible_projects:
all_items.extend(use_case.execute(proj.id))
items = all_items
# Generate download URLs for each video
responses = []
for item in items:
download_url = storage_service.get_download_url(item.file_url)
responses.append(_to_generated_video_response(item, download_url=download_url))
return ListGeneratedVideosResponse(items=responses)
@router.get("/{video_id}", response_model=GeneratedVideoResponse)
def get_generated_video(
video_id: str,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
generated_video_repository: Any = Depends(get_generated_video_repository),
storage_service: OSSStorageService = Depends(get_storage_service),
) -> GeneratedVideoResponse:
use_case = GetGeneratedVideoUseCase(generated_video_repository)
item = use_case.execute(video_id)
if item is None:
raise HTTPException(status_code=404, detail=f"GeneratedVideo {video_id} not found")
download_url = storage_service.get_download_url(item.file_url)
return _to_generated_video_response(item, download_url=download_url)
@router.patch("/{video_id}/review", response_model=GeneratedVideoResponse)
def update_generated_video_review_status(
video_id: str,
request: UpdateGeneratedVideoReviewRequest,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
generated_video_repository: Any = Depends(get_generated_video_repository),
storage_service: OSSStorageService = Depends(get_storage_service),
) -> GeneratedVideoResponse:
video = generated_video_repository.get(video_id)
if video is None:
raise HTTPException(status_code=404, detail=f"GeneratedVideo {video_id} not found")
video.review_status = request.review_status
updated = generated_video_repository.update(video)
download_url = storage_service.get_download_url(updated.file_url)
return _to_generated_video_response(updated, download_url=download_url)
@router.get("/{video_id}/download-url", response_model=GeneratedVideoDownloadUrlResponse)
def get_generated_video_download_url(
video_id: str,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
generated_video_repository: Any = Depends(get_generated_video_repository),
storage_service: OSSStorageService = Depends(get_storage_service),
) -> GeneratedVideoDownloadUrlResponse:
video = generated_video_repository.get(video_id)
if video is None:
raise HTTPException(status_code=404, detail=f"GeneratedVideo {video_id} not found")
use_case = GetGeneratedVideoDownloadUrlUseCase(generated_video_repository)
file_url = use_case.execute(video_id)
if file_url is None:
raise HTTPException(status_code=404, detail=f"GeneratedVideo {video_id} not found")
download_url = storage_service.get_download_url(file_url)
return GeneratedVideoDownloadUrlResponse(video_id=video_id, download_url=download_url)
+38 -208
View File
@@ -1,18 +1,9 @@
import logging
import random
import uuid
from typing import Any
from app.api.routes._helpers import check_project_access
from app.auth import AuthenticatedUser, get_current_user
from app.core.storage import OSSStorageService, get_storage_service
from app.core.task_enqueue import (
GLOBAL_PENDING_LIMIT,
USER_PENDING_LIMIT,
GlobalQueueFull,
UserPendingLimitExceeded,
safe_enqueue_generation_task,
)
from app.core.celery_app import celery_app
from app.dependencies import (
get_asset_library_repository,
get_asset_repository,
@@ -39,11 +30,18 @@ from packages.application import (
ListGeneratedVideosByTaskUseCase,
)
logger = logging.getLogger(__name__)
router = APIRouter()
def _check_project_access(project_id: str, user_id: str, project_repository) -> None:
"""检查用户是否有项目访问权限"""
project = project_repository.find_by_id(project_id)
if project is None:
raise HTTPException(status_code=404, detail=f"Project {project_id} not found")
if not project.can_access(user_id):
raise HTTPException(status_code=403, detail="Access denied to project")
def _to_generation_task_response(task) -> GenerationTaskResponse:
return GenerationTaskResponse(
id=task.id,
@@ -58,8 +56,6 @@ def _to_generation_task_response(task) -> GenerationTaskResponse:
source_edit_plan_id=task.source_edit_plan_id or "",
asset_select_mode=getattr(task, "asset_select_mode", ""),
batch_id=getattr(task, "batch_id", ""),
video_title=getattr(task, "video_title", ""),
logs=getattr(task, "logs", "[]"),
status=task.status,
progress=task.progress,
result_count=task.result_count,
@@ -67,7 +63,7 @@ def _to_generation_task_response(task) -> GenerationTaskResponse:
)
def _to_generated_video_response(item, download_url: str | None = None) -> GeneratedVideoResponse:
def _to_generated_video_response(item) -> GeneratedVideoResponse:
return GeneratedVideoResponse(
id=item.id,
project_id=item.project_id,
@@ -80,7 +76,6 @@ def _to_generated_video_response(item, download_url: str | None = None) -> Gener
width=item.width,
height=item.height,
fps=item.fps,
download_url=download_url,
)
@@ -184,37 +179,19 @@ def create_generation_task(
asset_library_repository: Any = Depends(get_asset_library_repository),
asset_repository: Any = Depends(get_asset_repository),
) -> BatchGenerationTaskResponse:
logger.info(
"[生成任务] 接收请求: user_id=%s, template_id=%s, asset_count=%d, mode=%s, count=%d",
authenticated_user.user.id,
request.template_id,
len(request.asset_ids),
request.asset_select_mode,
request.count,
project_id, asset_library_id = _resolve_project_and_library(
request, project_repository, asset_library_repository, asset_repository, authenticated_user
)
try:
project_id, asset_library_id = _resolve_project_and_library(
request, project_repository, asset_library_repository, asset_repository, authenticated_user
)
except HTTPException as e:
logger.warning("[生成任务] 校验失败: %s", e.detail)
raise
# asset_library 存在性校验(仅在提供了 asset_library_id 时)
resolved_asset_ids: list[str] = list(request.asset_ids)
if asset_library_id:
library = asset_library_repository.get(asset_library_id)
if library is None or (project_id and library.project_id != project_id):
logger.warning("[生成任务] 素材库不存在: library_id=%s", asset_library_id)
raise HTTPException(status_code=404, detail=f"AssetLibrary {asset_library_id} not found")
assets = asset_repository.find_by_library(asset_library_id)
try:
_ensure_library_has_ready_video_assets(assets)
except HTTPException as e:
logger.warning("[生成任务] 素材校验失败: %s", e.detail)
raise
_ensure_library_has_ready_video_assets(assets)
# 素材库自动匹配:当未显式指定 asset_ids 时,按模式自动选取
if not resolved_asset_ids:
@@ -227,88 +204,30 @@ def create_generation_task(
use_case = CreateGenerationTaskUseCase(generation_task_repository)
count = request.count
created_tasks = []
failed_tasks = []
user_id = authenticated_user.user.id
# 同批次任务共享 batch_id,用于视频查重时批次内比对
batch_id = uuid.uuid4().hex if count > 1 else ""
# 预检查:批量提交前先看会不会超限,避免建一半才拒
try:
user_pending = generation_task_repository.count_pending_by_user(user_id)
global_pending = generation_task_repository.count_pending_total()
if user_pending + count > USER_PENDING_LIMIT:
raise UserPendingLimitExceeded(
user_id=user_id, pending_count=user_pending + count, limit=USER_PENDING_LIMIT
for _ in range(count):
task = use_case.execute(
CreateGenerationTaskCommand(
project_id=project_id,
asset_library_id=asset_library_id,
strategy_id=request.strategy_id,
voice_library_id=request.voice_library_id,
template_id=request.template_id,
asset_ids=resolved_asset_ids,
title_ids=request.title_ids,
voice_ids=request.voice_ids,
created_by_user_id=authenticated_user.user.id,
source_edit_plan_id=request.source_edit_plan_id,
asset_select_mode=request.asset_select_mode,
batch_id=batch_id,
)
if global_pending + count > GLOBAL_PENDING_LIMIT:
raise GlobalQueueFull(pending_count=global_pending + count, limit=GLOBAL_PENDING_LIMIT)
except UserPendingLimitExceeded as e:
raise HTTPException(
status_code=429,
detail=f"您的待处理任务过多(当前 {e.pending_count - count}/{e.limit},本次提交 {count} 个),请等待完成后再提交",
) from e
except GlobalQueueFull as e:
raise HTTPException(
status_code=503,
detail="系统繁忙,请稍后再试",
) from e
)
celery_app.send_task("worker.generate_video", args=[task.id])
created_tasks.append(task)
try:
for _ in range(count):
task = use_case.execute(
CreateGenerationTaskCommand(
project_id=project_id,
asset_library_id=asset_library_id,
strategy_id=request.strategy_id,
voice_library_id=request.voice_library_id,
template_id=request.template_id,
asset_ids=resolved_asset_ids,
title_ids=request.title_ids,
voice_ids=request.voice_ids,
created_by_user_id=user_id,
source_edit_plan_id=request.source_edit_plan_id,
asset_select_mode=request.asset_select_mode,
batch_id=batch_id,
video_title=request.video_title,
auto_retry_enabled=request.auto_retry_enabled,
auto_retry_max=request.auto_retry_max,
)
)
try:
if safe_enqueue_generation_task(
task,
generation_task_repository,
user_id=user_id,
log_prefix="[生成任务]",
log_task_status=True,
):
created_tasks.append(task)
else:
failed_tasks.append(task)
except UserPendingLimitExceeded as _e:
# 兜底:如果预检查后又并发提交了,在这里也拦住
failed_tasks.append(task)
if not created_tasks:
raise HTTPException(
status_code=429,
detail="您的待处理任务过多,请等待完成后再提交",
) from _e
break
except GlobalQueueFull as _e:
failed_tasks.append(task)
if not created_tasks:
raise HTTPException(
status_code=503,
detail="系统繁忙,请稍后再试",
) from _e
break
except HTTPException:
raise
except Exception as e:
logger.error("[生成任务] 创建失败: %s", e, exc_info=True)
raise HTTPException(status_code=500, detail="创建生成任务失败,请稍后重试或查看任务日志") from e
items = [_to_generation_task_response(t) for t in created_tasks + failed_tasks]
items = [_to_generation_task_response(t) for t in created_tasks]
return BatchGenerationTaskResponse(items=items, total=len(items))
@@ -335,7 +254,7 @@ def get_generation_task(
if task is None:
raise HTTPException(status_code=404, detail=f"GenerationTask {task_id} not found")
if task.project_id:
check_project_access(task.project_id, authenticated_user.user.id, project_repository)
_check_project_access(task.project_id, authenticated_user.user.id, project_repository)
return _to_generation_task_response(task)
@@ -346,20 +265,15 @@ def list_generation_results(
generation_task_repository: Any = Depends(get_generation_task_repository),
generated_video_repository: Any = Depends(get_generated_video_repository),
project_repository: Any = Depends(get_project_repository),
storage_service: OSSStorageService = Depends(get_storage_service),
) -> ListGeneratedVideosResponse:
task = generation_task_repository.get(task_id)
if task is None:
raise HTTPException(status_code=404, detail=f"GenerationTask {task_id} not found")
if task.project_id:
check_project_access(task.project_id, authenticated_user.user.id, project_repository)
_check_project_access(task.project_id, authenticated_user.user.id, project_repository)
use_case = ListGeneratedVideosByTaskUseCase(generated_video_repository)
items = use_case.execute(task_id)
responses = []
for item in items:
download_url = storage_service.get_download_url(item.file_url, expires_seconds=86400)
responses.append(_to_generated_video_response(item, download_url=download_url))
return ListGeneratedVideosResponse(items=responses)
return ListGeneratedVideosResponse(items=[_to_generated_video_response(item) for item in items])
@router.post("/tasks/{task_id}/retry", response_model=GenerationTaskResponse)
@@ -378,21 +292,6 @@ def retry_generation_task(
if status_val != "failed":
raise HTTPException(status_code=409, detail="Only failed tasks can be retried")
user_id = authenticated_user.user.id
# 预检查:创建前判断,>= 上限就拒绝
user_pending = generation_task_repository.count_pending_by_user(user_id)
global_pending = generation_task_repository.count_pending_total()
if user_pending >= USER_PENDING_LIMIT:
raise HTTPException(
status_code=429,
detail=f"您的待处理任务过多(当前 {user_pending}/{USER_PENDING_LIMIT}),请等待完成后再提交",
)
if global_pending >= GLOBAL_PENDING_LIMIT:
raise HTTPException(
status_code=503,
detail="系统繁忙,请稍后再试",
)
use_case = CreateGenerationTaskUseCase(generation_task_repository)
retried = use_case.execute(
CreateGenerationTaskCommand(
@@ -404,79 +303,10 @@ def retry_generation_task(
asset_ids=task.asset_ids,
title_ids=task.title_ids,
voice_ids=task.voice_ids,
created_by_user_id=user_id,
created_by_user_id=authenticated_user.user.id,
source_edit_plan_id=task.source_edit_plan_id or "",
asset_select_mode=getattr(task, "asset_select_mode", ""),
video_title=getattr(task, "video_title", ""),
)
)
try:
if not safe_enqueue_generation_task(
retried,
generation_task_repository,
user_id=user_id,
log_prefix="[生成任务]",
log_task_status=True,
):
logger.warning("[生成任务] 重试入队失败: task_id=%s", retried.id)
except UserPendingLimitExceeded:
raise HTTPException(
status_code=429,
detail="您的待处理任务过多,请等待完成后再提交",
) from None
except GlobalQueueFull:
raise HTTPException(
status_code=503,
detail="系统繁忙,请稍后再试",
) from None
celery_app.send_task("worker.generate_video", args=[retried.id])
return _to_generation_task_response(retried)
@router.post("/tasks/{task_id}/cancel", response_model=GenerationTaskResponse)
def cancel_generation_task(
task_id: str,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
generation_task_repository: Any = Depends(get_generation_task_repository),
) -> GenerationTaskResponse:
"""取消生成任务。
仅 pending / running 状态的任务可取消;取消后状态变为 cancelled。
对于已在运行的 Celery 任务,标记为 cancelled 后,worker 在下次检查点会中止执行。
"""
task = generation_task_repository.get(task_id)
if task is None:
raise HTTPException(status_code=404, detail="Generation task not found")
# 权限校验
if task.created_by_user_id and task.created_by_user_id != authenticated_user.user.id:
raise HTTPException(status_code=403, detail="Access denied to this task")
status_val = task.status.value if hasattr(task.status, "value") else str(task.status)
# 终态不可取消
if status_val in ("completed", "failed", "cancelled"):
raise HTTPException(
status_code=409,
detail=f"Cannot cancel task in {status_val} status",
)
# 执行取消
try:
task.mark_cancelled()
task.append_log(
stage="cancelled",
message="用户主动取消任务",
level="INFO",
cancelled_by=authenticated_user.user.id,
)
generation_task_repository.update(task)
logger.info(
"生成任务已取消: task_id=%s user_id=%s previous_status=%s",
task_id,
authenticated_user.user.id,
status_val,
)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e)) from e
return _to_generation_task_response(task)
-120
View File
@@ -1,120 +0,0 @@
"""渲染结果内部下载接口。
通过内部 API Key 鉴权,为灰度对比工具等内部系统提供渲染结果下载能力。
API:
GET /api/v1/internal/render/videos/{video_id}/download-url - 获取单个视频下载URL
GET /api/v1/internal/render/tasks/{task_id}/videos - 获取任务下所有视频及下载URL
鉴权:X-API-Key header,走内部 API Key 验证
"""
from __future__ import annotations
import logging
from typing import Any
from app.api.routes.auth import _verify_internal_api_key
from app.core.storage import OSSStorageService, get_storage_service
from app.dependencies import get_generated_video_repository
from fastapi import APIRouter, Depends, HTTPException, Query
from pydantic import BaseModel
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/internal/render", tags=["Internal"])
class InternalRenderVideoItem(BaseModel):
"""内部渲染视频项。"""
video_id: str
generation_task_id: str
project_id: str
name: str
file_url: str
file_size: int | None = None
duration: float | None = None
width: int | None = None
height: int | None = None
fps: float | None = None
status: str
download_url: str
class InternalRenderTaskVideosResponse(BaseModel):
"""任务下所有渲染视频响应。"""
task_id: str
count: int
videos: list[InternalRenderVideoItem]
class InternalRenderDownloadUrlResponse(BaseModel):
"""单个视频下载URL响应。"""
video_id: str
download_url: str
def _video_to_item(video: Any, download_url: str) -> InternalRenderVideoItem:
"""将 GeneratedVideo 领域对象转为响应项。"""
return InternalRenderVideoItem(
video_id=video.id,
generation_task_id=video.generation_task_id,
project_id=video.project_id,
name=video.name,
file_url=video.file_url,
file_size=getattr(video, "file_size", None),
duration=getattr(video, "duration", None),
width=getattr(video, "width", None),
height=getattr(video, "height", None),
fps=getattr(video, "fps", None),
status=video.status,
download_url=download_url,
)
@router.get("/videos/{video_id}/download-url", response_model=InternalRenderDownloadUrlResponse)
def get_render_video_download_url(
video_id: str,
_: bool = Depends(_verify_internal_api_key),
generated_video_repository: Any = Depends(get_generated_video_repository),
storage_service: OSSStorageService = Depends(get_storage_service),
) -> InternalRenderDownloadUrlResponse:
"""获取单个渲染视频的下载URL(预签名)。"""
video = generated_video_repository.get(video_id)
if video is None:
raise HTTPException(status_code=404, detail=f"GeneratedVideo {video_id} not found")
download_url = storage_service.get_download_url(video.file_url, expires_seconds=86400)
logger.info("内部渲染下载URL生成: video_id=%s", video_id)
return InternalRenderDownloadUrlResponse(video_id=video_id, download_url=download_url)
@router.get("/tasks/{task_id}/videos", response_model=InternalRenderTaskVideosResponse)
def get_render_task_videos(
task_id: str,
status: str | None = Query(None, description="按状态筛选,如 completed/failed"),
_: bool = Depends(_verify_internal_api_key),
generated_video_repository: Any = Depends(get_generated_video_repository),
storage_service: OSSStorageService = Depends(get_storage_service),
) -> InternalRenderTaskVideosResponse:
"""获取生成任务下所有渲染视频及下载URL。"""
videos = generated_video_repository.list_by_generation_task(task_id)
# 状态筛选
if status:
videos = [v for v in videos if v.status == status]
items = []
for video in videos:
download_url = storage_service.get_download_url(video.file_url, expires_seconds=86400)
items.append(_video_to_item(video, download_url))
logger.info("内部渲染任务视频查询: task_id=%s count=%d", task_id, len(items))
return InternalRenderTaskVideosResponse(
task_id=task_id,
count=len(items),
videos=items,
)
+332
View File
@@ -0,0 +1,332 @@
"""Job API 路由 — Phase 8 任务 2.10.
提供统一异步任务管理 RESTful 接口:
- POST /api/v1/jobs 创建任务
- GET /api/v1/jobs/{job_id} 任务详情
- GET /api/v1/projects/{project_id}/jobs 项目任务列表
- GET /api/v1/projects/{project_id}/jobs/stats 任务统计
- PUT /api/v1/jobs/{job_id}/progress 更新进度
- POST /api/v1/jobs/{job_id}/complete 标记完成
- POST /api/v1/jobs/{job_id}/fail 标记失败
- POST /api/v1/jobs/{job_id}/retry 重试任务
- POST /api/v1/jobs/{job_id}/cancel 取消任务
- POST /api/v1/jobs/{job_id}/submit 提交执行
"""
from __future__ import annotations
import logging
from typing import Any
from app.auth import AuthenticatedUser, get_current_user
from app.core.celery_app import celery_app
from app.dependencies import get_db_session, get_job_repository, get_project_repository
from app.schemas.job import (
CompleteJobRequest,
CreateJobRequest,
FailJobRequest,
JobResponse,
JobStatisticsResponse,
ListJobsResponse,
UpdateProgressRequest,
job_to_response,
)
from fastapi import APIRouter, Depends, HTTPException, Query, status
from packages.application.jobs import (
CancelJobUseCase,
CompleteJobCommand,
CompleteJobUseCase,
CreateJobCommand,
CreateJobUseCase,
FailJobCommand,
FailJobUseCase,
GetJobStatisticsUseCase,
GetJobUseCase,
ListJobsUseCase,
RetryJobUseCase,
SubmitJobUseCase,
UpdateJobProgressCommand,
UpdateJobProgressUseCase,
)
from packages.domain.job import JobType
logger = logging.getLogger(__name__)
router = APIRouter()
# 任务类型 → Celery task name 映射
_JOB_TYPE_TO_CELERY_TASK: dict[str, str] = {
JobType.VIDEO_COMPOSE: "worker.compose_video",
JobType.RENDER_EDIT_PLAN: "worker.render_edit_plan",
JobType.ASSET_INGEST: "worker.ingest_asset",
JobType.CLASSIFICATION: "worker.classify_asset",
JobType.VOICE_EXTRACTION: "worker.extract_voice",
JobType.GENERATION: "worker.generate_video",
}
def _check_project_access(project_id: str, user_id: str, project_repository) -> None:
"""检查用户是否有项目访问权限。"""
project = project_repository.find_by_id(project_id)
if project is None:
raise HTTPException(status_code=404, detail=f"Project {project_id} not found")
if not project.can_access(user_id):
raise HTTPException(status_code=403, detail="Access denied to project")
# ── 创建任务 ──────────────────────────────────────────────────────────────────
@router.post("/jobs", response_model=JobResponse, status_code=status.HTTP_201_CREATED)
def create_job(
request: CreateJobRequest,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
job_repo: Any = Depends(get_job_repository),
project_repository: Any = Depends(get_project_repository),
) -> JobResponse:
"""创建异步任务。
创建后任务处于 pending 状态,需要调用 /submit 提交执行。
"""
_check_project_access(request.project_id, authenticated_user.user.id, project_repository)
# 校验 job_type
try:
JobType(request.job_type)
except ValueError:
raise HTTPException(
status_code=400,
detail=f"不支持的任务类型: {request.job_type}" f"可选值: {[t.value for t in JobType]}",
)
use_case = CreateJobUseCase(job_repo)
job = use_case.execute(
CreateJobCommand(
project_id=request.project_id,
job_type=request.job_type,
payload=request.payload,
source_id=request.source_id,
created_by_user_id=authenticated_user.user.id,
max_retries=request.max_retries,
)
)
return job_to_response(job)
# ── 提交执行 ──────────────────────────────────────────────────────────────────
@router.post("/jobs/{job_id}/submit", response_model=JobResponse)
def submit_job(
job_id: str,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
job_repo: Any = Depends(get_job_repository),
) -> JobResponse:
"""提交任务执行。
将任务状态从 pending 切换为 running,并 dispatch Celery 异步任务。
"""
# 权限检查:先获取任务并验证权限,再执行状态变更
job = job_repo.get(job_id)
if job is None:
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
if job.created_by_user_id and job.created_by_user_id != authenticated_user.user.id:
raise HTTPException(status_code=403, detail="Access denied to this job")
use_case = SubmitJobUseCase(job_repo)
try:
job = use_case.execute(job_id)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
# Dispatch Celery 任务
celery_task_name = _JOB_TYPE_TO_CELERY_TASK.get(job.job_type.value)
if celery_task_name:
result = celery_app.send_task(celery_task_name, args=[job.id], kwargs=job.payload)
job.celery_task_id = result.id
job_repo.update(job)
logger.info("已提交 Celery 任务: job_id=%s celery_task_id=%s", job.id, result.id)
return job_to_response(job)
# ── 查询接口 ──────────────────────────────────────────────────────────────────
@router.get("/jobs/{job_id}", response_model=JobResponse)
def get_job(
job_id: str,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
job_repo: Any = Depends(get_job_repository),
) -> JobResponse:
"""获取任务详情。"""
use_case = GetJobUseCase(job_repo)
job = use_case.execute(job_id)
if job is None:
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
return job_to_response(job)
@router.get("/projects/{project_id}/jobs", response_model=ListJobsResponse)
def list_project_jobs(
project_id: str,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
job_repo: Any = Depends(get_job_repository),
project_repository: Any = Depends(get_project_repository),
job_type: str | None = Query(default=None, description="按任务类型过滤"),
status_filter: str | None = Query(default=None, alias="status", description="按状态过滤"),
limit: int = Query(default=50, ge=1, le=200),
offset: int = Query(default=0, ge=0),
) -> ListJobsResponse:
"""获取项目下的任务列表。"""
_check_project_access(project_id, authenticated_user.user.id, project_repository)
use_case = ListJobsUseCase(job_repo)
jobs = use_case.execute(
project_id=project_id,
job_type=job_type,
status=status_filter,
limit=limit,
offset=offset,
)
items = [job_to_response(j) for j in jobs]
return ListJobsResponse(items=items, total=len(items))
@router.get("/projects/{project_id}/jobs/stats", response_model=JobStatisticsResponse)
def get_job_statistics(
project_id: str,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
job_repo: Any = Depends(get_job_repository),
project_repository: Any = Depends(get_project_repository),
) -> JobStatisticsResponse:
"""获取项目任务统计摘要。"""
_check_project_access(project_id, authenticated_user.user.id, project_repository)
use_case = GetJobStatisticsUseCase(job_repo)
stats = use_case.execute(project_id)
return JobStatisticsResponse(**stats)
# ── 进度更新 ──────────────────────────────────────────────────────────────────
@router.put("/jobs/{job_id}/progress", response_model=JobResponse)
def update_job_progress(
job_id: str,
request: UpdateProgressRequest,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
job_repo: Any = Depends(get_job_repository),
) -> JobResponse:
"""更新任务进度。"""
use_case = UpdateJobProgressUseCase(job_repo)
try:
job = use_case.execute(
UpdateJobProgressCommand(
job_id=job_id,
progress=request.progress,
current_stage=request.current_stage,
)
)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
return job_to_response(job)
# ── 完成 / 失败 ────────────────────────────────────────────────────────────────
@router.post("/jobs/{job_id}/complete", response_model=JobResponse)
def complete_job(
job_id: str,
request: CompleteJobRequest,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
job_repo: Any = Depends(get_job_repository),
) -> JobResponse:
"""标记任务完成。"""
use_case = CompleteJobUseCase(job_repo)
try:
job = use_case.execute(CompleteJobCommand(job_id=job_id, result=request.result))
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
return job_to_response(job)
@router.post("/jobs/{job_id}/fail", response_model=JobResponse)
def fail_job(
job_id: str,
request: FailJobRequest,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
job_repo: Any = Depends(get_job_repository),
) -> JobResponse:
"""标记任务失败。"""
use_case = FailJobUseCase(job_repo)
try:
job = use_case.execute(FailJobCommand(job_id=job_id, error_message=request.error_message))
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
return job_to_response(job)
# ── 重试 / 取消 ────────────────────────────────────────────────────────────────
@router.post("/jobs/{job_id}/retry", response_model=JobResponse)
def retry_job(
job_id: str,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
job_repo: Any = Depends(get_job_repository),
) -> JobResponse:
"""重试失败任务。
将任务重置为 pendingretry_count + 1,但不自动 dispatch。
需要再次调用 /submit 提交执行。
"""
# 权限检查:先获取任务并验证权限,再执行状态变更
job = job_repo.get(job_id)
if job is None:
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
if job.created_by_user_id and job.created_by_user_id != authenticated_user.user.id:
raise HTTPException(status_code=403, detail="Access denied to this job")
use_case = RetryJobUseCase(job_repo)
try:
job = use_case.execute(job_id)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
return job_to_response(job)
@router.post("/jobs/{job_id}/cancel", response_model=JobResponse)
def cancel_job(
job_id: str,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
job_repo: Any = Depends(get_job_repository),
) -> JobResponse:
"""取消任务。"""
# 权限检查:先获取任务并验证权限,再执行状态变更
job = job_repo.get(job_id)
if job is None:
raise HTTPException(status_code=404, detail=f"Job {job_id} not found")
if job.created_by_user_id and job.created_by_user_id != authenticated_user.user.id:
raise HTTPException(status_code=403, detail="Access denied to this job")
use_case = CancelJobUseCase(job_repo)
try:
job = use_case.execute(job_id)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
return job_to_response(job)
+6 -6
View File
@@ -7,7 +7,7 @@ from app.schemas.project import (
ListProjectsResponse,
ProjectResponse,
)
from fastapi import APIRouter, Depends, HTTPException, Response, status
from fastapi import APIRouter, Depends, HTTPException, status
from packages.application import (
CreateProjectCommand,
@@ -72,20 +72,20 @@ def create_project(
return _to_project_response(project)
@router.delete("/{project_id}", status_code=status.HTTP_204_NO_CONTENT, response_model=None, response_class=Response)
@router.delete("/{project_id}")
def delete_project(
project_id: str,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
) -> dict:
):
use_case = DeleteProjectUseCase(project_repository)
try:
deleted = use_case.execute(project_id, authenticated_user.user.id)
except PermissionError as _e:
except PermissionError:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Only the project owner can delete this project",
) from _e
)
if not deleted:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Project not found")
return # type: ignore[return-value]
return {"message": "Project deleted successfully"}
+212
View File
@@ -0,0 +1,212 @@
"""Recipe CRUD + use routes."""
from __future__ import annotations
from app.auth import AuthenticatedUser, get_current_user
from app.dependencies import get_db_session, get_user_repository
from app.schemas.recipe import (
CreateRecipeRequest,
ListRecipesResponse,
RecipeItemResponse,
RecipeResponse,
UpdateRecipeRequest,
UseRecipeResponse,
)
from fastapi import APIRouter, Depends, HTTPException, Query, Response, status
from sqlalchemy.orm import Session
from packages.adapters.sqlalchemy_impl.recipe_repository import SQLAlchemyRecipeRepository
from packages.application.recipe.commands import (
CreateRecipeCommand,
RecipeItemCommand,
UpdateRecipeCommand,
)
from packages.application.recipe.use_cases import (
CreateRecipeUseCase,
DeleteRecipeUseCase,
FeatureDisabledError,
GetRecipeUseCase,
ListRecipesUseCase,
NotFoundError,
UpdateRecipeUseCase,
UseRecipeUseCase,
)
from packages.ports.user_repository import UserRepository
router = APIRouter()
def _get_recipe_repository(session: Session = Depends(get_db_session)) -> SQLAlchemyRecipeRepository:
return SQLAlchemyRecipeRepository(session)
def _get_user_plan(user_id: str, user_repository: UserRepository) -> str:
user = user_repository.find_by_id(user_id)
if user is None:
return "free"
return getattr(user, "subscription_plan", "free") or "free"
def _item_to_response(item) -> RecipeItemResponse:
return RecipeItemResponse(
id=item.id,
recipe_id=item.recipe_id,
item_type=item.item_type,
item_id=item.item_id,
position=item.position,
metadata=item.metadata_,
)
def _to_response(recipe) -> RecipeResponse:
return RecipeResponse(
id=recipe.id,
user_id=recipe.user_id,
name=recipe.name,
description=recipe.description,
template_id=recipe.template_id,
generation_params=recipe.generation_params,
items=[_item_to_response(i) for i in getattr(recipe, "items", [])],
is_active=recipe.is_active,
metadata=recipe.metadata_,
created_at=recipe.created_at,
updated_at=recipe.updated_at,
)
@router.get("", response_model=ListRecipesResponse)
def list_recipes(
skip: int = Query(0, ge=0),
limit: int = Query(50, ge=1, le=200),
authenticated_user: AuthenticatedUser = Depends(get_current_user),
recipe_repository: SQLAlchemyRecipeRepository = Depends(_get_recipe_repository),
) -> ListRecipesResponse:
user_id = authenticated_user.user.id
use_case = ListRecipesUseCase(recipe_repository)
recipes = use_case.execute(user_id, skip=skip, limit=limit)
total = recipe_repository.count_by_user(user_id)
return ListRecipesResponse(
items=[_to_response(r) for r in recipes],
total=total,
)
@router.get("/{recipe_id}", response_model=RecipeResponse)
def get_recipe(
recipe_id: str,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
recipe_repository: SQLAlchemyRecipeRepository = Depends(_get_recipe_repository),
) -> RecipeResponse:
user_id = authenticated_user.user.id
use_case = GetRecipeUseCase(recipe_repository)
recipe = use_case.execute(recipe_id, user_id)
if recipe is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Recipe not found")
return _to_response(recipe)
@router.post("", response_model=RecipeResponse, status_code=status.HTTP_201_CREATED)
def create_recipe(
request: CreateRecipeRequest,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
recipe_repository: SQLAlchemyRecipeRepository = Depends(_get_recipe_repository),
) -> RecipeResponse:
user_id = authenticated_user.user.id
command = CreateRecipeCommand(
user_id=user_id,
name=request.name,
description=request.description,
template_id=request.template_id,
generation_params=request.generation_params,
items=[
RecipeItemCommand(
item_type=ic.item_type,
item_id=ic.item_id,
position=ic.position,
metadata_=ic.metadata_,
)
for ic in request.items
],
metadata_=request.metadata_,
)
use_case = CreateRecipeUseCase(recipe_repository)
recipe = use_case.execute(command)
return _to_response(recipe)
@router.patch("/{recipe_id}", response_model=RecipeResponse)
def update_recipe(
recipe_id: str,
request: UpdateRecipeRequest,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
recipe_repository: SQLAlchemyRecipeRepository = Depends(_get_recipe_repository),
) -> RecipeResponse:
user_id = authenticated_user.user.id
command = UpdateRecipeCommand(
recipe_id=recipe_id,
user_id=user_id,
name=request.name,
description=request.description,
template_id=request.template_id,
generation_params=request.generation_params,
items=(
[
RecipeItemCommand(
item_type=ic.item_type,
item_id=ic.item_id,
position=ic.position,
metadata_=ic.metadata_,
)
for ic in request.items
]
if request.items is not None
else None
),
metadata_=request.metadata_,
)
use_case = UpdateRecipeUseCase(recipe_repository)
try:
recipe = use_case.execute(command)
except NotFoundError:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Recipe not found")
return _to_response(recipe)
@router.delete("/{recipe_id}", status_code=status.HTTP_204_NO_CONTENT, response_model=None)
def delete_recipe(
recipe_id: str,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
recipe_repository: SQLAlchemyRecipeRepository = Depends(_get_recipe_repository),
) -> Response:
user_id = authenticated_user.user.id
use_case = DeleteRecipeUseCase(recipe_repository)
deleted = use_case.execute(recipe_id, user_id)
if not deleted:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Recipe not found")
return Response(status_code=204)
@router.post("/{recipe_id}/use", response_model=UseRecipeResponse)
def use_recipe(
recipe_id: str,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
recipe_repository: SQLAlchemyRecipeRepository = Depends(_get_recipe_repository),
user_repository: UserRepository = Depends(get_user_repository),
) -> UseRecipeResponse:
user_id = authenticated_user.user.id
plan_name = _get_user_plan(user_id, user_repository)
use_case = UseRecipeUseCase(recipe_repository)
try:
result = use_case.execute(recipe_id, user_id, user_plan=plan_name)
except FeatureDisabledError as exc:
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail=str(exc),
)
except NotFoundError:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Recipe not found")
return UseRecipeResponse(
recipe=_to_response(result.recipe),
warnings=[{"item_type": w.item_type, "item_id": w.item_id, "position": w.position} for w in result.warnings],
)
+8 -13
View File
@@ -2,7 +2,6 @@
from __future__ import annotations
import logging
from dataclasses import replace
from datetime import datetime, timezone
from typing import List
@@ -21,8 +20,6 @@ from fastapi import APIRouter, Depends, HTTPException, status
from packages.ports.user_repository import UserRepository
logger = logging.getLogger(__name__)
router = APIRouter()
@@ -95,7 +92,7 @@ def _build_subscription_info(user: AuthenticatedUser) -> SubscriptionInfo:
@router.get("/current", response_model=SubscriptionInfo)
async def get_current_subscription(
current_user: AuthenticatedUser = Depends(get_current_user),
) -> SubscriptionInfo:
):
"""获取当前订阅信息"""
return _build_subscription_info(current_user)
@@ -103,7 +100,7 @@ async def get_current_subscription(
@router.get("/billing-records", response_model=List[BillingRecord])
async def get_billing_records(
current_user: AuthenticatedUser = Depends(get_current_user),
) -> List[BillingRecord]:
):
"""获取账单记录列表"""
from packages.adapters.sqlalchemy_impl.billing_repository import SQLAlchemyBillingRepository
from packages.adapters.sqlalchemy_impl.session import SessionLocal
@@ -137,7 +134,7 @@ async def change_plan(
request: ChangePlanRequest,
current_user: AuthenticatedUser = Depends(get_current_user),
user_repository: UserRepository = Depends(get_user_repository),
) -> ChangePlanResponse:
):
"""变更订阅套餐(升级/降级)"""
# TODO: 接入支付验证(支付宝/微信支付)
valid_plans = {"free", "standard", "pro", "enterprise"}
@@ -189,7 +186,7 @@ async def change_plan(
async def cancel_subscription(
current_user: AuthenticatedUser = Depends(get_current_user),
user_repository: UserRepository = Depends(get_user_repository),
) -> SimpleResponse:
):
"""取消订阅"""
user = current_user.user
if user.subscription_plan == "free":
@@ -215,7 +212,7 @@ async def payment_callback(
amount: float,
payment_method: str = "alipay",
payment_id: str = "",
) -> dict:
):
"""支付回调 - 在事务中更新账单和订阅状态
注意:生产环境需要验证支付签名
@@ -235,7 +232,7 @@ async def payment_callback(
# 创建账单记录
record_id = uuid.uuid4().hex
repo.create(
record = repo.create(
{
"id": record_id,
"user_id": user_id,
@@ -257,9 +254,7 @@ async def payment_callback(
return {"success": True, "message": "支付成功", "record_id": record_id}
except Exception as e:
session.rollback()
logger.error(f"支付回调处理失败: user_id={user_id}, plan={plan}, error={e}")
# 不返回原始异常信息,避免泄漏内部实现细节
raise HTTPException(status_code=500, detail="支付处理失败,请稍后重试") from e
raise HTTPException(status_code=500, detail=f"支付处理失败: {str(e)}")
finally:
session.close()
@@ -268,7 +263,7 @@ async def payment_callback(
async def toggle_auto_renew(
request: ToggleAutoRenewRequest,
current_user: AuthenticatedUser = Depends(get_current_user),
) -> SimpleResponse:
):
"""切换自动续费"""
# TODO: 实际需要在数据库中存储 auto_renew 字段
status_text = "已开启自动续费" if request.enabled else "已关闭自动续费"
+2 -2
View File
@@ -10,7 +10,7 @@ from app.schemas.tag import (
ListTagsResponse,
TagResponse,
)
from fastapi import APIRouter, Depends, HTTPException, Response
from fastapi import APIRouter, Depends, HTTPException
from packages.domain import Tag
@@ -52,7 +52,7 @@ def create_tag(
return TagResponse(id=created.id, name=created.name, created_at=created.created_at)
@router.delete("/{tag_id}", status_code=204, response_class=Response)
@router.delete("/{tag_id}", status_code=204)
def delete_tag(
tag_id: str,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
+87 -211
View File
@@ -1,16 +1,7 @@
import logging
from typing import Any
from app.api.routes._helpers import format_utc_datetime
from app.auth import AuthenticatedUser, get_current_user
from app.core.celery_app import celery_app
from app.core.task_enqueue import (
GLOBAL_PENDING_LIMIT,
USER_PENDING_LIMIT,
GlobalQueueFull,
UserPendingLimitExceeded,
safe_enqueue_generation_task,
)
from app.dependencies import (
get_generation_task_repository,
get_ingest_job_repository,
@@ -22,21 +13,17 @@ from app.schemas.task_center import (
ProjectTaskResponse,
UserTaskResponse,
)
from fastapi import APIRouter, Depends, HTTPException, Query
from fastapi import APIRouter, Depends, HTTPException
from packages.application import (
RetryGenerationTaskUseCase,
CreateGenerationTaskCommand,
CreateGenerationTaskUseCase,
SubmitIngestJobCommand,
SubmitIngestJobUseCase,
)
logger = logging.getLogger(__name__)
router = APIRouter()
DEFAULT_PAGE_SIZE = 50
MAX_PAGE_SIZE = 200
def _humanize_task_error(error_message: str) -> str:
raw = (error_message or "").strip()
@@ -67,8 +54,6 @@ def _generation_step(task) -> str:
return "生成完成"
if s == "failed":
return "生成失败"
if s == "cancelled":
return "已取消"
return s
@@ -85,26 +70,6 @@ def _ingest_step(job) -> str:
return s
def _generation_task_to_user_response(task) -> UserTaskResponse:
return UserTaskResponse(
id=f"generation:{task.id}",
task_type="generation",
project_id=task.project_id,
template_id=task.template_id,
status=_status_value(task.status),
progress=task.progress,
current_step=_generation_step(task),
error_message=task.error_message,
error_info=task.error_info or {},
user_message=_humanize_task_error(task.error_message),
retryable=_status_value(task.status) == "failed",
retry_count=task.retry_count or 0,
source_id=task.id,
created_at=format_utc_datetime(task.created_at),
updated_at=format_utc_datetime(task.completed_at or task.started_at or task.created_at),
)
def _generation_task_to_project_response(task) -> ProjectTaskResponse:
return ProjectTaskResponse(
id=f"generation:{task.id}",
@@ -114,77 +79,49 @@ def _generation_task_to_project_response(task) -> ProjectTaskResponse:
progress=task.progress,
current_step=_generation_step(task),
error_message=task.error_message,
error_info=task.error_info or {},
user_message=_humanize_task_error(task.error_message),
retryable=_status_value(task.status) == "failed",
retry_count=task.retry_count or 0,
source_id=task.id,
template_id=task.template_id,
created_at=format_utc_datetime(task.created_at),
updated_at=format_utc_datetime(task.completed_at or task.started_at or task.created_at),
created_at=task.created_at,
updated_at=task.completed_at or task.started_at or task.created_at,
)
def _validate_status(status: str | None) -> str | None:
"""校验状态值合法性。"""
if status is None:
return None
valid = {"pending", "running", "completed", "failed", "cancelled"}
if status not in valid:
raise HTTPException(
status_code=400,
detail=f"无效的状态筛选值: {status},允许值: {', '.join(sorted(valid))}",
)
return status
def _clamp_page_size(page_size: int) -> int:
if page_size <= 0:
return DEFAULT_PAGE_SIZE
if page_size > MAX_PAGE_SIZE:
return MAX_PAGE_SIZE
return page_size
# ── 用户级端点(放在项目级端点之前,避免路由冲突) ──
@router.get("/tasks", response_model=ListTasksResponse)
def list_user_tasks(
status: str | None = Query(None, description="按状态筛选:pending/running/completed/failed/cancelled"),
task_type: str | None = Query(None, description="按任务类型筛选:generation/ingest"),
page: int = Query(1, ge=1, description="页码,从1开始"),
page_size: int = Query(DEFAULT_PAGE_SIZE, ge=1, le=MAX_PAGE_SIZE, description="每页数量"),
authenticated_user: AuthenticatedUser = Depends(get_current_user),
ingest_job_repository: Any = Depends(get_ingest_job_repository),
generation_task_repository: Any = Depends(get_generation_task_repository),
) -> ListTasksResponse:
"""用户级任务列表(跨 project),支持状态/类型筛选和分页"""
status = _validate_status(status)
page_size = _clamp_page_size(page_size)
"""用户级任务列表(跨 project),合并 ingest + generation 任务"""
user_id = authenticated_user.user.id
offset = (page - 1) * page_size
items: list[UserTaskResponse] = []
# 生成任务
if task_type is None or task_type == "generation":
gen_result = generation_task_repository.list_by_user_filtered(
user_id,
status=status,
limit=page_size + 1, # 多取一条判断是否还有下一页(简单起见这里用offset)
offset=offset,
for task in generation_task_repository.list_by_user(user_id):
items.append(
UserTaskResponse(
id=f"generation:{task.id}",
task_type="generation",
project_id=task.project_id,
template_id=task.template_id,
status=_status_value(task.status),
progress=task.progress,
current_step=_generation_step(task),
error_message=task.error_message,
user_message=_humanize_task_error(task.error_message),
retryable=_status_value(task.status) == "failed",
source_id=task.id,
created_at=task.created_at,
updated_at=task.completed_at or task.started_at or task.created_at,
)
)
for task in gen_result:
items.append(_generation_task_to_user_response(task))
# 按时间倒序
items.sort(key=lambda item: item.updated_at or item.created_at or "", reverse=True)
# 总数(仅generation,ingest暂不计入总数以保持简单)
total = generation_task_repository.count_by_user_filtered(user_id, status=status)
return ListTasksResponse(items=items[:page_size], total=total)
return ListTasksResponse(items=items)
@router.post("/tasks/{task_id}/retry", response_model=UserTaskResponse)
@@ -193,7 +130,7 @@ def retry_task_by_id(
authenticated_user: AuthenticatedUser = Depends(get_current_user),
generation_task_repository: Any = Depends(get_generation_task_repository),
) -> UserTaskResponse:
"""原地重试失败的生成任务(复用同一个task_idretry_count+1"""
"""简化重试:通过 task_id 直接重试失败的生成任务"""
task = generation_task_repository.get(task_id)
if task is None:
raise HTTPException(status_code=404, detail="Generation task not found")
@@ -202,44 +139,33 @@ def retry_task_by_id(
if _status_value(task.status) != "failed":
raise HTTPException(status_code=409, detail="Only failed tasks can be retried")
user_id = authenticated_user.user.id
# 预检查
user_pending = generation_task_repository.count_pending_by_user(user_id)
global_pending = generation_task_repository.count_pending_total()
if user_pending >= USER_PENDING_LIMIT:
raise HTTPException(
status_code=429,
detail=f"您的待处理任务过多(当前 {user_pending}/{USER_PENDING_LIMIT}),请等待完成后再提交",
use_case = CreateGenerationTaskUseCase(generation_task_repository)
retried = use_case.execute(
CreateGenerationTaskCommand(
project_id=task.project_id,
asset_library_id=task.asset_library_id,
strategy_id=task.strategy_id,
voice_library_id=task.voice_library_id,
template_id=task.template_id,
asset_ids=task.asset_ids,
title_ids=task.title_ids,
voice_ids=task.voice_ids,
created_by_user_id=authenticated_user.user.id,
)
if global_pending >= GLOBAL_PENDING_LIMIT:
raise HTTPException(
status_code=503,
detail="系统繁忙,请稍后再试",
)
# 原地重试
use_case = RetryGenerationTaskUseCase(generation_task_repository)
retried = use_case.execute(task_id)
# 重新入队
try:
if not safe_enqueue_generation_task(
retried, generation_task_repository, user_id=user_id, log_prefix="[任务中心]"
):
logger.warning("[任务中心] 用户级重试入队失败: task_id=%s", retried.id)
except UserPendingLimitExceeded:
raise HTTPException(
status_code=429,
detail="您的待处理任务过多,请等待完成后再提交",
) from None
except GlobalQueueFull:
raise HTTPException(
status_code=503,
detail="系统繁忙,请稍后再试",
) from None
return _generation_task_to_user_response(retried)
)
celery_app.send_task("worker.generate_video", args=[retried.id])
return UserTaskResponse(
id=f"generation:{retried.id}",
task_type="generation",
project_id=retried.project_id,
template_id=retried.template_id,
status=_status_value(retried.status),
progress=retried.progress,
current_step=_generation_step(retried),
source_id=retried.id,
created_at=retried.created_at,
updated_at=retried.created_at,
)
# ── 项目级端点 ──
@@ -248,64 +174,37 @@ def retry_task_by_id(
@router.get("/projects/{project_id}/tasks", response_model=ListProjectTasksResponse)
def list_project_tasks(
project_id: str,
status: str | None = Query(None, description="按状态筛选:pending/running/completed/failed/cancelled"),
task_type: str | None = Query(None, description="按任务类型筛选:generation/ingest"),
page: int = Query(1, ge=1, description="页码,从1开始"),
page_size: int = Query(DEFAULT_PAGE_SIZE, ge=1, le=MAX_PAGE_SIZE, description="每页数量"),
authenticated_user: AuthenticatedUser = Depends(get_current_user),
project_repository: Any = Depends(get_project_repository),
ingest_job_repository: Any = Depends(get_ingest_job_repository),
generation_task_repository: Any = Depends(get_generation_task_repository),
) -> ListProjectTasksResponse:
"""项目级任务列表,支持状态/类型筛选和分页。"""
project = project_repository.find_by_id(project_id)
if project is None:
raise HTTPException(status_code=404, detail="Project not found")
status = _validate_status(status)
page_size = _clamp_page_size(page_size)
offset = (page - 1) * page_size
items: list[ProjectTaskResponse] = []
# 导入任务
if task_type is None or task_type == "ingest":
for job in ingest_job_repository.list_by_project(project_id):
if status and _status_value(job.status) != status:
continue
items.append(
ProjectTaskResponse(
id=f"ingest:{job.id}",
task_type="ingest",
project_id=job.project_id,
status=_status_value(job.status),
progress=100.0 if _status_value(job.status) == "completed" else 0.0,
current_step=_ingest_step(job),
error_message=job.error_message,
user_message=_humanize_task_error(job.error_message),
retryable=_status_value(job.status) == "failed",
source_id=job.id,
created_at=format_utc_datetime(job.created_at),
updated_at=format_utc_datetime(job.updated_at),
)
for job in ingest_job_repository.list_by_project(project_id):
items.append(
ProjectTaskResponse(
id=f"ingest:{job.id}",
task_type="ingest",
project_id=job.project_id,
status=_status_value(job.status),
progress=100.0 if _status_value(job.status) == "completed" else 0.0,
current_step=_ingest_step(job),
error_message=job.error_message,
user_message=_humanize_task_error(job.error_message),
retryable=_status_value(job.status) == "failed",
source_id=job.id,
created_at=job.created_at,
updated_at=job.updated_at,
)
# 生成任务
if task_type is None or task_type == "generation":
gen_items = generation_task_repository.list_by_project_filtered(
project_id,
status=status,
limit=page_size + 1,
offset=offset,
)
for task in gen_items:
items.append(_generation_task_to_project_response(task))
for task in generation_task_repository.list_by_project(project_id):
items.append(_generation_task_to_project_response(task))
items.sort(key=lambda item: item.updated_at or item.created_at or "", reverse=True)
total = generation_task_repository.count_by_project_filtered(project_id, status=status)
return ListProjectTasksResponse(items=items[:page_size], total=total)
return ListProjectTasksResponse(items=items)
@router.post("/tasks/{task_type}/{source_id}/retry", response_model=ProjectTaskResponse)
@@ -316,60 +215,37 @@ def retry_project_task(
ingest_job_repository: Any = Depends(get_ingest_job_repository),
generation_task_repository: Any = Depends(get_generation_task_repository),
) -> ProjectTaskResponse:
"""项目级任务重试。"""
if task_type == "generation":
task = generation_task_repository.get(source_id)
if task is None:
raise HTTPException(status_code=404, detail="Generation task not found")
if _status_value(task.status) != "failed":
raise HTTPException(status_code=409, detail="Only failed tasks can be retried")
user_id = authenticated_user.user.id
# 预检查
user_pending = generation_task_repository.count_pending_by_user(user_id)
global_pending = generation_task_repository.count_pending_total()
if user_pending >= USER_PENDING_LIMIT:
raise HTTPException(
status_code=429,
detail=f"您的待处理任务过多(当前 {user_pending}/{USER_PENDING_LIMIT}),请等待完成后再提交",
use_case = CreateGenerationTaskUseCase(generation_task_repository)
retried = use_case.execute(
CreateGenerationTaskCommand(
project_id=task.project_id,
asset_library_id=task.asset_library_id,
strategy_id=task.strategy_id,
voice_library_id=task.voice_library_id,
template_id=task.template_id,
asset_ids=task.asset_ids,
title_ids=task.title_ids,
voice_ids=task.voice_ids,
created_by_user_id=authenticated_user.user.id,
)
if global_pending >= GLOBAL_PENDING_LIMIT:
raise HTTPException(
status_code=503,
detail="系统繁忙,请稍后再试",
)
# 原地重试
use_case = RetryGenerationTaskUseCase(generation_task_repository)
retried = use_case.execute(source_id)
try:
if not safe_enqueue_generation_task(
retried, generation_task_repository, user_id=user_id, log_prefix="[任务中心]"
):
logger.warning("[任务中心] 项目级重试入队失败: task_id=%s", retried.id)
except UserPendingLimitExceeded:
raise HTTPException(
status_code=429,
detail="您的待处理任务过多,请等待完成后再提交",
) from None
except GlobalQueueFull:
raise HTTPException(
status_code=503,
detail="系统繁忙,请稍后再试",
) from None
)
celery_app.send_task("worker.generate_video", args=[retried.id])
return _generation_task_to_project_response(retried)
if task_type == "ingest":
job = ingest_job_repository.get(source_id)
if job is None:
raise HTTPException(status_code=404, detail="Ingest job not found")
if _status_value(job.status) != "failed":
raise HTTPException(status_code=409, detail="Only failed tasks can be retried")
use_case = SubmitIngestJobUseCase(ingest_job_repository) # type: ignore[assignment]
use_case = SubmitIngestJobUseCase(ingest_job_repository)
retried = use_case.execute(
SubmitIngestJobCommand( # type: ignore[arg-type]
SubmitIngestJobCommand(
project_id=job.project_id,
library_id=job.library_id,
storage_key=job.storage_key,
@@ -384,7 +260,7 @@ def retry_project_task(
progress=0,
current_step=_ingest_step(retried),
source_id=retried.id,
created_at=format_utc_datetime(retried.created_at),
updated_at=format_utc_datetime(retried.updated_at), # type: ignore[attr-defined]
created_at=retried.created_at,
updated_at=retried.updated_at,
)
raise HTTPException(status_code=400, detail="Unsupported task type")
+19 -108
View File
@@ -8,16 +8,13 @@ from app.auth import AuthenticatedUser, get_current_user
from app.dependencies import get_db_session
from app.schemas.template import (
CategoryResponse,
CopyTemplateRequest,
CreateCategoryRequest,
CreateTemplateRequest,
GenerateWarningResponse,
ListCategoriesResponse,
ListTagsResponse,
ListTemplatesResponse,
SegmentResponse,
TemplateResponse,
TemplateUsageResponse,
ToggleFavoriteResponse,
UpdateTemplateRequest,
ValidateTemplateRequest,
@@ -30,24 +27,19 @@ logger = logging.getLogger(__name__)
from packages.adapters.sqlalchemy_impl.template_repository import SQLAlchemyTemplateRepository
from packages.application.template.commands import (
CopyTemplateCommand,
CreateCategoryCommand,
CreateTemplateCommand,
ListTemplatesFilter,
SegmentCommand,
UpdateTemplateCommand,
ValidateTemplateCommand,
)
from packages.application.template.use_cases import (
CopyTemplateUseCase,
CountTemplatesUseCase,
CreateCategoryUseCase,
CreateTemplateUseCase,
DeleteCategoryUseCase,
DeleteTemplateUseCase,
GetTemplateUseCase,
ListCategoriesUseCase,
ListTagsUseCase,
ListTemplatesUseCase,
NotFoundError,
UpdateTemplateUseCase,
@@ -75,7 +67,7 @@ def _segment_to_response(seg) -> SegmentResponse:
)
def _to_response(template, usage_count: int = 0) -> TemplateResponse:
def _to_response(template) -> TemplateResponse:
return TemplateResponse(
id=template.id,
user_id=template.user_id,
@@ -89,7 +81,6 @@ def _to_response(template, usage_count: int = 0) -> TemplateResponse:
estimated_duration=template.estimated_duration,
segments=[_segment_to_response(s) for s in getattr(template, "segments", [])],
is_active=template.is_active,
usage_count=usage_count,
created_at=template.created_at,
updated_at=template.updated_at,
)
@@ -102,36 +93,19 @@ def _to_response(template, usage_count: int = 0) -> TemplateResponse:
def list_templates(
skip: int = Query(0, ge=0),
limit: int = Query(50, ge=1, le=200),
category: str | None = Query(None, description="按分类筛选"),
tag: str | None = Query(None, description="按标签筛选"),
keyword: str | None = Query(None, description="按名称关键词搜索"),
mode: str | None = Query(None, description="按剪辑模式筛选"),
authenticated_user: AuthenticatedUser = Depends(get_current_user),
template_repository: SQLAlchemyTemplateRepository = Depends(_get_template_repository),
) -> ListTemplatesResponse:
user_id = authenticated_user.user.id
try:
tpl_filter = ListTemplatesFilter(
category=category,
tag=tag,
keyword=keyword,
mode=mode,
)
use_case = ListTemplatesUseCase(template_repository)
templates = use_case.execute(user_id, skip=skip, limit=limit, filter=tpl_filter)
count_use_case = CountTemplatesUseCase(template_repository)
total = count_use_case.execute(user_id, filter=tpl_filter)
# 批量查询使用次数
items = []
for t in templates:
usage = template_repository.get_usage_count(t.id)
items.append(_to_response(t, usage_count=usage))
templates = use_case.execute(user_id, skip=skip, limit=limit)
total = template_repository.count_by_user(user_id)
except Exception:
logger.exception("list_templates 查询失败: user_id=%s", user_id)
return ListTemplatesResponse(items=[], total=0)
return ListTemplatesResponse(
items=items,
items=[_to_response(t) for t in templates],
total=total,
)
@@ -146,13 +120,12 @@ def get_template(
try:
use_case = GetTemplateUseCase(template_repository)
template = use_case.execute(template_id, user_id)
usage = template_repository.get_usage_count(template_id)
except Exception as _e:
except Exception:
logger.exception("get_template 查询失败: template_id=%s", template_id)
raise HTTPException(status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, detail="模板查询失败") from _e
raise HTTPException(status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, detail="模板查询失败")
if template is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Template not found")
return _to_response(template, usage_count=usage)
return _to_response(template)
@router.post("", response_model=TemplateResponse, status_code=status.HTTP_201_CREATED)
@@ -186,7 +159,7 @@ def create_template(
try:
template = use_case.execute(command)
except ValidationError as exc:
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc)) from exc
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc))
return _to_response(template)
@@ -226,14 +199,14 @@ def update_template(
use_case = UpdateTemplateUseCase(template_repository)
try:
template = use_case.execute(command)
except NotFoundError as _e:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Template not found") from _e
except NotFoundError:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Template not found")
except ValidationError as exc:
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc)) from exc
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc))
return _to_response(template)
@router.delete("/{template_id}", status_code=status.HTTP_204_NO_CONTENT, response_model=None, response_class=Response)
@router.delete("/{template_id}", status_code=status.HTTP_204_NO_CONTENT, response_model=None)
def delete_template(
template_id: str,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
@@ -244,48 +217,7 @@ def delete_template(
deleted = use_case.execute(template_id, user_id)
if not deleted:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Template not found")
return
@router.post("/{template_id}/copy", response_model=TemplateResponse, status_code=status.HTTP_201_CREATED)
def copy_template(
template_id: str,
request: CopyTemplateRequest,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
template_repository: SQLAlchemyTemplateRepository = Depends(_get_template_repository),
) -> TemplateResponse:
"""复制模板(含所有片段配置)"""
user_id = authenticated_user.user.id
command = CopyTemplateCommand(
template_id=template_id,
user_id=user_id,
new_name=request.new_name,
)
use_case = CopyTemplateUseCase(template_repository)
try:
template = use_case.execute(command)
except NotFoundError as _e:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Template not found") from _e
except ValidationError as exc:
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc)) from exc
return _to_response(template)
@router.get("/{template_id}/usage", response_model=TemplateUsageResponse)
def get_template_usage(
template_id: str,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
template_repository: SQLAlchemyTemplateRepository = Depends(_get_template_repository),
) -> TemplateUsageResponse:
"""获取模板使用次数(关联的剪辑计划数量)"""
user_id = authenticated_user.user.id
# 鉴权:确保模板存在且属于当前用户
use_case = GetTemplateUseCase(template_repository)
template = use_case.execute(template_id, user_id)
if template is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Template not found")
usage = template_repository.get_usage_count(template_id)
return TemplateUsageResponse(template_id=template_id, usage_count=usage)
return Response(status_code=204)
@router.post("/{template_id}/toggle-favorite", response_model=ToggleFavoriteResponse)
@@ -299,9 +231,9 @@ def toggle_favorite(
use_case = GetTemplateUseCase(template_repository)
try:
template = use_case.execute(template_id, user_id)
except Exception as _e:
except Exception:
logger.exception("toggle_favorite 查询失败: template_id=%s", template_id)
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Template not found") from _e
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Template not found")
if template is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Template not found")
return ToggleFavoriteResponse(id=template_id, is_favorite=False)
@@ -326,10 +258,10 @@ def validate_template(
use_case = ValidateTemplateUseCase(template_repository)
try:
result = use_case.execute(command)
except NotFoundError as _e:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Template not found") from _e
except NotFoundError:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Template not found")
except ValidationError as exc:
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc)) from exc
raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc))
return ValidateTemplateResponse(
template=_to_response(result.template),
@@ -375,9 +307,7 @@ def create_category(
)
@router.delete(
"/categories/{category_id}", status_code=status.HTTP_204_NO_CONTENT, response_model=None, response_class=Response
)
@router.delete("/categories/{category_id}", status_code=status.HTTP_204_NO_CONTENT, response_model=None)
def delete_category(
category_id: str,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
@@ -389,22 +319,3 @@ def delete_category(
if not deleted:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Category not found")
return Response(status_code=204)
# ── Tags ──
@router.get("/tags/list", response_model=ListTagsResponse)
def list_tags(
authenticated_user: AuthenticatedUser = Depends(get_current_user),
template_repository: SQLAlchemyTemplateRepository = Depends(_get_template_repository),
) -> ListTagsResponse:
"""获取用户所有模板标签(去重排序)"""
user_id = authenticated_user.user.id
try:
use_case = ListTagsUseCase(template_repository)
tags = use_case.execute(user_id)
except Exception:
logger.exception("list_tags 查询失败: user_id=%s", user_id)
return ListTagsResponse(items=[])
return ListTagsResponse(items=tags)
+14 -48
View File
@@ -4,7 +4,6 @@ from __future__ import annotations
from typing import Optional
from app.api.routes._helpers import get_user_plan
from app.auth import AuthenticatedUser, get_current_user
from app.dependencies import get_db_session, get_user_repository
from app.schemas.title_library import (
@@ -17,18 +16,13 @@ from fastapi import APIRouter, Depends, HTTPException, Query, Response, status
from sqlalchemy.orm import Session
from packages.adapters.sqlalchemy_impl.title_library_repository import SQLAlchemyTitleLibraryRepository
from packages.application.title_library.commands import (
CreateTitleLibraryCommand,
PickTitleCommand,
UpdateTitleLibraryCommand,
)
from packages.application.title_library.commands import CreateTitleLibraryCommand, UpdateTitleLibraryCommand
from packages.application.title_library.use_cases import (
CreateTitleLibraryUseCase,
DeleteTitleLibraryUseCase,
GetTitleLibraryUseCase,
ListTitleLibraryUseCase,
NotFoundError,
PickTitleUseCase,
QuotaExceededError,
UpdateTitleLibraryUseCase,
)
@@ -57,6 +51,13 @@ def _to_response(item) -> TitleLibraryItemResponse:
)
def _get_user_plan(user_id: str, user_repository: UserRepository) -> str:
user = user_repository.find_by_id(user_id)
if user is None:
return "free"
return getattr(user, "subscription_plan", "free") or "free"
@router.get("", response_model=ListTitleLibraryResponse)
def list_titles(
category: Optional[str] = Query(None),
@@ -75,41 +76,6 @@ def list_titles(
)
@router.post("/pick", response_model=TitleLibraryItemResponse)
def pick_title(
category: Optional[str] = Query(None, description="按分类筛选,不填则从全部标题中选"),
exclude_ids: Optional[str] = Query(
None,
description="排除的标题ID(逗号分隔),用于批量生成时避免重复",
),
authenticated_user: AuthenticatedUser = Depends(get_current_user),
title_repository: SQLAlchemyTitleLibraryRepository = Depends(_get_title_repository),
) -> TitleLibraryItemResponse:
"""智能选择一个标题。
策略:优先使用次数少的,从最少的前5个中随机选一个,兼顾公平和多样性。
"""
user_id = authenticated_user.user.id
exclude_list: list[str] = []
if exclude_ids:
exclude_list = [t.strip() for t in exclude_ids.split(",") if t.strip()]
use_case = PickTitleUseCase(title_repository)
item = use_case.execute(
PickTitleCommand(
user_id=user_id,
category=category,
exclude_ids=exclude_list,
)
)
if item is None:
raise HTTPException(
status_code=status.HTTP_404_NOT_FOUND,
detail="标题库为空,请先添加标题",
)
return _to_response(item)
@router.get("/{title_id}", response_model=TitleLibraryItemResponse)
def get_title(
title_id: str,
@@ -132,7 +98,7 @@ def create_title(
user_repository: UserRepository = Depends(get_user_repository),
) -> TitleLibraryItemResponse:
user_id = authenticated_user.user.id
plan_name = get_user_plan(user_id, user_repository)
plan_name = _get_user_plan(user_id, user_repository)
command = CreateTitleLibraryCommand(
user_id=user_id,
name=request.name,
@@ -148,7 +114,7 @@ def create_title(
raise HTTPException(
status_code=status.HTTP_429_TOO_MANY_REQUESTS,
detail=f"标题库配额已满({exc.used}/{exc.limit}),请升级套餐",
) from exc
)
return _to_response(item)
@@ -172,12 +138,12 @@ def update_title(
use_case = UpdateTitleLibraryUseCase(title_repository)
try:
item = use_case.execute(command)
except NotFoundError as _e:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Title not found") from _e
except NotFoundError:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Title not found")
return _to_response(item)
@router.delete("/{title_id}", status_code=status.HTTP_204_NO_CONTENT, response_model=None, response_class=Response)
@router.delete("/{title_id}", status_code=status.HTTP_204_NO_CONTENT, response_model=None)
def delete_title(
title_id: str,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
@@ -188,4 +154,4 @@ def delete_title(
deleted = use_case.execute(title_id, user_id)
if not deleted:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Title not found")
return
return Response(status_code=204)
Executable → Regular
+18 -58
View File
@@ -7,7 +7,6 @@ from typing import Optional
from app.auth import AuthenticatedUser, get_current_user
from app.dependencies import (
get_audio_url_signer,
get_cosyvoice_service,
get_db_session,
get_user_repository,
@@ -46,7 +45,6 @@ from packages.application.voice_library.use_cases import (
CreateVoiceLibraryUseCase,
QuotaExceededError,
)
from packages.domain.voice_presets import list_voices
from packages.ports.user_repository import UserRepository
logger = logging.getLogger(__name__)
@@ -54,42 +52,11 @@ logger = logging.getLogger(__name__)
router = APIRouter()
@router.get("/presets", summary="获取预设音色列表")
def list_preset_voices(
gender: Optional[str] = Query(None, description="按性别筛选: male/female/child"),
style: Optional[str] = Query(None, description="按风格筛选: stable/lively/customer_service/narration/news/story"),
keyword: Optional[str] = Query(None, description="按关键词搜索"),
_user: AuthenticatedUser = Depends(get_current_user),
) -> list[dict]:
"""获取可用的预设音色列表。
用于配音功能的音色选择。
"""
voices = list_voices(gender=gender, style=style, keyword=keyword)
return [
{
"voice_id": v.voice_id,
"name": v.name,
"gender": v.gender.value,
"style": v.style.value,
"description": v.description,
"default_speed": v.default_speed,
"default_pitch": v.default_pitch,
"sample_rate": v.sample_rate,
"language": v.language,
}
for v in voices
]
def _get_repository(session: Session = Depends(get_db_session)) -> SQLAlchemyTTSJobRepository:
return SQLAlchemyTTSJobRepository(session)
def _to_response(job, sign_url=None) -> TTSJobResponse:
output_url = job.output_audio_url
if sign_url and output_url:
output_url = sign_url(output_url)
def _to_response(job) -> TTSJobResponse:
return TTSJobResponse(
id=job.id,
user_id=job.user_id,
@@ -99,7 +66,7 @@ def _to_response(job, sign_url=None) -> TTSJobResponse:
project_id=job.project_id,
voice_clone_profile_id=job.voice_clone_profile_id,
status=job.status,
output_audio_url=output_url,
output_audio_url=job.output_audio_url,
output_audio_key=job.output_audio_key,
duration=job.duration,
file_size=job.file_size,
@@ -209,7 +176,6 @@ def list_tts_jobs(
status_filter: Optional[str] = Query(None, alias="status"),
authenticated_user: AuthenticatedUser = Depends(get_current_user),
repository: SQLAlchemyTTSJobRepository = Depends(_get_repository),
sign_url=Depends(get_audio_url_signer),
) -> ListTTSJobResponse:
"""列出用户的 TTS 合成任务。"""
user_id = authenticated_user.user.id
@@ -217,7 +183,7 @@ def list_tts_jobs(
skip = (page - 1) * page_size
items, total = use_case.execute(user_id, status=status_filter, skip=skip, limit=page_size)
return ListTTSJobResponse(
items=[_to_response(j, sign_url) for j in items],
items=[_to_response(j) for j in items],
total=total,
page=page,
page_size=page_size,
@@ -229,16 +195,15 @@ def get_tts_job(
job_id: str,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
repository: SQLAlchemyTTSJobRepository = Depends(_get_repository),
sign_url=Depends(get_audio_url_signer),
) -> TTSJobResponse:
"""获取 TTS 任务详情。"""
user_id = authenticated_user.user.id
use_case = GetTTSJobUseCase(repository)
try:
job = use_case.execute(job_id, user_id)
except TTSJobNotFoundError as _e:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="TTS job not found") from _e
return _to_response(job, sign_url)
except TTSJobNotFoundError:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="TTS job not found")
return _to_response(job)
@router.get("/jobs/{job_id}/status", response_model=TTSStatusResponse)
@@ -246,22 +211,18 @@ def get_tts_job_status(
job_id: str,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
repository: SQLAlchemyTTSJobRepository = Depends(_get_repository),
sign_url=Depends(get_audio_url_signer),
) -> TTSStatusResponse:
"""查询 TTS 合成状态(用于前端轮询)。"""
user_id = authenticated_user.user.id
use_case = GetTTSJobStatusUseCase(repository)
try:
job = use_case.execute(job_id, user_id)
except TTSJobNotFoundError as _e:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="TTS job not found") from _e
output_url = job.output_audio_url
if output_url:
output_url = sign_url(output_url)
except TTSJobNotFoundError:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="TTS job not found")
return TTSStatusResponse(
id=job.id,
status=job.status,
output_audio_url=output_url,
output_audio_url=job.output_audio_url,
error_message=job.error_message,
duration=job.duration,
retry_count=job.retry_count,
@@ -270,7 +231,7 @@ def get_tts_job_status(
)
@router.delete("/jobs/{job_id}", status_code=status.HTTP_204_NO_CONTENT, response_model=None, response_class=Response)
@router.delete("/jobs/{job_id}", status_code=status.HTTP_204_NO_CONTENT, response_model=None)
def delete_tts_job(
job_id: str,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
@@ -282,7 +243,7 @@ def delete_tts_job(
deleted = use_case.execute(job_id, user_id)
if not deleted:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="TTS job not found")
return
return Response(status_code=204)
@router.post(
@@ -297,7 +258,6 @@ def save_tts_job_to_library(
tts_repository: SQLAlchemyTTSJobRepository = Depends(_get_repository),
voice_library_repository: SQLAlchemyVoiceLibraryRepository = Depends(get_voice_library_repository),
user_repository: UserRepository = Depends(get_user_repository),
sign_url=Depends(get_audio_url_signer),
) -> SaveToLibraryResponse:
"""将已完成的 TTS 合成结果保存到配音库。
@@ -309,8 +269,8 @@ def save_tts_job_to_library(
get_use_case = GetTTSJobUseCase(tts_repository)
try:
job = get_use_case.execute(job_id, user_id)
except TTSJobNotFoundError as _e:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="TTS job not found") from _e
except TTSJobNotFoundError:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="TTS job not found")
# 校验已完成
if not job.is_completed:
@@ -363,12 +323,12 @@ def save_tts_job_to_library(
raise HTTPException(
status_code=status.HTTP_429_TOO_MANY_REQUESTS,
detail=f"配音库配额已满({exc.used}/{exc.limit}),请升级套餐",
) from exc
)
return SaveToLibraryResponse(
id=item.id,
name=item.name,
audio_url=sign_url(item.audio_url) if item.audio_url else "",
audio_url=item.audio_url,
duration=item.duration,
voice_id=item.voice_id,
voice_name=item.voice_name,
@@ -401,10 +361,10 @@ async def tts_websocket_stream(
streaming_service = TTSStreamingService(cosyvoice_service)
await streaming_service.synthesize_and_stream(websocket, params)
except WebSocketDisconnect:
logger.info("WebSocket 客户端主动断开连接")
logger.info("WebSocket 客户端断开连接")
except Exception as e:
logger.error(f"WebSocket 流式合成异常: {e}", exc_info=True)
try:
await websocket.send_json({"type": "error", "message": f"服务异常: {e}"})
except Exception as send_err:
logger.warning("WebSocket 错误消息发送失败(连接可能已断开): %s", send_err)
except Exception:
pass
+21 -6
View File
@@ -1,8 +1,7 @@
import logging
from typing import Any
from typing import Annotated, Any
from uuid import uuid4
from app.api.routes._helpers import require_project_and_library
from app.auth import AuthenticatedUser, get_current_user
from app.config import get_settings
from app.core.celery_app import celery_app
@@ -18,11 +17,12 @@ from app.schemas.upload import (
DirectUploadCompleteResponse,
DirectUploadPrepareRequest,
DirectUploadPrepareResponse,
UploadAssetRequest,
UploadAssetResponse,
)
from fastapi import APIRouter, Depends, File, Form, HTTPException, UploadFile, status
from packages.application import SubmitIngestJobCommand, SubmitIngestJobUseCase
from packages.application import GetProjectUseCase, SubmitIngestJobCommand, SubmitIngestJobUseCase
logger = logging.getLogger(__name__)
@@ -80,6 +80,21 @@ def _validate_mime_type(content_type: str | None) -> str:
return base_type
def _require_project_and_library(
project_id: str,
library_id: str,
project_repository: Any,
asset_library_repository: Any,
) -> None:
project = GetProjectUseCase(project_repository).execute(project_id)
if project is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Project not found")
libraries = asset_library_repository.find_by_project(project_id)
if not any(item.id == library_id for item in libraries):
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Asset library not found")
def _submit_ingest_job(
project_id: str,
library_id: str,
@@ -120,7 +135,7 @@ async def prepare_direct_upload(
# P2-5: 服务端验证 MIME 类型
validated_content_type = _validate_mime_type(request.content_type)
require_project_and_library(
_require_project_and_library(
request.project_id,
request.library_id,
project_repository,
@@ -168,7 +183,7 @@ async def complete_direct_upload(
storage_service: OSSStorageService = Depends(get_storage_service),
) -> DirectUploadCompleteResponse:
"""确认浏览器直传完成并创建导入任务。"""
require_project_and_library(
_require_project_and_library(
request.project_id,
request.library_id,
project_repository,
@@ -237,7 +252,7 @@ async def upload_asset(
storage_service: OSSStorageService = Depends(get_storage_service),
) -> UploadAssetResponse:
"""上传素材文件并触发导入流水线。"""
require_project_and_library(project_id, library_id, project_repository, asset_library_repository)
_require_project_and_library(project_id, library_id, project_repository, asset_library_repository)
# ── 素材去重检测:上传前检查同素材库 + 同 file_hash ──
if file_hash:
-238
View File
@@ -1,238 +0,0 @@
import logging
from app.api.routes._helpers import format_utc_datetime
from app.auth import AuthenticatedUser, get_current_user
from app.core.celery_app import celery_app
from app.core.storage import OSSStorageService, get_storage_service
from app.dependencies import get_generated_video_repository
from app.schemas.video_center import (
BatchDeleteRequest,
BatchDownloadRequest,
BatchDownloadResponse,
BatchOperationResponse,
ListVideosResponse,
UpdateVideoReviewRequest,
VideoItemResponse,
)
from fastapi import APIRouter, Depends, HTTPException, Query, Response
from packages.application import (
GetGeneratedVideoUseCase,
GetVideosByIdsUseCase,
ListGeneratedVideosPaginatedUseCase,
UpdateVideoReviewStatusUseCase,
)
logger = logging.getLogger(__name__)
router = APIRouter()
def _to_video_response(item, storage: OSSStorageService | None = None) -> VideoItemResponse:
download_url = None
if storage and item.file_url:
try:
download_url = storage.get_download_url(item.file_url)
except Exception:
download_url = item.file_url
return VideoItemResponse(
id=item.id,
project_id=item.project_id,
generation_task_id=item.generation_task_id,
name=item.name,
file_url=item.file_url,
file_size=item.file_size,
duration=item.duration,
thumbnail_url=item.thumbnail_url,
width=item.width,
height=item.height,
fps=item.fps,
status=item.status,
review_status=item.review_status,
generation_params=item.generation_params,
download_url=download_url,
generated_at=format_utc_datetime(item.generated_at) if hasattr(item, "generated_at") else "",
)
@router.get("/videos", response_model=ListVideosResponse)
def list_videos(
project_id: str | None = Query(None, description="项目ID,可选过滤"),
status: str | None = Query(None, description="按状态筛选"),
review_status: str | None = Query(None, description="按复核状态筛选"),
page: int = Query(1, ge=1, description="页码"),
page_size: int = Query(20, ge=1, le=100, description="每页数量"),
repo=Depends(get_generated_video_repository),
storage: OSSStorageService = Depends(get_storage_service),
current_user: AuthenticatedUser = Depends(get_current_user),
):
"""成片列表,默认返回当前用户的所有成片,支持按项目/状态/复核状态筛选。"""
use_case = ListGeneratedVideosPaginatedUseCase(repo)
items, total = use_case.execute(
user_id=current_user.user.id,
project_id=project_id,
status=status,
review_status=review_status,
page=page,
page_size=page_size,
)
return ListVideosResponse(
items=[_to_video_response(item, storage) for item in items],
total=total,
page=page,
page_size=page_size,
)
@router.get("/videos/{video_id}", response_model=VideoItemResponse)
def get_video(
video_id: str,
repo=Depends(get_generated_video_repository),
storage: OSSStorageService = Depends(get_storage_service),
current_user: AuthenticatedUser = Depends(get_current_user),
):
"""获取单个成片详情。"""
use_case = GetGeneratedVideoUseCase(repo)
item = use_case.execute(video_id)
if item is None:
raise HTTPException(status_code=404, detail="Video not found")
return _to_video_response(item, storage)
@router.patch("/videos/{video_id}/review", response_model=VideoItemResponse)
def update_video_review_status(
video_id: str,
request: UpdateVideoReviewRequest,
repo=Depends(get_generated_video_repository),
storage: OSSStorageService = Depends(get_storage_service),
current_user: AuthenticatedUser = Depends(get_current_user),
):
"""更新成片复核状态:pending_review / approved / rejected。"""
use_case = UpdateVideoReviewStatusUseCase(repo)
item = use_case.execute(video_id, request.review_status)
if item is None:
raise HTTPException(status_code=404, detail="Video not found")
logger.info(
"Video %s review status updated to %s by user %s", video_id, request.review_status, current_user.user_id
)
return _to_video_response(item, storage)
@router.delete("/videos/{video_id}", status_code=204, response_class=Response)
def delete_video(
video_id: str,
repo=Depends(get_generated_video_repository),
storage: OSSStorageService = Depends(get_storage_service),
current_user: AuthenticatedUser = Depends(get_current_user),
):
"""删除单个成片(硬删除)。"""
use_case = GetGeneratedVideoUseCase(repo)
item = use_case.execute(video_id)
if item is None:
raise HTTPException(status_code=404, detail="Video not found")
# 尝试删除 OSS 文件,失败不影响数据库删除
if item.file_url:
try:
key = storage._normalize_storage_key(item.file_url)
storage.delete_file(key)
except Exception:
logger.warning("删除 OSS 视频文件失败,跳过: video_id=%s", video_id)
repo.delete(video_id)
return Response(status_code=204)
@router.post("/videos/batch-delete", response_model=BatchOperationResponse)
def batch_delete_videos(
request: BatchDeleteRequest,
repo=Depends(get_generated_video_repository),
storage: OSSStorageService = Depends(get_storage_service),
current_user: AuthenticatedUser = Depends(get_current_user),
):
"""批量删除成片。"""
videos = repo.get_by_ids(request.video_ids)
existing_ids = {v.id for v in videos}
failed_ids = [vid for vid in request.video_ids if vid not in existing_ids]
failed_details = {vid: "Video not found" for vid in failed_ids}
# 尝试删除 OSS 文件
for video in videos:
if video.file_url:
try:
key = storage._normalize_storage_key(video.file_url)
storage.delete_file(key)
except Exception:
logger.warning("批量删除 OSS 视频文件失败,跳过: video_id=%s", video.id)
success_count = repo.batch_delete(list(existing_ids))
return BatchOperationResponse(
success_count=success_count,
failed_ids=failed_ids,
failed_details=failed_details,
)
@router.post("/videos/batch-download", response_model=BatchDownloadResponse)
def batch_download_videos(
request: BatchDownloadRequest,
repo=Depends(get_generated_video_repository),
current_user: AuthenticatedUser = Depends(get_current_user),
):
"""批量下载成片,异步打包 zip。
传入 video_ids 列表,创建一个批量下载任务,任务完成后返回 zip 下载链接。
"""
if not request.video_ids:
raise HTTPException(status_code=400, detail="video_ids cannot be empty")
if len(request.video_ids) > 50:
raise HTTPException(status_code=400, detail="Maximum 50 videos per batch download")
# 校验视频都存在
use_case = GetVideosByIdsUseCase(repo)
videos = use_case.execute(request.video_ids)
if len(videos) != len(request.video_ids):
raise HTTPException(status_code=404, detail="Some videos not found")
# 发送 celery 任务
task = celery_app.send_task(
"worker.batch_download_videos",
args=[request.video_ids, current_user.user_id],
)
logger.info("Batch download job created: %s, videos=%d", task.id, len(request.video_ids))
return BatchDownloadResponse(job_id=task.id, status="pending")
@router.get("/videos/batch-download/{job_id}", response_model=BatchDownloadResponse)
def get_batch_download_status(
job_id: str,
current_user: AuthenticatedUser = Depends(get_current_user),
):
"""查询批量下载任务状态。"""
from celery.result import AsyncResult
task = AsyncResult(job_id, app=celery_app)
status_map = {
"PENDING": "pending",
"STARTED": "running",
"SUCCESS": "success",
"FAILURE": "failed",
"RETRY": "pending",
"REVOKED": "cancelled",
}
api_status = status_map.get(task.state, "pending")
download_url = None
if task.state == "SUCCESS" and task.result:
if isinstance(task.result, dict):
download_url = task.result.get("download_url")
elif isinstance(task.result, str):
download_url = task.result
return BatchDownloadResponse(
job_id=job_id,
status=api_status,
download_url=download_url,
)
+10 -11
View File
@@ -28,6 +28,7 @@ from packages.application.voice_clone.use_cases import (
VoiceCloneNotRetryableError,
)
from packages.application.voice_clone.workflow import (
VoiceCloneWorkflowError,
VoiceCloneWorkflowService,
)
@@ -37,7 +38,6 @@ router = APIRouter()
def _to_response(profile) -> VoiceCloneProfileResponse:
# source_audio_url 是用户传入的原始 URL(可能是外部地址),不做预签名转换
return VoiceCloneProfileResponse(
id=profile.id,
user_id=profile.user_id,
@@ -141,8 +141,8 @@ def get_voice_clone(
use_case = GetVoiceCloneUseCase(repository)
try:
profile = use_case.execute(clone_id, user_id)
except VoiceCloneNotFoundError as _e:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Voice clone not found") from _e
except VoiceCloneNotFoundError:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Voice clone not found")
return _to_response(profile)
@@ -157,8 +157,8 @@ def get_voice_clone_status(
use_case = GetVoiceCloneStatusUseCase(repository)
try:
profile = use_case.execute(clone_id, user_id)
except VoiceCloneNotFoundError as _e:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Voice clone not found") from _e
except VoiceCloneNotFoundError:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Voice clone not found")
return VoiceCloneStatusResponse(
id=profile.id,
status=profile.status,
@@ -172,7 +172,6 @@ def get_voice_clone_status(
"/{clone_id}",
status_code=status.HTTP_204_NO_CONTENT,
response_model=None,
response_class=Response,
)
def delete_voice_clone(
clone_id: str,
@@ -185,7 +184,7 @@ def delete_voice_clone(
deleted = use_case.execute(clone_id, user_id)
if not deleted:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Voice clone not found")
return
return Response(status_code=204)
@router.post("/{clone_id}/retry", response_model=VoiceCloneProfileResponse)
@@ -201,13 +200,13 @@ def retry_voice_clone(
user_id = authenticated_user.user.id
try:
profile = workflow.retry_clone(clone_id, user_id)
except VoiceCloneNotFoundError as _e:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Voice clone not found") from _e
except VoiceCloneNotRetryableError as _e:
except VoiceCloneNotFoundError:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Voice clone not found")
except VoiceCloneNotRetryableError:
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Voice clone is not retryable (only failed clones can be retried)",
) from _e
)
# 如果 profile 处于 processing 且有 task_id,触发 Celery 异步轮询
task_id = (profile.metadata or {}).get("cosyvoice_task_id", "")
+24 -90
View File
@@ -7,9 +7,8 @@ from __future__ import annotations
from typing import Literal, Optional
from app.api.routes._helpers import get_user_plan
from app.auth import AuthenticatedUser, get_current_user
from app.dependencies import get_audio_url_signer, get_cosyvoice_service, get_db_session, get_user_repository
from app.dependencies import get_db_session, get_user_repository
from app.schemas.voice import (
PresetVoiceItemResponse,
PresetVoiceListResponse,
@@ -27,7 +26,6 @@ from sqlalchemy.orm import Session
from packages.adapters.sqlalchemy_impl.voice_clone_profile_repository import SQLAlchemyVoiceCloneProfileRepository
from packages.adapters.sqlalchemy_impl.voice_library_repository import SQLAlchemyVoiceLibraryRepository
from packages.application.cosyvoice_service import CosyVoiceError, CosyVoiceService
from packages.application.voice_library.commands import CreateVoiceLibraryCommand, UpdateVoiceLibraryCommand
from packages.application.voice_library.use_cases import (
CreateVoiceLibraryUseCase,
@@ -38,18 +36,11 @@ from packages.application.voice_library.use_cases import (
QuotaExceededError,
UpdateVoiceLibraryUseCase,
)
from packages.domain.preset_voices import PRESET_VOICES, get_preset_voice_by_id
from packages.domain.preset_voices import PRESET_VOICES
from packages.ports.user_repository import UserRepository
router = APIRouter()
# 预置音色试听音频缓存(内存缓存,减少重复TTS调用)
# key: voice_id, value: (audio_url, timestamp)
_preset_preview_cache: dict[str, tuple[str, float]] = {}
PREVIEW_CACHE_TTL = 7 * 24 * 3600 # 7天TTL
# 每个预置音色的默认试听文本
PREVIEW_TEMPLATE = "你好,我是{name},很高兴认识你。"
def _get_voice_repository(session: Session = Depends(get_db_session)) -> SQLAlchemyVoiceLibraryRepository:
return SQLAlchemyVoiceLibraryRepository(session)
@@ -59,10 +50,7 @@ def _get_clone_profile_repository(session: Session = Depends(get_db_session)) ->
return SQLAlchemyVoiceCloneProfileRepository(session)
def _to_response(item, sign_url=None) -> VoiceLibraryItemResponse:
audio = item.audio_url
if sign_url and audio:
audio = sign_url(audio)
def _to_response(item) -> VoiceLibraryItemResponse:
return VoiceLibraryItemResponse(
id=item.id,
user_id=item.user_id,
@@ -71,7 +59,7 @@ def _to_response(item, sign_url=None) -> VoiceLibraryItemResponse:
voice_provider=item.voice_provider,
voice_id=item.voice_id,
voice_name=item.voice_name,
audio_url=audio,
audio_url=item.audio_url,
duration=item.duration,
file_size=item.file_size,
status=item.status,
@@ -82,20 +70,16 @@ def _to_response(item, sign_url=None) -> VoiceLibraryItemResponse:
)
def _to_unified_response(item, profile_id_map: dict | None = None, sign_url=None) -> UnifiedVoiceItemResponse:
def _to_unified_response(item, profile_id_map: dict | None = None) -> UnifiedVoiceItemResponse:
"""将数据库音色转换为统一响应格式。
Args:
item: VoiceLibraryItem
profile_id_map: voice_id → profile_id 映射,用于填充 voice_clone_profile_id
sign_url: 音频URL预签名函数
"""
profile_id = None
if profile_id_map and item.voice_id:
profile_id = profile_id_map.get(item.voice_id)
audio = item.audio_url
if sign_url and audio:
audio = sign_url(audio)
return UnifiedVoiceItemResponse(
id=item.id,
type="clone",
@@ -105,7 +89,7 @@ def _to_unified_response(item, profile_id_map: dict | None = None, sign_url=None
language="zh-CN",
voice_id=item.voice_id,
voice_provider=item.voice_provider or "cosyvoice",
audio_url=audio,
audio_url=item.audio_url,
duration=item.duration,
file_size=item.file_size,
status=item.status,
@@ -134,6 +118,13 @@ def _preset_to_unified_response(preset) -> UnifiedVoiceItemResponse:
)
def _get_user_plan(user_id: str, user_repository: UserRepository) -> str:
user = user_repository.find_by_id(user_id)
if user is None:
return "free"
return getattr(user, "subscription_plan", "free") or "free"
# ==================== 统一配音列表(预置 + 克隆)====================
@@ -149,7 +140,6 @@ def list_voices_unified(
authenticated_user: AuthenticatedUser = Depends(get_current_user),
voice_repository: SQLAlchemyVoiceLibraryRepository = Depends(_get_voice_repository),
clone_profile_repository: SQLAlchemyVoiceCloneProfileRepository = Depends(_get_clone_profile_repository),
sign_url=Depends(get_audio_url_signer),
) -> UnifiedVoiceListResponse:
"""获取配音列表(预置音色 + 用户克隆音色)。
@@ -177,7 +167,7 @@ def list_voices_unified(
# 批量查询 voice_id → profile_id 映射,填充 voice_clone_profile_id
voice_ids = [i.voice_id for i in clone_items_raw if i.voice_id]
profile_id_map = clone_profile_repository.find_profile_ids_by_voice_ids(voice_ids) if voice_ids else {}
clone_items = [_to_unified_response(i, profile_id_map, sign_url) for i in clone_items_raw]
clone_items = [_to_unified_response(i, profile_id_map) for i in clone_items_raw]
# 组装结果
if type == "preset":
@@ -224,58 +214,6 @@ def list_preset_voices() -> PresetVoiceListResponse:
return PresetVoiceListResponse(items=items, total=len(items))
@router.get("/presets/{voice_id}/preview")
def get_preset_voice_preview(
voice_id: str,
text: str = Query("", description="自定义试听文本,为空则使用默认示例"),
cosyvoice: CosyVoiceService = Depends(get_cosyvoice_service),
) -> dict:
"""获取预置音色试听音频(实时 TTS 合成)。
- 首次调用会合成并缓存7天
- 相同 voice_id 重复调用直接返回缓存的音频URL
- 可传入自定义 text 参数试听不同文本
"""
import time
preset = get_preset_voice_by_id(voice_id)
if preset is None:
raise HTTPException(status_code=404, detail=f"预置音色不存在: {voice_id}")
# 有自定义文本时不缓存
use_cache = not text.strip()
if use_cache and voice_id in _preset_preview_cache:
audio_url, cached_at = _preset_preview_cache[voice_id]
if time.time() - cached_at < PREVIEW_CACHE_TTL:
return {"voice_id": voice_id, "audio_url": audio_url, "cached": True}
# 合成试听音频
preview_text = text.strip() or PREVIEW_TEMPLATE.format(name=preset.name)
try:
result = cosyvoice.synthesize_speech(
text=preview_text,
voice_id=preset.voice_id,
format="mp3",
speed=1.0,
)
except CosyVoiceError as e:
raise HTTPException(status_code=502, detail=f"TTS 合成失败: {e}") from e
audio_url = result.audio_url
# 缓存(仅默认试听文本)
if use_cache:
_preset_preview_cache[voice_id] = (audio_url, time.time())
return {
"voice_id": voice_id,
"audio_url": audio_url,
"text": preview_text,
"cached": False,
}
# ==================== 原有 CRUD 端点(保持向后兼容)====================
@@ -286,7 +224,6 @@ def list_voices_legacy(
limit: int = Query(50, ge=1, le=200),
authenticated_user: AuthenticatedUser = Depends(get_current_user),
voice_repository: SQLAlchemyVoiceLibraryRepository = Depends(_get_voice_repository),
sign_url=Depends(get_audio_url_signer),
) -> ListVoiceLibraryResponse:
"""原有配音列表接口(仅返回用户克隆音色)。
@@ -296,7 +233,7 @@ def list_voices_legacy(
use_case = ListVoiceLibraryUseCase(voice_repository)
items, total = use_case.execute(user_id, status=status_filter, skip=skip, limit=limit)
return ListVoiceLibraryResponse(
items=[_to_response(i, sign_url) for i in items],
items=[_to_response(i) for i in items],
total=total,
)
@@ -306,14 +243,13 @@ def get_voice(
voice_id: str,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
voice_repository: SQLAlchemyVoiceLibraryRepository = Depends(_get_voice_repository),
sign_url=Depends(get_audio_url_signer),
) -> VoiceLibraryItemResponse:
user_id = authenticated_user.user.id
use_case = GetVoiceLibraryUseCase(voice_repository)
item = use_case.execute(voice_id, user_id)
if item is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Voice not found")
return _to_response(item, sign_url)
return _to_response(item)
@router.post("", response_model=VoiceLibraryItemResponse, status_code=status.HTTP_201_CREATED)
@@ -322,10 +258,9 @@ def create_voice(
authenticated_user: AuthenticatedUser = Depends(get_current_user),
voice_repository: SQLAlchemyVoiceLibraryRepository = Depends(_get_voice_repository),
user_repository: UserRepository = Depends(get_user_repository),
sign_url=Depends(get_audio_url_signer),
) -> VoiceLibraryItemResponse:
user_id = authenticated_user.user.id
plan_name = get_user_plan(user_id, user_repository)
plan_name = _get_user_plan(user_id, user_repository)
command = CreateVoiceLibraryCommand(
user_id=user_id,
name=request.name,
@@ -347,8 +282,8 @@ def create_voice(
raise HTTPException(
status_code=status.HTTP_429_TOO_MANY_REQUESTS,
detail=f"配音库配额已满({exc.used}/{exc.limit}),请升级套餐",
) from exc
return _to_response(item, sign_url)
)
return _to_response(item)
@router.put("/{voice_id}", response_model=VoiceLibraryItemResponse)
@@ -357,7 +292,6 @@ def update_voice(
request: UpdateVoiceLibraryRequest,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
voice_repository: SQLAlchemyVoiceLibraryRepository = Depends(_get_voice_repository),
sign_url=Depends(get_audio_url_signer),
) -> VoiceLibraryItemResponse:
user_id = authenticated_user.user.id
command = UpdateVoiceLibraryCommand(
@@ -377,12 +311,12 @@ def update_voice(
use_case = UpdateVoiceLibraryUseCase(voice_repository)
try:
item = use_case.execute(command)
except NotFoundError as _e:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Voice not found") from _e
return _to_response(item, sign_url)
except NotFoundError:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Voice not found")
return _to_response(item)
@router.delete("/{voice_id}", status_code=status.HTTP_204_NO_CONTENT, response_model=None, response_class=Response)
@router.delete("/{voice_id}", status_code=status.HTTP_204_NO_CONTENT, response_model=None)
def delete_voice(
voice_id: str,
authenticated_user: AuthenticatedUser = Depends(get_current_user),
@@ -393,4 +327,4 @@ def delete_voice(
deleted = use_case.execute(voice_id, user_id)
if not deleted:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Voice not found")
return
return Response(status_code=204)
+10 -31
View File
@@ -19,25 +19,27 @@ class Settings(BaseSettings):
# Container bind address; external expose is controlled by Docker/Nginx.
API_HOST: str = "0.0.0.0" # nosec: B104
API_PORT: int = 8000
API_PREFIX: str = "/api/v1"
DATABASE_URL: str = "postgresql+psycopg://postgres:postgres@localhost:5432/xiaoxia_saas"
DATABASE_POOL_SIZE: int = 20
DATABASE_MAX_OVERFLOW: int = 10 # 调整为合理值:pool_size(20) + max_overflow(10) = 最大30连接
DATABASE_POOL_TIMEOUT: int = 30
DATABASE_POOL_RECYCLE: int = 3600
DATABASE_POOL_RECYLE: int = 3600
USE_IN_MEMORY_DB: bool = False
AUTO_CREATE_SCHEMA: bool = False
REDIS_URL: str = "redis://localhost:6379/0"
REDIS_MAX_CONNECTION: int = 50
ENABLE_REDIS_SESSIONS: bool = False
# JWT secret key - MUST be set via environment variable, no default allowed
JWT_SECRET_KEY: Optional[str] = None
# JWT 算法与过期时间(与 .env.example 对齐)
JWT_ALGORITHM: str = "HS256"
JWT_ACCESS_TOKEN_EXPIRE_MINUTES: int = 30
JWT_REFRESH_TOKEN_EXPIRE_DAYS: int = 30
# 旧的 JWT secret key(用于密钥轮换期间验证旧 token)
# 在密钥轮换时,先设置新密钥,旧密钥保留在此处直到所有旧 token 过期
JWT_SECRET_KEY_OLD: Optional[str] = None
# 密钥轮换天数(到达此天数后建议更换密钥)
SECRET_ROTATION_DAYS: int = 90
@field_validator("JWT_SECRET_KEY", mode="before")
@classmethod
@@ -73,42 +75,19 @@ class Settings(BaseSettings):
CELERY_RESULT_BACKEND: str = "redis://localhost:6379/1"
# OSS 七牛云相关
OSS_ENDPOINT: str = "oss-cn-hangzhou.aliyuncs.com"
OSS_ENDPOINT: str = "oss-cn-hangzhou.aliiyuncs.com"
OSS_ACCESS_KEY_ID: str = ""
OSS_ACCESS_KEY_SECRET: str = ""
OSS_BUCKET_NAME: str = "xiaoxia-autocut"
@field_validator("OSS_ACCESS_KEY_ID", mode="before")
@classmethod
def validate_oss_access_key_id(cls, v):
if (v is None or v == "") and os.getenv("APP_ENV", "development") != "development":
raise ValueError(
"OSS_ACCESS_KEY_ID must be set via environment variable in non-development environments. "
"Check the server .env file (e.g. /var/lib/xiaoxia-saas-staging/.env)."
)
return v or ""
@field_validator("OSS_ACCESS_KEY_SECRET", mode="before")
@classmethod
def validate_oss_access_key_secret(cls, v):
if (v is None or v == "") and os.getenv("APP_ENV", "development") != "development":
raise ValueError(
"OSS_ACCESS_KEY_SECRET must be set via environment variable in non-development environments. "
"Check the server .env file (e.g. /var/lib/xiaoxia-saas-staging/.env)."
)
return v or ""
OSS_DIRECT_UPLOAD_MAX_MB: int = Field(
default=2000,
validation_alias=AliasChoices("OSS_DIRECT_UPLOAD_MAX_MB", "MAX_UPLOAD_SIZE_MB"),
)
OSS_DIRECT_UPLOAD_EXPIRE_SECONDS: int = 900
LOG_LEVEL: str = "INFO"
CORS_ORIGINS_RAW: str = "http://localhost:3000,http://localhost:5173,http://localhost:8000"
# 渲染引擎选择:legacy=旧VideoComposeServiceunified=新UnifiedRenderService
RENDER_ENGINE: str = "legacy"
model_config = SettingsConfigDict(
env_file=".env",
env_file_encoding="utf-8",
+1
View File
@@ -0,0 +1 @@
"""Core configuration package."""
+240 -11
View File
@@ -1,14 +1,243 @@
"""Backward-compatible re-export from shared storage.
"""阿里云 OSS 存储服务"""
All storage logic now lives in ``packages.shared.storage``.
This module keeps old import paths working so existing code
does not need to change.
"""
import base64
import datetime as dt
import hashlib
import hmac
import json
import logging
import os
from urllib.parse import urlparse
from packages.shared.storage import SharedStorageService as OSSStorageService
from packages.shared.storage import (
get_shared_storage_service,
get_storage_service,
)
try:
import oss2
except ImportError: # pragma: no cover - exercised in minimal local/test environments
oss2 = None
from app.config import get_settings
__all__ = ["OSSStorageService", "get_storage_service", "get_shared_storage_service"]
logger = logging.getLogger(__name__)
class OSSStorageService:
"""阿里云 OSS 存储服务"""
def __init__(self):
settings = get_settings()
self.bucket_name = settings.OSS_BUCKET_NAME
self.public_url = f"https://{settings.OSS_BUCKET_NAME}.{settings.OSS_ENDPOINT}"
self.local_url_prefix = os.getenv("GENERATED_FILES_URL_PREFIX", "/generated-files")
self.bucket = None
has_key_id = bool(settings.OSS_ACCESS_KEY_ID)
has_key_secret = bool(settings.OSS_ACCESS_KEY_SECRET)
if has_key_id and has_key_secret:
if oss2 is not None:
try:
auth = oss2.Auth(
settings.OSS_ACCESS_KEY_ID,
settings.OSS_ACCESS_KEY_SECRET,
)
self.bucket = oss2.Bucket(
auth,
settings.OSS_ENDPOINT,
settings.OSS_BUCKET_NAME,
)
logger.info(
"OSS initialized: endpoint=%s bucket=%s",
settings.OSS_ENDPOINT,
settings.OSS_BUCKET_NAME,
)
except Exception as error:
logger.error("Failed to initialize OSS bucket client: %s", error)
else:
logger.error("oss2 SDK is not installed — OSS operations will fail")
else:
missing = []
if not has_key_id:
missing.append("OSS_ACCESS_KEY_ID")
if not has_key_secret:
missing.append("OSS_ACCESS_KEY_SECRET")
logger.error("OSS credentials not configured — missing: %s", ", ".join(missing))
self.access_key_id = settings.OSS_ACCESS_KEY_ID
self.access_key_secret = settings.OSS_ACCESS_KEY_SECRET
self.endpoint = settings.OSS_ENDPOINT
def _is_local_generated_url(self, storage_key_or_url: str) -> bool:
parsed = urlparse(storage_key_or_url)
path = parsed.path if parsed.scheme else storage_key_or_url
return path.startswith(f"{self.local_url_prefix}/")
def create_direct_upload_post(
self,
storage_key: str,
content_type: str,
max_size_bytes: int,
expires_seconds: int,
) -> dict[str, object]:
"""创建浏览器直传 OSS 的 PostObject 表单。"""
if not self.access_key_id or not self.access_key_secret:
raise RuntimeError("OSS storage is not configured")
normalized_key = self._normalize_storage_key(storage_key)
if not normalized_key.startswith("uploads/"):
raise ValueError("direct upload key must be under uploads/")
expiration = (dt.datetime.now(dt.timezone.utc) + dt.timedelta(seconds=expires_seconds)).strftime(
"%Y-%m-%dT%H:%M:%S.000Z"
)
policy = {
"expiration": expiration,
"conditions": [
{"bucket": self.bucket_name},
{"key": normalized_key},
["content-length-range", 1, max_size_bytes],
["starts-with", "$Content-Type", content_type.split("/", 1)[0] + "/" if "/" in content_type else ""],
],
}
encoded_policy = base64.b64encode(json.dumps(policy, separators=(",", ":")).encode("utf-8")).decode("ascii")
signature = base64.b64encode(
hmac.new(self.access_key_secret.encode("utf-8"), encoded_policy.encode("utf-8"), hashlib.sha1).digest()
).decode("ascii")
return {
"url": self.public_url,
"method": "POST",
"storage_key": normalized_key,
"expires_at": expiration,
"fields": {
"key": normalized_key,
"OSSAccessKeyId": self.access_key_id,
"policy": encoded_policy,
"Signature": signature,
"success_action_status": "201",
"Content-Type": content_type,
},
}
def upload_file(
self,
file_or_path,
storage_key: str,
content_type: str = "application/octet-stream",
) -> str:
"""
上传文件到 OSS
Args:
file_or_path: 文件对象或本地文件路径
storage_key: 存储键(文件路径)
content_type: 内容类型
Returns:
文件公网 URL
"""
if self.bucket is None:
raise RuntimeError("OSS storage is not configured")
try:
# 如果是字符串路径,从本地文件上传
if isinstance(file_or_path, str):
self.bucket.put_object_from_file(storage_key, file_or_path, headers={"Content-Type": content_type})
else:
# 文件对象
file_or_path.seek(0)
self.bucket.put_object(storage_key, file_or_path, headers={"Content-Type": content_type})
return f"{self.public_url}/{storage_key}"
except Exception as e:
raise Exception(f"Failed to upload file to OSS: {e}")
def get_url(self, storage_key: str) -> str:
"""获取文件公网 URL"""
return f"{self.public_url}/{storage_key}"
def get_download_url(self, storage_key_or_url: str, expires_seconds: int = 3600) -> str:
"""
获取文件下载签名 URL(用于私有文件)
Args:
storage_key_or_url: 存储键或完整 URL
expires_seconds: 过期时间(秒)
Returns:
签名 URL
"""
if self.bucket is None:
if self._is_local_generated_url(storage_key_or_url):
return storage_key_or_url
return self.get_url(self._normalize_storage_key(storage_key_or_url))
storage_key = self._normalize_storage_key(storage_key_or_url)
try:
return self.bucket.sign_url("GET", storage_key, expires_seconds)
except Exception:
return self.get_url(storage_key)
def _normalize_storage_key(self, storage_key_or_url: str) -> str:
"""从 URL 中提取存储键"""
if storage_key_or_url.startswith("http://") or storage_key_or_url.startswith("https://"):
parsed = urlparse(storage_key_or_url)
# 移除开头的 /
return parsed.path.lstrip("/")
return storage_key_or_url.lstrip("/")
def download_file(self, storage_key: str, local_path: str):
"""
从 OSS 下载文件到本地
Args:
storage_key: 存储键
local_path: 本地文件路径
"""
if self.bucket is None:
raise RuntimeError("OSS storage is not configured")
try:
os.makedirs(os.path.dirname(local_path), exist_ok=True)
self.bucket.get_object_to_file(storage_key, local_path)
except Exception as e:
raise Exception(f"Failed to download file from OSS: {e}")
def delete_file(self, storage_key: str):
"""
删除 OSS 文件
Args:
storage_key: 存储键
"""
if self.bucket is None:
return
try:
self.bucket.delete_object(storage_key)
except Exception as error:
logger.warning(
"Failed to delete file from OSS",
extra={"storage_key": storage_key, "error": str(error)},
)
def file_exists(self, storage_key: str) -> bool:
"""
检查文件是否存在
Args:
storage_key: 存储键
Returns:
是否存在
"""
if self.bucket is None:
return False
return self.bucket.object_exists(storage_key)
_storage_service = None
def get_storage_service() -> OSSStorageService:
"""获取存储服务实例(全局单例)"""
global _storage_service
if _storage_service is None:
_storage_service = OSSStorageService()
return _storage_service
-221
View File
@@ -1,221 +0,0 @@
import logging
from typing import Any
from app.core.celery_app import celery_app
logger = logging.getLogger(__name__)
# ── 限流阈值常量(全系统统一管理,不要在业务代码里硬编码) ──
USER_PENDING_LIMIT = 3 # 单用户 pending 上限
GLOBAL_PENDING_LIMIT = 20 # 全局 pending 上限
class UserPendingLimitExceeded(Exception):
"""用户 pending 任务数超限,返回 429。"""
def __init__(self, user_id: str, pending_count: int, limit: int):
self.user_id = user_id
self.pending_count = pending_count
self.limit = limit
super().__init__(f"用户 {user_id} pending 任务数 {pending_count} 超过上限 {limit}")
class GlobalQueueFull(Exception):
"""全局限流,返回 503。"""
def __init__(self, pending_count: int, limit: int):
self.pending_count = pending_count
self.limit = limit
super().__init__(f"系统 pending 任务数 {pending_count} 超过上限 {limit}")
def check_queue_limits(
user_id: str,
generation_task_repository: Any,
*,
user_pending_limit: int = USER_PENDING_LIMIT,
global_pending_limit: int = GLOBAL_PENDING_LIMIT,
) -> None:
"""检查队列限流(预检查用,任务创建前调用),超限抛对应异常。
边界语义:>= 上限即拒绝(达到上限就不能再加新任务)。
Args:
user_id: 用户 ID
generation_task_repository: 任务仓储
user_pending_limit: 单用户 pending 上限,默认 USER_PENDING_LIMIT
global_pending_limit: 全局 pending 上限,默认 GLOBAL_PENDING_LIMIT
Raises:
GlobalQueueFull: 全局超限时抛出(优先级更高,先查全局)
UserPendingLimitExceeded: 用户超限时抛出
"""
# 先查全局(系统级保护优先级更高)
global_pending = generation_task_repository.count_pending_total()
if global_pending >= global_pending_limit:
logger.warning(
"[队列限流] 全局 pending 任务数超限: %d/%d, user_id=%s",
global_pending,
global_pending_limit,
user_id,
)
raise GlobalQueueFull(pending_count=global_pending, limit=global_pending_limit)
# 再查用户级
if user_id:
user_pending = generation_task_repository.count_pending_by_user(user_id)
if user_pending >= user_pending_limit:
logger.warning(
"[队列限流] 用户 pending 任务数超限: user_id=%s, count=%d/%d",
user_id,
user_pending,
user_pending_limit,
)
raise UserPendingLimitExceeded(user_id=user_id, pending_count=user_pending, limit=user_pending_limit)
def _mark_task_failed_safely(
task: Any,
generation_task_repository: Any,
log_prefix: str,
reason: str,
) -> None:
"""安全地把任务标记为 failed,更新失败只打日志不崩溃。"""
try:
task.mark_failed(f"任务被限流拒绝: {reason}")
generation_task_repository.update(task)
except Exception as update_err:
logger.error(
"%s 限流后更新状态也失败: task_id=%s error=%s",
log_prefix,
task.id,
update_err,
exc_info=True,
)
def safe_enqueue_generation_task(
task: Any,
generation_task_repository: Any,
*,
user_id: str = "",
log_prefix: str = "[任务队列]",
log_task_status: bool = False,
user_pending_limit: int = USER_PENDING_LIMIT,
global_pending_limit: int = GLOBAL_PENDING_LIMIT,
) -> bool:
"""安全入队:入队前限流检查 → 发送 Celery 任务 → 入队后最终校验兜底。
边界说明:
入队前检查用 > 而非 >=。因为调用此函数时 task 已经是 pending 状态并计入 DB,
pending 总数包含了当前任务本身。pending > limit 等价于"其他任务数 >= limit"
与预检查的 >= 语义一致(都是达到上限就拒绝新任务)。
入队后最终校验:发送 Celery 成功后再查一次 DB 计数,处理并发竞态场景
(两个请求同时通过入队前检查,后到的那个在这里被兜住)。
Args:
task: 生成任务对象,需有 id 属性和 mark_failed 方法(状态已为 pending
generation_task_repository: 任务仓储,用于更新状态
user_id: 用户 ID,传了才做用户级限流检查
log_prefix: 日志前缀,便于区分调用来源
log_task_status: 成功日志中是否额外打印任务状态
user_pending_limit: 单用户 pending 上限,默认 USER_PENDING_LIMIT
global_pending_limit: 全局 pending 上限,默认 GLOBAL_PENDING_LIMIT
Returns:
True 表示入队成功,False 表示入队失败(已标记为 failed)
Raises:
GlobalQueueFull: 全局 pending 超限时抛出,任务会被标记为 failed
UserPendingLimitExceeded: 用户 pending 超限时抛出,任务会被标记为 failed
"""
# ── 入队前检查:任务已是 pending,用 > 判断(包含当前任务) ──
# 全局限流检查(始终生效)
global_pending = generation_task_repository.count_pending_total()
if global_pending > global_pending_limit:
logger.warning(
"[队列限流] 全局 pending 任务数超限(入队前): %d/%d, user_id=%s",
global_pending,
global_pending_limit,
user_id or "unknown",
)
exc: Exception = GlobalQueueFull(pending_count=global_pending, limit=global_pending_limit)
_mark_task_failed_safely(task, generation_task_repository, log_prefix, str(exc))
raise exc
# 用户级限流检查(传了 user_id 才做)
if user_id:
user_pending = generation_task_repository.count_pending_by_user(user_id)
if user_pending > user_pending_limit:
logger.warning(
"[队列限流] 用户 pending 任务数超限(入队前): user_id=%s, count=%d/%d",
user_id,
user_pending,
user_pending_limit,
)
exc = UserPendingLimitExceeded(user_id=user_id, pending_count=user_pending, limit=user_pending_limit)
_mark_task_failed_safely(task, generation_task_repository, log_prefix, str(exc))
raise exc
# ── 发送 Celery 任务 ──
try:
celery_app.send_task("worker.generate_video", args=[task.id])
except Exception as e:
logger.error(
"%s 入队失败,标记为失败: task_id=%s error=%s",
log_prefix,
task.id,
e,
exc_info=True,
)
try:
task.mark_failed(f"任务入队失败: {e}")
generation_task_repository.update(task)
except Exception as update_err:
logger.error(
"%s 入队失败后更新状态也失败: task_id=%s error=%s",
log_prefix,
task.id,
update_err,
exc_info=True,
)
return False
# ── 入队后最终校验:并发竞态兜底 ──
# 发送成功后再查一次,防止两个请求同时通过入队前检查导致超限
global_after = generation_task_repository.count_pending_total()
user_after = generation_task_repository.count_pending_by_user(user_id) if user_id else 0
global_over = global_after > global_pending_limit
user_over = bool(user_id and user_after > user_pending_limit)
if global_over or user_over:
if global_over:
reason = f"全局 pending 超限(入队后): {global_after}/{global_pending_limit}"
exc = GlobalQueueFull(pending_count=global_after, limit=global_pending_limit)
else:
reason = f"用户 pending 超限(入队后): {user_after}/{user_pending_limit}"
exc = UserPendingLimitExceeded(user_id=user_id, pending_count=user_after, limit=user_pending_limit)
logger.warning(
"[队列限流] %s, task_id=%s, user_id=%s — 回滚状态为 failed",
reason,
task.id,
user_id or "unknown",
)
_mark_task_failed_safely(task, generation_task_repository, log_prefix, reason)
raise exc
# 入队成功日志
if log_task_status:
logger.info(
"%s 入队成功: task_id=%s, status=%s",
log_prefix,
task.id,
task.status,
)
else:
logger.info("%s 入队成功: task_id=%s", log_prefix, task.id)
return True
Regular → Executable
+15 -33
View File
@@ -50,8 +50,20 @@ from packages.adapters.sqlalchemy_impl.voice_clone_profile_repository import (
from packages.adapters.sqlalchemy_impl.voice_library_repository import (
SQLAlchemyVoiceLibraryRepository,
)
from packages.ports.asset_library_repository import AssetLibraryRepository
from packages.ports.asset_repository import AssetRepository
from packages.ports.classification_job_repository import ClassificationJobRepository
from packages.ports.duplication_repository import DuplicationRecordRepository
from packages.ports.generated_video_repository import GeneratedVideoRepository
from packages.ports.generation_task_repository import GenerationTaskRepository
from packages.ports.ingest_job_repository import IngestJobRepository
from packages.ports.job_repository import JobRepository
from packages.ports.project_repository import ProjectRepository
from packages.ports.tag_repository import TagRepository
from packages.ports.title_library_repository import TitleLibraryRepository
from packages.ports.user_repository import UserRepository
from packages.ports.voice_clone_profile_repository import VoiceCloneProfileRepository
from packages.ports.voice_library_repository import VoiceLibraryRepository
_engine, _SessionLocal = build_session_factory(settings.DATABASE_URL)
@@ -132,7 +144,7 @@ def get_tag_repository(
session: Session = Depends(get_db_session),
) -> TagRepository:
"""Provide the SQLAlchemy tag repository implementation."""
return SQLAlchemyTagRepository(session) # type: ignore[return-value]
return SQLAlchemyTagRepository(session)
def get_user_repository(
@@ -189,37 +201,7 @@ def get_voice_clone_profile_repository(
def get_cosyvoice_service():
"""Provide the CosyVoice service instance.
注入 OSS 音频URL预签名函数,确保私有bucket下的参考音频
能被 CosyVoice 服务器下载。
"""
from app.core.storage import get_storage_service
"""Provide the CosyVoice service instance."""
from packages.application.cosyvoice_service import CosyVoiceService
storage = get_storage_service()
def _sign_audio_url(url: str) -> str:
"""对音频URL做预签名,私有bucket下 CosyVoice 服务器才能下载."""
return storage.get_download_url(url, expires_seconds=86400)
return CosyVoiceService(audio_url_signer=_sign_audio_url)
def get_audio_url_signer():
"""提供音频URL预签名函数(24小时有效期)。
用于所有 API 返回给前端的音频 URL,确保私有 OSS bucket 下可正常访问。
空 URL、非 OSS URL 直接原样返回;签名失败时回退到原始 URL。
"""
from app.core.storage import get_storage_service
storage = get_storage_service()
def sign_audio_url(url: str) -> str:
if not url:
return url
return storage.get_download_url(url, expires_seconds=86400)
return sign_audio_url
return CosyVoiceService()
+2 -10
View File
@@ -7,16 +7,12 @@ common auth dependencies for backward compatibility.
from __future__ import annotations
import logging
from app.auth import AuthenticatedUser
from app.auth import get_current_user as get_authenticated_user
from app.dependencies import get_user_repository
from fastapi import Depends, HTTPException
from fastapi import Depends
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
logger = logging.getLogger(__name__)
from packages.domain.entities import User
from packages.ports.user_repository import UserRepository
@@ -37,10 +33,6 @@ async def get_current_user_optional(
return None
try:
authenticated_user = await get_authenticated_user(credentials, user_repository)
except HTTPException as exc:
if exc.status_code >= 500:
# 服务端错误不应被静默吞掉,记录日志
logger.error("可选认证遇到服务端错误,status=%s", exc.status_code, exc_info=True)
# 4xx 认证失败(如 token 无效、用户不存在)属于正常流程,返回 None
except HTTPException:
return None
return authenticated_user.user
+1 -1
View File
@@ -105,7 +105,7 @@ class RateLimitMiddleware(BaseHTTPMiddleware):
self.max_requests = max_requests
self.window_seconds = window_seconds
self.paths = set(paths) if paths else None
self.requests: dict[str, list[float]] = {}
self.requests = {} # {ip: [timestamps]}
async def dispatch(self, request: Request, call_next):
# 如果配置了路径过滤,只对指定路径限流
+1 -1
View File
@@ -6,7 +6,7 @@ import logging
import time
from typing import Callable
from fastapi import Request
from fastapi import Request, Response
from starlette.middleware.base import BaseHTTPMiddleware
logger = logging.getLogger(__name__)
+1 -1
View File
@@ -61,7 +61,7 @@ class APIVersionMiddleware(BaseHTTPMiddleware):
class VersionNotFoundMiddleware(BaseHTTPMiddleware):
"""处理已下线的 API 版本"""
SUNSET_VERSIONS: list[str] = [] # 已下线的版本列表
SUNSET_VERSIONS = [] # 已下线的版本列表
async def dispatch(self, request: Request, call_next):
version = self._extract_version(request.url.path)
Executable → Regular
+6 -35
View File
@@ -50,50 +50,21 @@ class AssetResponse(BaseModel):
status: str
classification_status: str
quality_score: float | None = None
created_at: str
uploaded_by_user_id: str
tag_ids: list[str] = Field(default_factory=list)
MAX_BATCH_SIZE = 200
class BatchDeleteRequest(BaseModel):
"""批量删除请求(软删除)"""
"""批量删除请求。"""
asset_ids: list[str] = Field(..., min_length=1, max_length=MAX_BATCH_SIZE, description="要删除的素材 ID 列表")
ids: list[str] = Field(..., min_length=1, max_length=100, description="要删除的素材 ID 列表")
class BatchOperationResponse(BaseModel):
"""批量操作通用响应。"""
class BatchDeleteResponse(BaseModel):
"""批量删除响应。"""
success_count: int = Field(..., ge=0, description="成功数量")
failed_ids: list[str] = Field(default_factory=list, description="失败的 ID 列表")
failed_details: dict[str, str] = Field(default_factory=dict, description="失败详情 {asset_id: reason}")
class BatchTagRequest(BaseModel):
"""批量打标签请求。"""
asset_ids: list[str] = Field(..., min_length=1, max_length=MAX_BATCH_SIZE, description="素材 ID 列表")
tag_ids: list[str] = Field(..., min_length=1, max_length=50, description="标签 ID 列表")
mode: str = Field(default="add", pattern="^(add|replace)$", description="add=添加合并,replace=全量替换")
class BatchClassifyRequest(BaseModel):
"""批量修改分类请求。"""
asset_ids: list[str] = Field(..., min_length=1, max_length=MAX_BATCH_SIZE, description="素材 ID 列表")
category: str = Field(..., min_length=1, max_length=50, description="内容分类,如 person/scenic/product")
class BatchMarkRequest(BaseModel):
"""批量设置智能视图标记请求。"""
asset_ids: list[str] = Field(..., min_length=1, max_length=MAX_BATCH_SIZE, description="素材 ID 列表")
smart_view: str = Field(
..., pattern="^(recommended|caution|high_risk)$", description="智能视图标记:recommended/caution/high_risk"
)
deleted_count: int = Field(..., ge=0, description="实际删除数量")
failed_ids: list[str] = Field(default_factory=list, description="删除失败的 ID 列表")
class ListAssetsResponse(BaseModel):
+32
View File
@@ -0,0 +1,32 @@
from datetime import datetime
from pydantic import BaseModel, Field
class RecentTaskItem(BaseModel):
id: str
task_type: str = "generation"
status: str
current_step: str = ""
error_message: str = ""
updated_at: datetime | None = None
class SubscriptionInfo(BaseModel):
"""用户订阅信息。"""
plan: str = "free"
is_active: bool = False
class DashboardOverviewResponse(BaseModel):
"""Dashboard 概览数据。"""
total_assets: int = 0
used_storage_bytes: int = 0
total_titles: int = 0
total_voices: int = 0
total_tasks: int = 0
total_products: int = 0
subscription: SubscriptionInfo = Field(default_factory=SubscriptionInfo)
recent_tasks: list[RecentTaskItem] = Field(default_factory=list)
+1 -36
View File
@@ -1,6 +1,4 @@
import json
from pydantic import BaseModel, Field, field_validator, model_validator
from pydantic import BaseModel, Field, model_validator
class CreateGenerationTaskRequest(BaseModel):
@@ -23,8 +21,6 @@ class CreateGenerationTaskRequest(BaseModel):
voice_ids: list[str] = Field(default_factory=list)
# ── 来源剪辑计划 ──
source_edit_plan_id: str = ""
# ── 视频标题 ──
video_title: str = Field(default="", description="生成视频的标题/名称,为空则使用默认命名")
# ── 批量生成 ──
count: int = Field(default=1, ge=1, le=50, description="批量生成数量,默认1,最大50")
# ── 素材库自动匹配 ──
@@ -35,17 +31,6 @@ class CreateGenerationTaskRequest(BaseModel):
asset_select_count: int = Field(
default=0, ge=0, le=100, description="选取数量,0表示全部(仅 random/smart 模式有效)"
)
# ── 自动重试 ──
auto_retry_enabled: bool = Field(
default=False,
description="是否开启失败自动重试,默认关闭",
)
auto_retry_max: int = Field(
default=0,
ge=0,
le=5,
description="最大自动重试次数,0表示不自动重试,最大5次",
)
@model_validator(mode="after")
def _check_at_least_one_mode(self) -> "CreateGenerationTaskRequest":
@@ -73,30 +58,10 @@ class GenerationTaskResponse(BaseModel):
source_edit_plan_id: str = ""
asset_select_mode: str = ""
batch_id: str = ""
video_title: str = ""
status: str
progress: float
result_count: int
error_message: str
error_info: dict = Field(default_factory=dict)
retry_count: int = 0
auto_retry_enabled: bool = False
auto_retry_max: int = 0
logs: list[dict] = Field(default_factory=list)
@field_validator("logs", mode="before")
@classmethod
def _parse_logs(cls, v: object) -> list[dict]:
"""将 JSON 字符串解析为 list[dict]。"""
if isinstance(v, str):
try:
parsed = json.loads(v)
return parsed if isinstance(parsed, list) else []
except (json.JSONDecodeError, TypeError):
return []
if isinstance(v, list):
return v
return []
class BatchGenerationTaskResponse(BaseModel):
+109
View File
@@ -0,0 +1,109 @@
"""Job API schemas — Phase 8 任务 2.10."""
from __future__ import annotations
from datetime import datetime
from typing import Any, Optional
from pydantic import BaseModel, Field
class CreateJobRequest(BaseModel):
"""创建任务请求体。"""
project_id: str = Field(..., min_length=1, description="项目 ID")
job_type: str = Field(
...,
description="任务类型: video_compose / render_edit_plan / asset_ingest / classification / voice_extraction / generation",
)
payload: dict[str, Any] = Field(default_factory=dict, description="任务输入参数")
source_id: str = Field(default="", description="关联的业务实体 ID(如 edit_plan_id")
max_retries: int = Field(default=3, ge=0, le=10, description="最大重试次数")
class UpdateProgressRequest(BaseModel):
"""更新任务进度请求体。"""
progress: float = Field(..., ge=0.0, le=100.0, description="进度百分比")
current_stage: str = Field(default="", description="当前阶段描述")
class CompleteJobRequest(BaseModel):
"""完成任务请求体。"""
result: dict[str, Any] = Field(default_factory=dict, description="任务结果")
class FailJobRequest(BaseModel):
"""标记任务失败请求体。"""
error_message: str = Field(..., min_length=1, description="错误信息")
class JobResponse(BaseModel):
"""任务响应体。"""
id: str
project_id: str
job_type: str
status: str
progress: float
current_stage: str
payload: dict[str, Any]
result: dict[str, Any]
error_message: str
retry_count: int
max_retries: int
celery_task_id: str
source_id: str
created_by_user_id: str
is_retryable: bool
started_at: Optional[datetime] = None
completed_at: Optional[datetime] = None
created_at: datetime
updated_at: datetime
model_config = {"from_attributes": True}
class ListJobsResponse(BaseModel):
"""任务列表响应体。"""
items: list[JobResponse]
total: int
class JobStatisticsResponse(BaseModel):
"""任务统计响应体。"""
project_id: str
total: int
pending: int
running: int
success: int
failed: int
def job_to_response(job) -> JobResponse:
"""将 Job 领域对象转换为 API 响应。"""
return JobResponse(
id=job.id,
project_id=job.project_id,
job_type=job.job_type.value if hasattr(job.job_type, "value") else str(job.job_type),
status=job.status.value if hasattr(job.status, "value") else str(job.status),
progress=job.progress,
current_stage=job.current_stage,
payload=job.payload,
result=job.result,
error_message=job.error_message,
retry_count=job.retry_count,
max_retries=job.max_retries,
celery_task_id=job.celery_task_id,
source_id=job.source_id,
created_by_user_id=job.created_by_user_id,
is_retryable=job.is_retryable,
started_at=job.started_at,
completed_at=job.completed_at,
created_at=job.created_at,
updated_at=job.updated_at,
)
+86
View File
@@ -0,0 +1,86 @@
"""Recipe API schemas."""
from __future__ import annotations
from datetime import datetime
from typing import Any, Dict, List, Optional
from pydantic import BaseModel, Field
# ── Response ──
class RecipeItemResponse(BaseModel):
id: str
recipe_id: str
item_type: str
item_id: str
position: int
metadata_: Dict[str, Any] = Field(default_factory=dict, alias="metadata")
class Config:
populate_by_name = True
class RecipeResponse(BaseModel):
id: str
user_id: str
name: str
description: str = ""
template_id: str = ""
generation_params: Dict[str, Any] = Field(default_factory=dict)
items: List[RecipeItemResponse] = Field(default_factory=list)
is_active: bool = True
metadata_: Dict[str, Any] = Field(default_factory=dict, alias="metadata")
created_at: datetime
updated_at: datetime
class Config:
populate_by_name = True
class ListRecipesResponse(BaseModel):
items: List[RecipeResponse]
total: int = 0
class UseRecipeResponse(BaseModel):
recipe: RecipeResponse
warnings: List[Dict[str, Any]] = Field(default_factory=list)
# ── Request ──
class RecipeItemRequest(BaseModel):
item_type: str
item_id: str
position: int = 0
metadata_: Dict[str, Any] = Field(default_factory=dict, alias="metadata")
class Config:
populate_by_name = True
class CreateRecipeRequest(BaseModel):
name: str
description: str = ""
template_id: str = ""
generation_params: Dict[str, Any] = Field(default_factory=dict)
items: List[RecipeItemRequest] = Field(default_factory=list)
metadata_: Dict[str, Any] = Field(default_factory=dict, alias="metadata")
class Config:
populate_by_name = True
class UpdateRecipeRequest(BaseModel):
name: Optional[str] = None
description: Optional[str] = None
template_id: Optional[str] = None
generation_params: Optional[Dict[str, Any]] = None
items: Optional[List[RecipeItemRequest]] = None
metadata_: Optional[Dict[str, Any]] = Field(default=None, alias="metadata")
class Config:
populate_by_name = True
+1 -1
View File
@@ -2,7 +2,7 @@
from __future__ import annotations
from typing import Optional
from typing import List, Optional
from pydantic import BaseModel, Field
+6 -10
View File
@@ -1,3 +1,5 @@
from datetime import datetime
from pydantic import BaseModel, Field
@@ -9,19 +11,16 @@ class ProjectTaskResponse(BaseModel):
progress: float
current_step: str
error_message: str = ""
error_info: dict = Field(default_factory=dict)
user_message: str = ""
retryable: bool = False
retry_count: int = 0
source_id: str = ""
template_id: str = ""
created_at: str = ""
updated_at: str = ""
created_at: datetime | None = None
updated_at: datetime | None = None
class ListProjectTasksResponse(BaseModel):
items: list[ProjectTaskResponse] = Field(default_factory=list)
total: int = 0
class UserTaskResponse(BaseModel):
@@ -35,17 +34,14 @@ class UserTaskResponse(BaseModel):
progress: float
current_step: str
error_message: str = ""
error_info: dict = Field(default_factory=dict)
user_message: str = ""
retryable: bool = False
retry_count: int = 0
source_id: str = ""
created_at: str = ""
updated_at: str = ""
created_at: datetime | None = None
updated_at: datetime | None = None
class ListTasksResponse(BaseModel):
"""用户级任务列表响应(GET /api/v1/tasks)。"""
items: list[UserTaskResponse] = Field(default_factory=list)
total: int = 0
Executable → Regular
-23
View File
@@ -45,7 +45,6 @@ class TemplateResponse(BaseModel):
segments: List[SegmentResponse] = Field(default_factory=list)
is_active: bool = True
is_favorite: bool = False
usage_count: int = 0
created_at: datetime
updated_at: datetime
@@ -121,25 +120,3 @@ class CreateCategoryRequest(BaseModel):
class ListCategoriesResponse(BaseModel):
items: List[CategoryResponse]
# ── Copy Template ──
class CopyTemplateRequest(BaseModel):
new_name: str
# ── Tags ──
class ListTagsResponse(BaseModel):
items: List[str]
# ── Usage Stats ──
class TemplateUsageResponse(BaseModel):
template_id: str
usage_count: int
-58
View File
@@ -1,58 +0,0 @@
from typing import Literal
from pydantic import BaseModel, Field
VideoReviewStatus = Literal["pending_review", "approved", "rejected"]
class VideoItemResponse(BaseModel):
id: str
project_id: str
generation_task_id: str
name: str
file_url: str
file_size: int
duration: float
thumbnail_url: str | None = None
width: int
height: int
fps: float
status: str = "completed"
review_status: str = "pending_review"
generation_params: dict = Field(default_factory=dict)
download_url: str | None = None
generated_at: str = ""
class ListVideosResponse(BaseModel):
items: list[VideoItemResponse]
total: int
page: int
page_size: int
class UpdateVideoReviewRequest(BaseModel):
review_status: VideoReviewStatus
MAX_BATCH_DELETE_SIZE = 200
class BatchDeleteRequest(BaseModel):
video_ids: list[str] = Field(..., min_length=1, max_length=MAX_BATCH_DELETE_SIZE, description="要删除的成片ID列表")
class BatchOperationResponse(BaseModel):
success_count: int = Field(..., ge=0, description="成功删除数量")
failed_ids: list[str] = Field(default_factory=list, description="失败的ID列表")
failed_details: dict[str, str] = Field(default_factory=dict, description="失败详情")
class BatchDownloadRequest(BaseModel):
video_ids: list[str]
class BatchDownloadResponse(BaseModel):
job_id: str
status: str = "pending"
download_url: str | None = None
-2
View File
@@ -4,7 +4,6 @@ from .auto_clip_service import AutoClipService
from .edit_plan_service import EditPlanService
from .edit_template_service import EditTemplateService
from .job_service import JobService
from .plan_generator_service import PlanGeneratorService
from .video_compose_service import VideoComposeService
__all__ = [
@@ -12,6 +11,5 @@ __all__ = [
"EditPlanService",
"EditTemplateService",
"JobService",
"PlanGeneratorService",
"VideoComposeService",
]
+2 -3
View File
@@ -12,7 +12,6 @@
from __future__ import annotations
import logging
from collections.abc import Mapping
from dataclasses import dataclass
from sqlalchemy.orm import Session
@@ -25,7 +24,7 @@ from packages.adapters.sqlalchemy_impl import (
)
from packages.domain.asset import AssetType
from packages.domain.classification import AssetClassification
from packages.domain.edit_plan_clip import EditPlanClip
from packages.domain.edit_plan_clip import EditPlanClip, EditPlanClipStatus
logger = logging.getLogger(__name__)
@@ -156,7 +155,7 @@ class AutoClipService:
self,
clip: EditPlanClip,
project_id: str,
config_map: Mapping[str, object],
config_map: dict[str, object],
) -> ClipAssignDetail:
"""为单个片段分配素材。"""
config = config_map.get(clip.template_clip_config_id) if clip.template_clip_config_id else None
-276
View File
@@ -1,276 +0,0 @@
"""封面管理服务.
提供封面配置管理和从视频抽帧生成封面的能力。
抽帧使用 FFmpeg,上传使用共享存储服务。
"""
from __future__ import annotations
import logging
import tempfile
from pathlib import Path
from typing import Any, Dict
logger = logging.getLogger(__name__)
# ── 常量 ──────────────────────────────────────────────────────────────────────
DEFAULT_COVER_WIDTH = 1080
DEFAULT_COVER_HEIGHT = 1920
DEFAULT_COVER_QUALITY = 5 # JPEG quality (1-31, 越小越好)
COVER_STORAGE_PREFIX = "covers"
class CoverService:
"""封面管理服务."""
def __init__(self, storage_service: Any, asset_repository: Any) -> None:
self._storage = storage_service
self._asset_repo = asset_repository
# ── 配置读写 ──────────────────────────────────────────────────────────
@staticmethod
def get_cover_config(plan_config: Dict[str, Any]) -> Dict[str, Any]:
"""从 plan.config 中提取封面配置.
Args:
plan_config: 剪辑计划的 config 字段
Returns:
封面配置 dict
"""
cover = plan_config.get("cover", {})
if not isinstance(cover, dict):
cover = {}
# 确保默认字段存在
return {
"type": cover.get("type", "ai_frame"),
"image_url": cover.get("image_url", ""),
"frame_time": cover.get("frame_time"),
}
# ── 抽帧生成封面 ──────────────────────────────────────────────────────
def extract_cover_from_clip(
self,
plan_id: str,
asset_id: str,
frame_time: float = 1.0,
*,
width: int = DEFAULT_COVER_WIDTH,
height: int = DEFAULT_COVER_HEIGHT,
quality: int = DEFAULT_COVER_QUALITY,
) -> Dict[str, Any]:
"""从指定素材的指定时间点抽取一帧作为封面.
Args:
plan_id: 剪辑计划 ID(用于生成存储路径)
asset_id: 素材 ID
frame_time: 抽帧时间点(秒)
width: 输出宽度
height: 输出高度
quality: JPEG 质量
Returns:
封面数据 dict,包含 type / image_url / frame_time
Raises:
ValueError: 素材不存在或不是视频
RuntimeError: 抽帧或上传失败
"""
# 1. 获取素材
asset = self._asset_repo.get(asset_id) if self._asset_repo else None
if not asset:
raise ValueError(f"素材不存在: {asset_id}")
storage_key = getattr(asset, "storage_key", "")
if not storage_key:
raise ValueError(f"素材没有文件: {asset_id}")
mime_type = getattr(asset, "mime_type", "")
if mime_type and not mime_type.startswith("video"):
raise ValueError(f"素材不是视频类型: {mime_type}")
# 2. 下载视频到临时目录
with tempfile.TemporaryDirectory(prefix="cover_extract_") as tmp_dir:
tmp_path = Path(tmp_dir)
video_path = tmp_path / f"source_{asset_id[:8]}"
logger.info("下载素材用于封面抽帧: asset_id=%s", asset_id)
try:
self._storage.download_file(storage_key, str(video_path))
except Exception as e:
raise RuntimeError(f"下载素材失败: {e}") from e
if not video_path.exists() or video_path.stat().st_size == 0:
raise RuntimeError("下载的素材文件为空")
# 3. FFmpeg 抽帧
output_path = tmp_path / "cover.jpg"
self._extract_frame(
video_path=video_path,
output_path=output_path,
time_sec=frame_time,
width=width,
height=height,
quality=quality,
)
if not output_path.exists() or output_path.stat().st_size == 0:
raise RuntimeError("封面抽帧失败")
# 4. 上传到 OSS
cover_key = f"{COVER_STORAGE_PREFIX}/{plan_id}/cover_{int(frame_time * 1000)}.jpg"
logger.info("上传封面到存储: key=%s", cover_key)
try:
self._storage.upload_file(
file_or_path=str(output_path),
storage_key=cover_key,
content_type="image/jpeg",
)
except Exception as e:
raise RuntimeError(f"上传封面失败: {e}") from e
# 5. 获取访问 URL
try:
image_url = self._storage.get_url(cover_key)
except Exception:
image_url = cover_key # 降级为 storage_key
logger.info(
"封面抽帧完成: plan_id=%s asset_id=%s time=%.2fs size=%d",
plan_id,
asset_id,
frame_time,
output_path.stat().st_size if output_path.exists() else 0,
)
return {
"type": "manual",
"image_url": image_url,
"frame_time": frame_time,
}
def generate_smart_cover(
self,
plan_id: str,
asset_id: str,
*,
width: int = DEFAULT_COVER_WIDTH,
height: int = DEFAULT_COVER_HEIGHT,
quality: int = DEFAULT_COVER_QUALITY,
) -> Dict[str, Any]:
"""智能选帧:从视频中选取多帧,选最清晰的一帧.
Args:
plan_id: 剪辑计划 ID
asset_id: 素材 ID
width: 输出宽度
height: 输出高度
quality: JPEG 质量
Returns:
封面数据 dict
"""
# 简单实现:取视频 1/3 处的帧作为智能封面
# 更复杂的多帧选清晰帧可以后续优化
frame_time = 3.0 # 默认第3秒,后续可以根据视频时长动态计算
result = self.extract_cover_from_clip(
plan_id=plan_id,
asset_id=asset_id,
frame_time=frame_time,
width=width,
height=height,
quality=quality,
)
result["type"] = "ai_frame"
return result
# ── 内部方法 ──────────────────────────────────────────────────────────
@staticmethod
def _extract_frame(
video_path: Path,
output_path: Path,
*,
time_sec: float,
width: int,
height: int,
quality: int,
) -> None:
"""使用 FFmpeg 从视频中抽取一帧.
Args:
video_path: 视频文件路径
output_path: 输出图片路径
time_sec: 抽帧时间点(秒)
width: 输出宽度
height: 输出高度
quality: JPEG 质量
"""
import subprocess
# scale + crop 实现 cover 裁剪
vf = f"scale={width}:{height}:force_original_aspect_ratio=increase," f"crop={width}:{height}"
command = [
"ffmpeg",
"-y",
"-ss",
f"{time_sec:.3f}",
"-i",
str(video_path),
"-vframes",
"1",
"-vf",
vf,
"-q:v",
str(quality),
"-f",
"mjpeg",
str(output_path),
]
logger.debug("FFmpeg 抽帧命令: %s", " ".join(command))
try:
result = subprocess.run(
command,
capture_output=True,
text=True,
timeout=60,
)
if result.returncode != 0:
logger.warning("FFmpeg 抽帧返回非零: %s\nstderr: %s", result.returncode, result.stderr[-500:])
# 尝试不使用 scale+crop 的简化命令
simple_command = [
"ffmpeg",
"-y",
"-ss",
f"{time_sec:.3f}",
"-i",
str(video_path),
"-vframes",
"1",
"-q:v",
str(quality),
"-f",
"mjpeg",
str(output_path),
]
result2 = subprocess.run(
simple_command,
capture_output=True,
text=True,
timeout=60,
)
if result2.returncode != 0:
raise RuntimeError(f"FFmpeg 抽帧失败: {result2.stderr[-300:]}")
except subprocess.TimeoutExpired as e:
raise RuntimeError("FFmpeg 抽帧超时") from e
except FileNotFoundError as e:
raise RuntimeError("FFmpeg 不可用") from e
+4 -611
View File
@@ -18,6 +18,7 @@ from packages.adapters.sqlalchemy_impl import (
)
from packages.domain.edit_plan import EditPlan, EditPlanStatus
from packages.domain.edit_plan_clip import EditPlanClip, EditPlanClipStatus
from packages.domain.generation_task import GenerationTaskStatus
logger = logging.getLogger(__name__)
@@ -141,19 +142,6 @@ class EditPlanService:
logger.info("创建剪辑计划: id=%s name=%s", created.id, created.name)
return created
def _auto_resume_editing(self, plan_id: str) -> None:
"""如果计划处于 completed/failed 状态,自动切回 editing(编辑操作前置)"""
plan = self._plan_repo.get(plan_id)
if plan is None:
return
if plan.status in (EditPlanStatus.COMPLETED, EditPlanStatus.FAILED):
try:
plan.resume_editing()
self._plan_repo.update(plan)
logger.info("自动重新编辑: plan_id=%s", plan_id)
except ValueError:
pass
def update_plan(
self,
plan_id: str,
@@ -169,10 +157,6 @@ class EditPlanService:
"""
existing = self.get_plan_or_raise(plan_id)
# 自动从 completed/failed 切回 editing
self._auto_resume_editing(plan_id)
existing = self.get_plan_or_raise(plan_id)
updated = EditPlan(
id=existing.id,
template_id=existing.template_id,
@@ -229,24 +213,8 @@ class EditPlanService:
return plan
# 根据目标状态调用对应的状态机方法
# EDITING 支持从 draft / completed / failed 进入
if target_status == EditPlanStatus.EDITING:
if plan.status == EditPlanStatus.DRAFT:
plan.start_editing()
elif plan.status in (EditPlanStatus.COMPLETED, EditPlanStatus.FAILED):
plan.resume_editing()
else:
raise ValueError(f"无法从 {plan.status} 切换到 {target_status}")
result = self._plan_repo.update(plan)
logger.info(
"状态流转: plan_id=%s %s%s",
plan_id,
plan.status,
target_status,
)
return result
transition_map = {
EditPlanStatus.EDITING: plan.start_editing,
EditPlanStatus.RENDERING: plan.start_rendering,
EditPlanStatus.COMPLETED: plan.mark_completed,
EditPlanStatus.FAILED: plan.mark_failed,
@@ -314,8 +282,6 @@ class EditPlanService:
start_time: float = 0.0,
duration: float = 0.0,
transition_effect: str = "cut",
transition_duration: float = 0.0,
playback_speed: float = 1.0,
config: Optional[dict[str, Any]] = None,
) -> EditPlanClip:
"""创建片段
@@ -325,8 +291,6 @@ class EditPlanService:
"""
# 确保计划存在
self.get_plan_or_raise(plan_id)
# 自动从 completed/failed 切回 editing
self._auto_resume_editing(plan_id)
clip = EditPlanClip.create(
plan_id=plan_id,
@@ -338,8 +302,6 @@ class EditPlanService:
start_time=start_time,
duration=duration,
transition_effect=transition_effect,
transition_duration=transition_duration,
playback_speed=playback_speed,
config=config,
)
created = self._clip_repo.create(clip)
@@ -363,8 +325,6 @@ class EditPlanService:
start_time: Optional[float] = None,
duration: Optional[float] = None,
transition_effect: Optional[str] = None,
transition_duration: Optional[float] = None,
playback_speed: Optional[float] = None,
config: Optional[dict[str, Any]] = None,
) -> EditPlanClip:
"""更新片段
@@ -374,18 +334,6 @@ class EditPlanService:
"""
existing = self.get_clip_or_raise(clip_id)
# 自动从 completed/failed 切回 editing
self._auto_resume_editing(existing.plan_id)
# 速度边界钳制
if playback_speed is not None:
if playback_speed <= 0:
playback_speed = 1.0
elif playback_speed < 0.25:
playback_speed = 0.25
elif playback_speed > 4.0:
playback_speed = 4.0
updated = EditPlanClip(
id=existing.id,
plan_id=existing.plan_id,
@@ -399,10 +347,6 @@ class EditPlanService:
transition_effect=(
transition_effect.strip() if transition_effect is not None else existing.transition_effect
),
transition_duration=(
transition_duration if transition_duration is not None else existing.transition_duration
),
playback_speed=playback_speed if playback_speed is not None else existing.playback_speed,
status=existing.status,
config=config if config is not None else existing.config,
created_at=existing.created_at,
@@ -419,8 +363,6 @@ class EditPlanService:
ValueError: 片段不存在或 asset_id 为空
"""
clip = self.get_clip_or_raise(clip_id)
# 自动从 completed/failed 切回 editing
self._auto_resume_editing(clip.plan_id)
clip.assign_asset(asset_id)
result = self._clip_repo.update(clip)
logger.info("分配素材: clip_id=%s asset_id=%s", clip_id, asset_id)
@@ -447,463 +389,7 @@ class EditPlanService:
logger.info("删除所有片段: plan_id=%s count=%d", plan_id, count)
return count
def create_clips_from_assets(
self,
plan_id: str,
asset_ids: list[str],
*,
clip_type: str = "main",
) -> list[EditPlanClip]:
"""从素材批量创建片段(追加到时间线末尾)。
Args:
plan_id: 计划 ID
asset_ids: 素材 ID 列表(按顺序追加)
clip_type: 片段类型
Returns:
list[EditPlanClip]: 创建的片段列表
"""
if not asset_ids:
return []
# 确保计划存在 + 自动回退状态
self.get_plan_or_raise(plan_id)
self._auto_resume_editing(plan_id)
# 查询素材信息(取 duration)
from packages.adapters.sqlalchemy_impl.models import AssetModel
session = self._clip_repo.session # type: ignore[attr-defined]
assets = session.query(AssetModel).filter(AssetModel.id.in_(asset_ids)).all()
asset_map = {a.id: a for a in assets}
# 从现有片段数量开始追加
existing_count = self._clip_repo.count(plan_id=plan_id)
# 批量创建片段
created: list[EditPlanClip] = []
for i, asset_id in enumerate(asset_ids):
asset = asset_map.get(asset_id)
duration = asset.duration if asset and asset.duration else 0.0
clip = self.create_clip(
plan_id=plan_id,
clip_type=clip_type,
order=existing_count + i,
asset_id=asset_id,
duration=duration,
)
created.append(clip)
logger.info(
"从素材批量创建片段: plan_id=%s count=%d",
plan_id,
len(created),
)
return created
# ── 渲染生成流程 ────────────────────────────────────────────────────────
# ── 片段分割与合并 ──────────────────────────────────────────────────────
def split_clip(self, clip_id: str, split_time: float) -> Dict[str, Any]:
"""将一个片段从指定位置分割为两个片段
Args:
clip_id: 要分割的片段 ID
split_time: 分割点(相对于片段起始的秒数),必须在 (0, duration) 范围内
Returns:
dict: {"left_clip": EditPlanClip, "right_clip": EditPlanClip}
Raises:
ValueError: 片段不存在、分割时间越界
"""
clip = self.get_clip_or_raise(clip_id)
plan_id = clip.plan_id
if split_time <= 0 or split_time >= clip.duration:
raise ValueError(f"分割时间必须在 (0, {clip.duration:.3f}) 范围内,当前: {split_time}")
self._auto_resume_editing(plan_id)
original_duration = clip.duration
left_duration = round(split_time, 3)
right_duration = round(original_duration - split_time, 3)
original_order = clip.order
# 更新左半部分(原片段)
clip.duration = left_duration
left_clip = self._clip_repo.update(clip)
# 后面片段的 order 全部 +1(给右半部分腾位置)
all_clips = self._clip_repo.list_by_plan(plan_id)
for c in all_clips:
if c.order > original_order and c.id != clip_id:
c.order += 1
self._clip_repo.update(c)
# 创建右半部分新片段(继承原片段的大部分属性)
right_config = dict(clip.config) if clip.config else {}
# 素材裁剪信息
if clip.asset_id:
# 右半部分从 split_time 开始播放
right_config["trim_start"] = left_duration
# 左半部分在 split_time 处结束
left_config = dict(left_clip.config) if left_clip.config else {}
left_config["trim_end"] = right_duration
left_clip.config = left_config
left_clip = self._clip_repo.update(left_clip)
right_clip = EditPlanClip.create(
plan_id=plan_id,
clip_type=clip.clip_type,
order=original_order + 1,
template_clip_config_id=clip.template_clip_config_id,
asset_id=clip.asset_id,
text_content=clip.text_content,
start_time=clip.start_time + left_duration,
duration=right_duration,
transition_effect=clip.transition_effect,
transition_duration=clip.transition_duration,
playback_speed=clip.playback_speed,
config=right_config,
)
created_right = self._clip_repo.create(right_clip)
logger.info(
"分割片段: clip_id=%s plan_id=%s split_time=%.3fs left_dur=%.3fs right_dur=%.3fs",
clip_id,
plan_id,
split_time,
left_duration,
right_duration,
)
return {
"left_clip": left_clip,
"right_clip": created_right,
}
def merge_clips(self, clip_ids: List[str]) -> EditPlanClip:
"""合并多个连续片段为一个片段
Args:
clip_ids: 要合并的片段 ID 列表(至少2个),必须属于同一个计划且 order 连续
Returns:
EditPlanClip: 合并后的新片段
Raises:
ValueError: 数量不足、不属于同一计划、不连续、类型不一致
"""
if len(clip_ids) < 2:
raise ValueError("至少需要 2 个片段才能合并")
# 读取所有片段
clips = []
for cid in clip_ids:
clip = self.get_clip_or_raise(cid)
clips.append(clip)
# 校验:同一计划
plan_id = clips[0].plan_id
for c in clips[1:]:
if c.plan_id != plan_id:
raise ValueError("只能合并同一计划下的片段")
# 按 order 排序
clips.sort(key=lambda c: c.order)
# 校验:order 连续
for i in range(1, len(clips)):
if clips[i].order != clips[i - 1].order + 1:
raise ValueError(f"片段不连续:order {clips[i-1].order}{clips[i].order}")
# 校验:类型一致
clip_type = clips[0].clip_type
for c in clips[1:]:
if c.clip_type != clip_type:
raise ValueError("只能合并相同类型的片段")
self._auto_resume_editing(plan_id)
# 计算合并后的属性
first_clip = clips[0]
total_duration = round(sum(c.duration for c in clips), 3)
first_order = first_clip.order
# 合并文案(用换行连接)
merged_text = "\n".join(c.text_content for c in clips if c.text_content.strip())
# 合并 config(后面的覆盖前面的)
merged_config: Dict[str, Any] = {}
for c in clips:
if c.config:
merged_config.update(c.config)
# 清理 trim 相关字段(合并后就是完整片段了)
merged_config.pop("trim_start", None)
merged_config.pop("trim_end", None)
# 更新第一个片段(保留它作为合并结果)
first_clip.duration = total_duration
first_clip.text_content = merged_text
first_clip.config = merged_config
# 转场保留第一个的(合并后的入点转场)
# playback_speed 取第一个的
merged_clip = self._clip_repo.update(first_clip)
# 删除其余片段
for c in clips[1:]:
self._clip_repo.delete(c.id)
# 后面的片段 order 前移 (len - 1) 位
shift = len(clips) - 1
all_clips = self._clip_repo.list_by_plan(plan_id)
for c in all_clips:
if c.order > first_order and c.id != merged_clip.id:
c.order -= shift
self._clip_repo.update(c)
logger.info(
"合并片段: plan_id=%s count=%d total_duration=%.3fs",
plan_id,
len(clips),
total_duration,
)
return merged_clip
# ── 字幕管理 ──────────────────────────────────────────────────────────
def list_subtitles(self, clip_id: str) -> List[Dict[str, Any]]:
"""获取片段的所有字幕
Returns:
List[dict]: 字幕列表,按 start 时间排序
"""
clip = self.get_clip_or_raise(clip_id)
config = clip.config or {}
subtitles = config.get("subtitles", [])
# 按开始时间排序
subtitles.sort(key=lambda s: s.get("start", 0))
return subtitles
def get_subtitle(self, clip_id: str, subtitle_id: str) -> Optional[Dict[str, Any]]:
"""获取单条字幕"""
subtitles = self.list_subtitles(clip_id)
for s in subtitles:
if s.get("id") == subtitle_id:
return s
return None
def add_subtitle(
self,
clip_id: str,
start: float,
end: float,
text: str,
*,
style: Optional[Dict[str, Any]] = None,
) -> Dict[str, Any]:
"""添加一条字幕
Args:
clip_id: 片段 ID
start: 开始时间(秒,相对于片段)
end: 结束时间(秒)
text: 字幕文本
style: 样式配置(字体、大小、颜色、位置等)
Returns:
dict: 新增的字幕条目
Raises:
ValueError: 时间非法或文本为空
"""
clip = self.get_clip_or_raise(clip_id)
if start < 0 or end <= start:
raise ValueError(f"字幕时间非法: start={start}, end={end}")
if not text.strip():
raise ValueError("字幕文本不能为空")
if end > clip.duration + 0.001:
raise ValueError(f"字幕结束时间不能超过片段时长: end={end:.3f}, duration={clip.duration:.3f}")
self._auto_resume_editing(clip.plan_id)
from uuid import uuid4
config = dict(clip.config) if clip.config else {}
subtitles = list(config.get("subtitles", []))
subtitle = {
"id": uuid4().hex,
"start": round(start, 3),
"end": round(end, 3),
"text": text.strip(),
"style": style or {},
}
subtitles.append(subtitle)
subtitles.sort(key=lambda s: s.get("start", 0))
config["subtitles"] = subtitles
clip.config = config
self._clip_repo.update(clip)
logger.info(
"添加字幕: clip_id=%s subtitle_id=%s start=%.3fs end=%.3fs",
clip_id,
subtitle["id"],
start,
end,
)
return subtitle
def update_subtitle(
self,
clip_id: str,
subtitle_id: str,
*,
start: Optional[float] = None,
end: Optional[float] = None,
text: Optional[str] = None,
style: Optional[Dict[str, Any]] = None,
) -> Dict[str, Any]:
"""更新一条字幕
Returns:
dict: 更新后的字幕条目
Raises:
ValueError: 字幕不存在或参数非法
"""
clip = self.get_clip_or_raise(clip_id)
config = dict(clip.config) if clip.config else {}
subtitles = list(config.get("subtitles", []))
found = False
for i, s in enumerate(subtitles):
if s.get("id") == subtitle_id:
# 更新字段
updated_s = dict(s)
if start is not None:
updated_s["start"] = round(start, 3)
if end is not None:
updated_s["end"] = round(end, 3)
if text is not None:
if not text.strip():
raise ValueError("字幕文本不能为空")
updated_s["text"] = text.strip()
if style is not None:
updated_s["style"] = style
# 校验时间
if updated_s["start"] < 0 or updated_s["end"] <= updated_s["start"]:
raise ValueError(f"字幕时间非法: start={updated_s['start']}, end={updated_s['end']}")
if updated_s["end"] > clip.duration + 0.001:
raise ValueError("字幕结束时间不能超过片段时长")
subtitles[i] = updated_s
found = True
break
if not found:
raise ValueError(f"字幕不存在: {subtitle_id}")
self._auto_resume_editing(clip.plan_id)
subtitles.sort(key=lambda s: s.get("start", 0))
config["subtitles"] = subtitles
clip.config = config
self._clip_repo.update(clip)
logger.info("更新字幕: clip_id=%s subtitle_id=%s", clip_id, subtitle_id)
return subtitles[next(i for i, s in enumerate(subtitles) if s["id"] == subtitle_id)]
def delete_subtitle(self, clip_id: str, subtitle_id: str) -> bool:
"""删除一条字幕
Returns:
bool: 是否删除成功
"""
clip = self.get_clip_or_raise(clip_id)
config = dict(clip.config) if clip.config else {}
subtitles = list(config.get("subtitles", []))
new_subtitles = [s for s in subtitles if s.get("id") != subtitle_id]
if len(new_subtitles) == len(subtitles):
return False
self._auto_resume_editing(clip.plan_id)
config["subtitles"] = new_subtitles
clip.config = config
self._clip_repo.update(clip)
logger.info("删除字幕: clip_id=%s subtitle_id=%s", clip_id, subtitle_id)
return True
def batch_update_subtitles(
self,
clip_id: str,
subtitles: List[Dict[str, Any]],
) -> List[Dict[str, Any]]:
"""批量更新字幕(全量替换,用于批量编辑或导入)
Args:
clip_id: 片段 ID
subtitles: 字幕列表,每条需包含 start/end/text,已有 id 则保留
Returns:
List[dict]: 更新后的字幕列表
"""
clip = self.get_clip_or_raise(clip_id)
from uuid import uuid4
validated = []
for s in subtitles:
start = float(s.get("start", 0))
end = float(s.get("end", 0))
text = str(s.get("text", ""))
if start < 0 or end <= start:
raise ValueError(f"字幕时间非法: start={start}, end={end}")
if not text.strip():
continue # 跳过空字幕
if end > clip.duration + 0.001:
raise ValueError(f"字幕结束时间不能超过片段时长: end={end}")
subtitle_id = s.get("id") or uuid4().hex
validated.append(
{
"id": subtitle_id,
"start": round(start, 3),
"end": round(end, 3),
"text": text.strip(),
"style": s.get("style", {}),
}
)
validated.sort(key=lambda s: s["start"])
self._auto_resume_editing(clip.plan_id)
config = dict(clip.config) if clip.config else {}
config["subtitles"] = validated
clip.config = config
self._clip_repo.update(clip)
logger.info(
"批量更新字幕: clip_id=%s count=%d",
clip_id,
len(validated),
)
return validated
def get_plan_with_clips(self, plan_id: str) -> Dict[str, Any]:
"""获取计划及其所有片段
@@ -927,8 +413,6 @@ class EditPlanService:
"clips": List[EditPlanClip],
"generation_task_id": Optional[str],
"generation_task_status": Optional[str],
"progress": float,
"error_message": str,
}
Raises:
@@ -940,23 +424,17 @@ class EditPlanService:
# 从 plan.config 中获取 generation_task_id
generation_task_id = plan.config.get("generation_task_id")
generation_task_status = None
progress = 0.0
error_message = ""
if generation_task_id:
task = self._generation_task_repo.get(generation_task_id)
if task:
generation_task_status = task.status.value if hasattr(task.status, "value") else task.status
progress = getattr(task, "progress", 0.0) or 0.0
error_message = getattr(task, "error_message", "") or ""
return {
"plan": plan,
"clips": clips,
"generation_task_id": generation_task_id,
"generation_task_status": generation_task_status,
"progress": progress,
"error_message": error_message,
}
def can_generate(self, plan_id: str) -> tuple[bool, str]:
@@ -969,12 +447,12 @@ class EditPlanService:
# 检查状态
if plan.status != EditPlanStatus.EDITING:
return False, "请先编辑并保存模板后再生成视频"
return False, f"只有 editing 状态的计划可以触发渲染,当前状态: {plan.status}"
# 检查是否有片段
clips = self._clip_repo.list_by_plan(plan_id)
if not clips:
return False, "请先添加片段后再生成视频"
return False, "计划下没有片段,无法触发渲染"
return True, ""
@@ -1007,9 +485,6 @@ class EditPlanService:
更新后的计划
"""
plan = self.get_plan_or_raise(plan_id)
# 自动从 completed/failed 切回 editing
self._auto_resume_editing(plan_id)
plan = self.get_plan_or_raise(plan_id)
new_config = {**plan.config, **config_updates}
updated = EditPlan(
@@ -1026,85 +501,3 @@ class EditPlanService:
updated_at=plan.updated_at,
)
return self._plan_repo.update(updated)
# ── 复制计划 ────────────────────────────────────────────────────────────
def copy_plan(
self,
plan_id: str,
*,
new_name: Optional[str] = None,
project_id: Optional[str] = None,
) -> EditPlan:
"""复制一个剪辑计划(含所有片段配置)。
新计划状态为 editing,不含生成任务和结果记录。
Args:
plan_id: 源计划 ID
new_name: 新计划名称,不传则为「原名 - 副本」
project_id: 新计划的项目 ID,不传则复用源计划
Returns:
EditPlan: 新创建的计划
Raises:
ValueError: 源计划不存在
"""
source = self.get_plan_or_raise(plan_id)
source_clips = self._clip_repo.list_by_plan(plan_id)
# 新计划名称
name = new_name or f"{source.name} - 副本"
new_project_id = project_id if project_id is not None else source.project_id
# 复制 plan 配置(去除渲染结果相关字段)
new_config = dict(source.config)
new_config.pop("rendered_url", None)
new_config.pop("rendered_storage_key", None)
new_config.pop("generation_task_id", None)
# 创建新计划
new_plan = EditPlan.create(
template_id=source.template_id,
name=name,
config=new_config,
total_duration=source.total_duration,
project_id=new_project_id,
created_by_user_id=source.created_by_user_id,
source_edit_plan_id=plan_id,
)
# 强制切到 editing 状态
if new_plan.status != EditPlanStatus.EDITING:
try:
new_plan.start_editing()
except ValueError:
pass
created_plan = self._plan_repo.create(new_plan)
logger.info(
"复制剪辑计划: source=%s target=%s name=%s clips=%d",
plan_id,
created_plan.id,
name,
len(source_clips),
)
# 复制所有片段
for clip in source_clips:
new_clip = self.create_clip(
plan_id=created_plan.id,
clip_type=clip.clip_type,
order=clip.order,
asset_id=clip.asset_id or "",
text_content=clip.text_content or "",
start_time=clip.start_time,
duration=clip.duration,
transition_effect=clip.transition_effect or "cut",
transition_duration=clip.transition_duration or 0.0,
playback_speed=clip.playback_speed or 1.0,
config=dict(clip.config) if clip.config else None,
)
logger.debug("复制片段: source=%s target=%s order=%d", clip.id, new_clip.id, clip.order)
return self.get_plan_or_raise(created_plan.id)
-143
View File
@@ -12,8 +12,6 @@ from typing import Any, List, Optional
from sqlalchemy.orm import Session
from packages.adapters.sqlalchemy_impl import (
SQLAlchemyEditPlanClipRepository,
SQLAlchemyEditPlanRepository,
SQLAlchemyEditTemplateRepository,
SQLAlchemyTemplateClipConfigRepository,
)
@@ -39,9 +37,6 @@ class EditTemplateService:
def __init__(self, db: Session) -> None:
self._template_repo = SQLAlchemyEditTemplateRepository(db)
self._clip_config_repo = SQLAlchemyTemplateClipConfigRepository(db)
self._plan_repo = SQLAlchemyEditPlanRepository(db)
self._plan_clip_repo = SQLAlchemyEditPlanClipRepository(db)
self._db = db
# ── 模板 CRUD ──────────────────────────────────────────────────────────
@@ -105,7 +100,6 @@ class EditTemplateService:
*,
description: str = "",
template_type: str = "default",
editing_mode: str = "one_take",
config: Optional[dict[str, Any]] = None,
preview_url: str = "",
sort_weight: int = 0,
@@ -130,7 +124,6 @@ class EditTemplateService:
name=clean_name,
description=description,
template_type=template_type,
editing_mode=editing_mode,
config=config,
preview_url=preview_url,
sort_weight=sort_weight,
@@ -146,7 +139,6 @@ class EditTemplateService:
name: Optional[str] = None,
description: Optional[str] = None,
template_type: Optional[str] = None,
editing_mode: Optional[str] = None,
config: Optional[dict[str, Any]] = None,
preview_url: Optional[str] = None,
sort_weight: Optional[int] = None,
@@ -173,7 +165,6 @@ class EditTemplateService:
name=new_name,
description=description.strip() if description is not None else existing.description,
template_type=template_type.strip() if template_type is not None else existing.template_type,
editing_mode=editing_mode.strip() if editing_mode is not None else existing.editing_mode,
config=config if config is not None else existing.config,
preview_url=preview_url.strip() if preview_url is not None else existing.preview_url,
sort_weight=sort_weight if sort_weight is not None else existing.sort_weight,
@@ -399,137 +390,3 @@ class EditTemplateService:
"template": template,
"clip_configs": clip_configs,
}
# ── 从剪辑计划保存为模板 ──────────────────────────────────────────────
def save_plan_as_template(
self,
plan_id: str,
name: str,
*,
description: str = "",
template_type: str = "custom",
preview_url: str = "",
) -> dict[str, Any]:
"""将剪辑计划保存为模板
将指定剪辑计划的配置和片段结构另存为一个新模板,
方便后续基于该模板快速创建新的剪辑计划。
转换规则:
- 计划名称 → 模板名称(调用方传入,支持自定义)
- 计划 config → 模板 config(整体迁移)
- 计划 editing_mode 从 config 中提取,默认 one_take
- 每个片段转换为模板片段配置:
- clip_type 直接映射
- order 保持不变
- duration → min_duration = max_duration = duration(固定时长)
- text_content → text_template
- transition_effect 直接映射
- playback_speed 等播放参数存入 config
- 不保留 asset_id(模板不绑定具体素材)
Args:
plan_id: 源剪辑计划 ID
name: 新模板名称
description: 模板描述
template_type: 模板类型,默认 custom(用户自定义)
preview_url: 预览图 URL
Returns:
dict: {"template": EditTemplate, "clip_configs": List[TemplateClipConfig]}
Raises:
ValueError: 计划不存在或名称为空/重复
"""
# 1. 读取源计划
plan = self._plan_repo.get(plan_id)
if plan is None:
raise ValueError(f"剪辑计划不存在: {plan_id}")
# 2. 读取所有片段(按 order 排序)
clips = self._plan_clip_repo.list_by_plan(plan_id)
clips.sort(key=lambda c: c.order)
# 3. 提取 editing_mode
editing_mode = plan.config.get("editing_mode", "one_take") if plan.config else "one_take"
# 4. 创建模板(复用 create_template 的校验逻辑,但手动构建避免重复查询)
clean_name = name.strip()
if not clean_name:
raise ValueError("模板名称不能为空")
# 名称重复检查
existing = self._template_repo.list_all(skip=0, limit=1000)
for t in existing:
if t.name == clean_name and t.status == EditTemplateStatus.ACTIVE:
raise ValueError(f"模板名称已存在: {clean_name}")
# 从计划 config 中提取模板级配置,去掉运行时/素材相关字段
plan_config = plan.config or {}
template_config: dict[str, Any] = {}
for key, value in plan_config.items():
# 跳过明显的运行时/实例字段,保留风格/模式类配置
if key not in {"asset_ids", "source_edit_plan_id", "generation_task_id"}:
template_config[key] = value
template = EditTemplate.create(
name=clean_name,
description=description,
template_type=template_type,
editing_mode=editing_mode,
config=template_config,
preview_url=preview_url,
)
created_template = self._template_repo.create(template)
logger.info(
"从剪辑计划创建模板: plan_id=%s template_id=%s name=%s clip_count=%d",
plan_id,
created_template.id,
clean_name,
len(clips),
)
# 5. 转换每个片段为模板片段配置
created_configs: List[TemplateClipConfig] = []
for clip in clips:
clip_config: dict[str, Any] = {}
# 播放速度存入 config
if clip.playback_speed and clip.playback_speed != 1.0:
clip_config["playback_speed"] = clip.playback_speed
# 片段自有 config 合并(优先级:clip.config 覆盖上面的)
if clip.config:
clip_config.update(clip.config)
# 去掉素材相关字段
clip_config.pop("asset_info", None)
clip_config.pop("source_asset_id", None)
# 转场效果兼容校验
try:
transition = TransitionEffect(clip.transition_effect)
except ValueError:
transition = TransitionEffect.CUT
# 片段类型兼容校验
try:
clip_type = ClipType(clip.clip_type)
except ValueError:
clip_type = ClipType.MAIN
clip_config_obj = TemplateClipConfig.create(
template_id=created_template.id,
clip_type=clip_type,
order=clip.order,
min_duration=clip.duration,
max_duration=clip.duration,
text_template=clip.text_content or "",
transition_effect=transition,
config=clip_config,
)
created = self._clip_config_repo.create(clip_config_obj)
created_configs.append(created)
return {
"template": created_template,
"clip_configs": created_configs,
}
+2
View File
@@ -13,6 +13,8 @@ from __future__ import annotations
import logging
from typing import Any
from sqlalchemy.orm import Session
from packages.application.jobs import (
CancelJobUseCase,
CompleteJobCommand,

Some files were not shown because too many files have changed in this diff Show More