feat(generation): worker孤儿任务自动恢复 + 429限流结构化提示 (#1677) (#1710)
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 2s
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 3s
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 3s
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / Check push changed paths (push) Successful in 5s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Validate - Style (pull_request) Has been skipped
CI/CD Pipeline / Validate - Security (pull_request) Has been skipped
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Has been skipped
CI/CD Pipeline / Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 21s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 23s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 24s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 32s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 32s
CI/CD Pipeline / Build Staging API Image (push) Successful in 32s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 7s
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 1m54s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m14s
CI/CD Pipeline / Integration Tests (push) Successful in 2m31s
CI/CD Pipeline / Validate - Style (push) Successful in 2m58s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 1m30s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m41s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m31s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1m56s
CI/CD Pipeline / Frontend Unit Tests (push) Successful in 5m29s
CI/CD Pipeline / Validate - Security (push) Successful in 6m18s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 3m14s
AI Code Review / AI Code Review (pull_request) Successful in 6m28s
CI/CD Pipeline / Unit Tests (push) Successful in 8m25s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped

Co-authored-by: xiaoxia <dev@xiaoxiajianji.com>
Co-committed-by: xiaoxia <dev@xiaoxiajianji.com>
This commit was merged in pull request #1710.
This commit is contained in:
2026-09-05 11:37:49 +08:00
committed by auto-approve-bot
parent cf83c0df9f
commit 21c26b5b26
9 changed files with 614 additions and 41 deletions
@@ -138,6 +138,52 @@ class SQLAlchemyGenerationTaskRepository:
.count()
)
def count_running_by_user(self, user_id: str) -> int:
"""统计指定用户处于 running 状态的任务数(用于限流提示展示)。"""
return (
self.session.query(GenerationTaskModel)
.filter(
GenerationTaskModel.created_by_user_id == user_id,
GenerationTaskModel.status == GenerationTaskStatus.RUNNING.value,
)
.count()
)
def count_running_total(self) -> int:
"""统计全局处于 running 状态的任务数(worker 实际在执行的任务数)。"""
return (
self.session.query(GenerationTaskModel)
.filter(GenerationTaskModel.status == GenerationTaskStatus.RUNNING.value)
.count()
)
def estimate_avg_duration_seconds(self, limit: int = 20, default_seconds: float = 120.0) -> float:
"""估算最近完成任务的平均耗时(秒),用于 429 限流提示的等待预估。
取最近 N 条 completed 任务的 (completed_at - started_at) 平均值;
无足够历史数据时返回 default_seconds。
用 Python 侧计算差值,避免 SQLite/PostgreSQL 方言差异。
"""
rows = (
self.session.query(GenerationTaskModel.started_at, GenerationTaskModel.completed_at)
.filter(
GenerationTaskModel.status == GenerationTaskStatus.COMPLETED.value,
GenerationTaskModel.started_at.isnot(None),
GenerationTaskModel.completed_at.isnot(None),
)
.order_by(GenerationTaskModel.completed_at.desc())
.limit(limit)
.all()
)
durations = [
(completed - started).total_seconds()
for started, completed in rows
if completed and started and (completed - started).total_seconds() > 0
]
if not durations:
return default_seconds
return sum(durations) / len(durations)
def list_recent_by_user(self, user_id: str, limit: int = 5) -> list[GenerationTask]:
models = (
self.session.query(GenerationTaskModel)