feat(generation): worker孤儿任务自动恢复 + 429限流结构化提示 (#1677) (#1710)
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 2s
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 3s
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 3s
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / Check push changed paths (push) Successful in 5s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Validate - Style (pull_request) Has been skipped
CI/CD Pipeline / Validate - Security (pull_request) Has been skipped
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Has been skipped
CI/CD Pipeline / Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 21s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 23s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 24s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 32s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 32s
CI/CD Pipeline / Build Staging API Image (push) Successful in 32s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 7s
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 1m54s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m14s
CI/CD Pipeline / Integration Tests (push) Successful in 2m31s
CI/CD Pipeline / Validate - Style (push) Successful in 2m58s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 1m30s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m41s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m31s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1m56s
CI/CD Pipeline / Frontend Unit Tests (push) Successful in 5m29s
CI/CD Pipeline / Validate - Security (push) Successful in 6m18s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 3m14s
AI Code Review / AI Code Review (pull_request) Successful in 6m28s
CI/CD Pipeline / Unit Tests (push) Successful in 8m25s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 2s
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 3s
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 3s
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / Check push changed paths (push) Successful in 5s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Validate - Style (pull_request) Has been skipped
CI/CD Pipeline / Validate - Security (pull_request) Has been skipped
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Has been skipped
CI/CD Pipeline / Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 21s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 23s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 24s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 32s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 32s
CI/CD Pipeline / Build Staging API Image (push) Successful in 32s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 7s
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 1m54s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m14s
CI/CD Pipeline / Integration Tests (push) Successful in 2m31s
CI/CD Pipeline / Validate - Style (push) Successful in 2m58s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 1m30s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m41s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m31s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1m56s
CI/CD Pipeline / Frontend Unit Tests (push) Successful in 5m29s
CI/CD Pipeline / Validate - Security (push) Successful in 6m18s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 3m14s
AI Code Review / AI Code Review (pull_request) Successful in 6m28s
CI/CD Pipeline / Unit Tests (push) Successful in 8m25s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
Co-authored-by: xiaoxia <dev@xiaoxiajianji.com> Co-committed-by: xiaoxia <dev@xiaoxiajianji.com>
This commit was merged in pull request #1710.
This commit is contained in:
@@ -138,6 +138,52 @@ class SQLAlchemyGenerationTaskRepository:
|
||||
.count()
|
||||
)
|
||||
|
||||
def count_running_by_user(self, user_id: str) -> int:
|
||||
"""统计指定用户处于 running 状态的任务数(用于限流提示展示)。"""
|
||||
return (
|
||||
self.session.query(GenerationTaskModel)
|
||||
.filter(
|
||||
GenerationTaskModel.created_by_user_id == user_id,
|
||||
GenerationTaskModel.status == GenerationTaskStatus.RUNNING.value,
|
||||
)
|
||||
.count()
|
||||
)
|
||||
|
||||
def count_running_total(self) -> int:
|
||||
"""统计全局处于 running 状态的任务数(worker 实际在执行的任务数)。"""
|
||||
return (
|
||||
self.session.query(GenerationTaskModel)
|
||||
.filter(GenerationTaskModel.status == GenerationTaskStatus.RUNNING.value)
|
||||
.count()
|
||||
)
|
||||
|
||||
def estimate_avg_duration_seconds(self, limit: int = 20, default_seconds: float = 120.0) -> float:
|
||||
"""估算最近完成任务的平均耗时(秒),用于 429 限流提示的等待预估。
|
||||
|
||||
取最近 N 条 completed 任务的 (completed_at - started_at) 平均值;
|
||||
无足够历史数据时返回 default_seconds。
|
||||
用 Python 侧计算差值,避免 SQLite/PostgreSQL 方言差异。
|
||||
"""
|
||||
rows = (
|
||||
self.session.query(GenerationTaskModel.started_at, GenerationTaskModel.completed_at)
|
||||
.filter(
|
||||
GenerationTaskModel.status == GenerationTaskStatus.COMPLETED.value,
|
||||
GenerationTaskModel.started_at.isnot(None),
|
||||
GenerationTaskModel.completed_at.isnot(None),
|
||||
)
|
||||
.order_by(GenerationTaskModel.completed_at.desc())
|
||||
.limit(limit)
|
||||
.all()
|
||||
)
|
||||
durations = [
|
||||
(completed - started).total_seconds()
|
||||
for started, completed in rows
|
||||
if completed and started and (completed - started).total_seconds() > 0
|
||||
]
|
||||
if not durations:
|
||||
return default_seconds
|
||||
return sum(durations) / len(durations)
|
||||
|
||||
def list_recent_by_user(self, user_id: str, limit: int = 5) -> list[GenerationTask]:
|
||||
models = (
|
||||
self.session.query(GenerationTaskModel)
|
||||
|
||||
Reference in New Issue
Block a user