fix(P0): gpu-encoder 冷启动防护 — pre-warm + 首字节短超时兜底 #2072
Merged
auto-approve-bot
merged 4 commits from 2026-09-28 00:08:51 +08:00
fix/p0-gpu-encoder-cold-start-guard into develop
4 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 CI Bot
|
0bdffda8e0 |
style: auto-format with black + isort + ruff + prettier [skip ci-format-check]
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 1s
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 47s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 49s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m11s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 2m23s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m59s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 3m6s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 5m8s
AI Code Review / AI Code Review (pull_request) Successful in 6m35s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 9m23s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 9m35s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 1s
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 7m20s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 39s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Successful in 4m14s
|
||
|
saas-backend-agent
|
5eb57c6fa6 |
fix(gpu-encoder): harden resp.fp mock compat + keep "connection error" keyword in timeout msg for test compat
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 46s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m14s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 2m14s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 3m26s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 3m46s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 7m17s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 9m16s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 10m29s
AI Code Review / AI Code Review (pull_request) Successful in 10m48s
CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / CI Gate (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Style (pull_request) Has been cancelled
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been cancelled
|
||
|
CI Bot
|
3eba883ab6 |
style: auto-format with black + isort + ruff + prettier [skip ci-format-check]
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 1s
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 48s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m26s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m58s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 5m4s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 5m57s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 6m3s
AI Code Review / AI Code Review (pull_request) Successful in 6m33s
CI/CD Pipeline / PR Build Worker Image (pull_request) Failing after 7m24s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 11m14s
CI/CD Pipeline / Unit Tests (pull_request) Failing after 11m19s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Failing after 1s
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 8m37s
|
||
|
saas-backend-agent
|
114ce95c04 |
fix(gpu-encoder): add pre-warm + first-byte short timeout to guard against P4000 cold-start 150s stalls
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 23s
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m35s
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 1m55s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m12s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 3m30s
AI Code Review / AI Code Review (pull_request) Successful in 6m51s
CI/CD Pipeline / Unit Tests (pull_request) Failing after 9m39s
CI/CD Pipeline / Validate - Style (pull_request) Failing after 10m37s
CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / CI Gate (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Security (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Has been cancelled
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been cancelled
P0 issue: 2026-09-27 staging job a15a4667 observed 149.6s delay between worker POST and P4000 starting mezzanine download (nginx access log confirmed GET mezz at 14:12:55 vs POST at 14:10:26). Root cause is server/network-side (likely Tailscale DERP hole-punching or httpx connection pool rebuild after long idle), hard to reproduce on demand. This client-side fix adds two layers of defense: 1. pre_warm: before POST, if idle >60s since last successful comm, send a GET /health with 3s timeout to warm the Tailscale path. Failure is non-fatal; POST proceeds anyway. 2. post_first_byte_timeout (default 20s): split POST timeout into two phases — first byte uses short timeout so a stalled link fails fast and caller can CPU-fallback; after first byte, relax to ffmpeg_timeout+60s for real encode+upload. Both configurable via settings.gpu_encode_pre_warm / settings.gpu_encode_post_first_byte_timeout. Also updates check_health() to refresh _last_ok_ts on healthy response. |