fix(gpu-encoder): split relay_base_url into external/internal URLs
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 0s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 1s
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 49s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 53s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m7s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m54s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 3m13s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 3m26s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 3m41s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 8m24s
AI Code Review / AI Code Review (pull_request) Successful in 10m4s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 10m48s
CI/CD Pipeline / Unit Tests (pull_request) Failing after 15m33s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Failing after 1s
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 0s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 1s
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 49s
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 53s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m7s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m54s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 3m13s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 3m26s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 3m41s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 8m24s
AI Code Review / AI Code Review (pull_request) Successful in 10m4s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 10m48s
CI/CD Pipeline / Unit Tests (pull_request) Failing after 15m33s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Failing after 1s
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
- Add gpu_encode_relay_internal_base_url config (Docker DNS: http://xiaoxia-api-staging:8000) - P4000 PUT uses external URL via nginx (host:8092 plain HTTP to avoid SSL issues) - Worker GET/DELETE uses internal URL (direct Docker network, bypasses UFW/nginx) - Falls back to relay_base_url when internal URL not set (local dev) On staging: - New nginx vhost on :8092 provides plain HTTP proxy for P4000 relay PUTs - Docker network worker→API reachable at xiaoxia-api-staging:8000 (verified)
This commit is contained in:
committed by
saas-backend
parent
9648ce09d0
commit
d0e999f691
@@ -3,11 +3,13 @@
|
||||
完整链路(encode_video_file):
|
||||
1. CPU 滤镜已在本地生成 mezzanine 中间片(libx264 ultrafast)
|
||||
2. 上传 mezzanine 到 OSS 临时前缀,拿到签名 GET URL
|
||||
3. 生成 relay 一次性 key,构造带 token 的 PUT URL(指向 API 服务 /api/v1/internal/gpu-relay/<key>)
|
||||
4. POST P4000 /api/render/sync:inputs={"in.mp4": "<oss-signed-url>"}, output_url="<relay-put-url>"
|
||||
3. 生成 relay 一次性 key,构造两个带 token 的 URL:
|
||||
- put_url:给 P4000 回传结果,走 relay_base_url(外部可达,通常是 host:port 经 nginx)
|
||||
- get/del_url:worker 自己下载+清理用,走 relay_internal_base_url(Docker DNS 直连 API)
|
||||
4. POST P4000 /api/render/sync:inputs={"in.mp4": "<oss-signed-url>"}, output_url="<put_url>"
|
||||
ffmpeg_args: -i in.mp4 [-vf <vf>] -c:v h264_nvenc ... -an/-c:a aac -f mp4 pipe:1
|
||||
5. P4000 编码完成后 PUT 最终 mp4 到 relay,API 服务落盘到 /app/generated/gpu_relay/<key>
|
||||
6. 本客户端从 relay GET 下载最终文件到 output_path,然后调用 relay DELETE 清理
|
||||
5. P4000 编码完成后 PUT 最终 mp4 到 put_url,API 服务落盘到 /app/generated/gpu_relay/<key>
|
||||
6. 本客户端通过 get_url(Docker 内网)下载最终文件到 output_path,然后 DELETE 清理
|
||||
7. 删除 OSS 临时 mezzanine
|
||||
|
||||
任何环节失败抛 GpuEncodeError,调用方应 fallback 到 CPU libx264。
|
||||
@@ -55,6 +57,7 @@ class GpuEncoderClient:
|
||||
endpoint: str,
|
||||
relay_base_url: str,
|
||||
*,
|
||||
relay_internal_base_url: str = "",
|
||||
sync_timeout: int = 300,
|
||||
health_timeout: float = 3.0,
|
||||
vcodec: str = "h264_nvenc",
|
||||
@@ -66,6 +69,11 @@ class GpuEncoderClient:
|
||||
) -> None:
|
||||
self.endpoint = endpoint.rstrip("/")
|
||||
self.relay_base_url = relay_base_url.rstrip("/")
|
||||
# Worker→API 内网访问地址(Docker DNS 直连,如 http://xiaoxia-api-staging:8000)。
|
||||
# 未配置时回退到 relay_base_url(本地开发/单节点)。
|
||||
self.relay_internal_base_url = (
|
||||
relay_internal_base_url.rstrip("/") if relay_internal_base_url else self.relay_base_url
|
||||
)
|
||||
self.sync_timeout = sync_timeout
|
||||
self.health_timeout = health_timeout
|
||||
self.vcodec = vcodec
|
||||
@@ -75,6 +83,22 @@ class GpuEncoderClient:
|
||||
self._relay_secret = relay_secret
|
||||
self.oss_tmp_prefix = oss_tmp_prefix.rstrip("/") + "/" if oss_tmp_prefix else "tmp/gpu-mezzanine/"
|
||||
|
||||
RELAY_PATH_PREFIX = "/api/v1/internal/gpu-relay"
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# URL builders
|
||||
# ------------------------------------------------------------------
|
||||
def _relay_url_from_base(self, base_url: str, key: str, secret: str) -> str:
|
||||
return f"{base_url}{self.RELAY_PATH_PREFIX}/{key}?token={urllib.parse.quote(secret, safe='')}"
|
||||
|
||||
def _relay_put_url(self, key: str, secret: str) -> str:
|
||||
"""给 P4000 回传结果用的 URL(外部可达)。"""
|
||||
return self._relay_url_from_base(self.relay_base_url, key, secret)
|
||||
|
||||
def _relay_internal_url(self, key: str, secret: str) -> str:
|
||||
"""Worker 自己 GET/DELETE 用的 URL(Docker 内网)。"""
|
||||
return self._relay_url_from_base(self.relay_internal_base_url, key, secret)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Health
|
||||
# ------------------------------------------------------------------
|
||||
@@ -128,12 +152,12 @@ class GpuEncoderClient:
|
||||
input_url, oss_key = self._upload_mezzanine(mezzanine_path)
|
||||
logger.debug("[gpu-encoder] mezzanine uploaded: oss_key=%s", oss_key)
|
||||
|
||||
# 2. prepare relay put/get URLs
|
||||
# 2. prepare relay URLs (PUT 走外部 URL 给 P4000;GET/DELETE 走内部 Docker 网络)
|
||||
relay_key = uuid.uuid4().hex
|
||||
secret = self._get_relay_secret()
|
||||
put_url = self._relay_url(relay_key, secret)
|
||||
get_url = put_url
|
||||
del_url = put_url # same URL, DELETE method
|
||||
put_url = self._relay_put_url(relay_key, secret)
|
||||
get_url = self._relay_internal_url(relay_key, secret)
|
||||
del_url = get_url # 内部 URL,DELETE method
|
||||
|
||||
# 3. build ffmpeg args
|
||||
ffmpeg_args = ["-y", "-i", "in.mp4"]
|
||||
@@ -212,9 +236,6 @@ class GpuEncoderClient:
|
||||
raise GpuEncodeError("GPU_ENCODE_RELAY_SECRET not set")
|
||||
return secret
|
||||
|
||||
def _relay_url(self, key: str, secret: str) -> str:
|
||||
return f"{self.relay_base_url}/api/v1/internal/gpu-relay/{key}?token={urllib.parse.quote(secret, safe='')}"
|
||||
|
||||
def _post_sync(self, body: dict[str, Any], *, mezzanine_path: Path) -> dict[str, Any]:
|
||||
url = f"{self.endpoint}/api/render/sync"
|
||||
req_timeout = body.get("timeout", self.sync_timeout) + 60
|
||||
@@ -337,11 +358,13 @@ def _build_client_from_settings() -> Optional[GpuEncoderClient]:
|
||||
return None
|
||||
endpoint = (getattr(settings, "gpu_encode_endpoint", "") or "").strip()
|
||||
relay = (getattr(settings, "gpu_encode_relay_base_url", "") or "").strip()
|
||||
relay_internal = (getattr(settings, "gpu_encode_relay_internal_base_url", "") or "").strip()
|
||||
if not endpoint or not relay:
|
||||
return None
|
||||
return GpuEncoderClient(
|
||||
endpoint=endpoint,
|
||||
relay_base_url=relay,
|
||||
relay_internal_base_url=relay_internal,
|
||||
sync_timeout=getattr(settings, "gpu_encode_sync_timeout", 300),
|
||||
health_timeout=getattr(settings, "gpu_encode_health_timeout", 3.0),
|
||||
vcodec=getattr(settings, "gpu_encode_vcodec", "h264_nvenc"),
|
||||
|
||||
Reference in New Issue
Block a user