Files
xiaoxia-saas/packages/config/api_settings.py
T
saas-backend-agent 1a917a3472
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 1s
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 47s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m56s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 2m21s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 2m47s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m11s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 3m35s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 6m23s
AI Code Review / AI Code Review (pull_request) Successful in 6m36s
CI/CD Pipeline / Unit Tests (pull_request) Failing after 8m57s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 12m37s
CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / CI Gate (pull_request) Has been cancelled
CI/CD Pipeline / Validate - Style (pull_request) Has been cancelled
feat(p1): OSS 双 endpoint 分离 — 上传/下载走 VPC 内网,签名 URL 走公网
问题:
- 116 是阿里云杭州 ECS,之前 oss_endpoint=oss-cn-hangzhou.aliyuncs.com(公网),
  worker 上传/下载 OSS 走公网卡到 0.3~0.7 MB/s,mezzanine 上传 104s + 成片上传 55s,
  吃掉 160s+ 渲染耗时。
- oss_helpers 自己维护一套 bucket 实例,和 SharedStorageService 重复实现,且
  normalize_storage_key 没做 URL decode(unquote),带编码字符的 key 签名出来
  返回 403,触发 _verify_url_accessible 2 次重试延迟。

改造:
1. config/base.py 新增 oss_internal_endpoint 配置项 + effective_oss_internal_endpoint
   属性:阿里云公网域名(oss-cn-<region>.aliyuncs.com)自动推导 -internal 内网
   域名,显式配置可覆盖;非阿里云/MinIO 回退公网。
2. packages/shared/storage.py 维护两个 bucket:
   - self.bucket:internal endpoint(VPC 千兆带宽),SDK 上传/下载/删除/exists
   - self.public_bucket:public endpoint,仅用于 sign_url 生成公网可访问 URL
   public_url 永远拼公网域名,对外不可感知。
3. apps/worker/video_processing/oss_helpers.py 收敛为 SharedStorageService 的
   薄 wrapper(-226 行),保留所有旧函数签名兼容现有调用:
   - oss_bucket() 返回 internal bucket
   - upload_to_oss/download_asset/delete_from_oss 转发到 storage 走 internal
   - get_signed_download_url 转发到 storage.get_download_url 用 public_bucket 签名
   - resolve_asset_path 转发到 storage 统一逻辑
4. normalize_storage_key 统一 unquote,修复签名 403。
5. config/api_settings.py 增加 OSS_INTERNAL_ENDPOINT 属性方便诊断。

预期效果:
- OSS 上传/下载从 0.3~0.7 MB/s → VPC 千兆(~100 MB/s),两段上传 160s 压到 <3s
- 签名 URL 不再 403,_verify_url_accessible 首试即过,不再重试
- 整体渲染耗时目标 <30s(GPU 编码链路 ~10s + OSS 传输 <5s + 合成/封面 <15s)
- 前端/P4000/MediaKit 外网访问 OSS 完全不变(public_url 仍是公网域名)
2026-09-28 09:58:33 +08:00

291 lines
9.5 KiB
Python
Executable File
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
"""API 服务配置 — 继承 SharedSettings,只追加 API 特有字段。
通用配置统一在 packages/config/base.py 的 SharedSettings 中定义,这里不重复。
历史上 API 端使用 UPPER_CASE 命名风格的字段,目前通过 property 别名向后兼容。
新代码统一使用 snake_case(继承自 SharedSettings)。
"""
import os
from typing import Optional
from pydantic import AliasChoices, Field, field_validator
from pydantic_settings import SettingsConfigDict
from packages.config.base import SharedSettings, get_cached_settings
class APISettings(SharedSettings):
"""API 服务专用配置。
通用配置继承自 SharedSettings,这里只定义 API 独有字段。
"""
# ── 应用基本信息 ────────────────────────────────────────────────────
app_name: str = "xiaoxia-saas"
app_version: str = "0.1.61"
# 应用基础 URL,用于生成认证邮件中的链接
app_base_url: str = "http://localhost:3000"
# Container bind address; external expose is controlled by Docker/Nginx.
api_host: str = "0.0.0.0" # nosec: B104
api_port: int = 8000
# ── 数据库特有 ──────────────────────────────────────────────────────
use_in_memory_db: bool = False
# ── Redis 特有 ──────────────────────────────────────────────────────
enable_redis_sessions: bool = False
# ── JWT ────────────────────────────────────────────────────────────
# JWT secret key - MUST be set via environment variable, no default allowed
jwt_secret_key: Optional[str] = None
# JWT 算法与过期时间
jwt_algorithm: str = "HS256"
jwt_access_token_expire_minutes: int = 30
jwt_refresh_token_expire_days: int = 30
@field_validator("jwt_secret_key", mode="before")
@classmethod
def validate_jwt_secret_key(cls, v):
if v is None or v == "":
raise ValueError(
"JWT_SECRET_KEY must be set via environment variable. Do not use default value in production!"
)
# Block known insecure default values
insecure_defaults = [
"your-secret-key-change-in-production",
"your-secret-key",
"secret",
"changeme",
"password",
]
if v.lower() in [d.lower() for d in insecure_defaults]:
raise ValueError(
f"JWT_SECRET_KEY '{v}' is insecure. Please set a strong random secret via environment variable."
)
return v
# ── 邮件 ────────────────────────────────────────────────────────────
enable_email_delivery: bool = False
smtp_host: str = "smtp.gmail.com"
smtp_port: int = 587
smtp_user: str = ""
smtp_password: str = ""
smtp_from_email: str = ""
smtp_from_name: str = "小虾 SaaS"
smtp_use_tls: bool = True
# ── OSS 特有校验 ────────────────────────────────────────────────────
@field_validator("oss_access_key_id", mode="before")
@classmethod
def validate_oss_access_key_id(cls, v):
if (v is None or v == "") and os.getenv("APP_ENV", "development") != "development":
raise ValueError(
"OSS_ACCESS_KEY_ID must be set via environment variable in non-development environments. "
"Check the server .env file (e.g. /var/lib/xiaoxia-saas-staging/.env)."
)
return v or ""
@field_validator("oss_access_key_secret", mode="before")
@classmethod
def validate_oss_access_key_secret(cls, v):
if (v is None or v == "") and os.getenv("APP_ENV", "development") != "development":
raise ValueError(
"OSS_ACCESS_KEY_SECRET must be set via environment variable in non-development environments. "
"Check the server .env file (e.g. /var/lib/xiaoxia-saas-staging/.env)."
)
return v or ""
oss_direct_upload_max_mb: int = Field(
default=2000,
validation_alias=AliasChoices("oss_direct_upload_max_mb", "max_upload_size_mb"),
)
# ── CORS ────────────────────────────────────────────────────────────
cors_origins_raw: str = "http://localhost:3000,http://localhost:5173,http://localhost:8000"
# ── 渲染引擎 ────────────────────────────────────────────────────────
# 渲染引擎选择:legacy=旧VideoComposeService,unified=新UnifiedRenderService
render_engine: str = "legacy"
model_config = SettingsConfigDict(
env_file=".env",
env_file_encoding="utf-8",
case_sensitive=False,
extra="ignore",
)
@property
def cors_origins(self) -> list[str]:
return [origin.strip() for origin in self.cors_origins_raw.split(",") if origin.strip()]
# ── 向后兼容:UPPER_CASE property 别名 ──────────────────────────────
# 新代码请使用 snake_case(继承的字段名),以下别名仅用于兼容旧代码
@property
def APP_NAME(self) -> str:
return self.app_name
@property
def APP_VERSION(self) -> str:
return self.app_version
@property
def ENVIRONMENT(self) -> str:
return self.environment
@property
def DEBUG(self) -> bool:
return self.debug
@property
def APP_BASE_URL(self) -> str:
return self.app_base_url
@property
def API_HOST(self) -> str:
return self.api_host
@property
def API_PORT(self) -> int:
return self.api_port
@property
def DATABASE_URL(self) -> str:
return self.database_url
@property
def DATABASE_POOL_SIZE(self) -> int:
return self.database_pool_size
@property
def DATABASE_MAX_OVERFLOW(self) -> int:
return self.database_max_overflow
@property
def DATABASE_POOL_TIMEOUT(self) -> int:
return self.database_pool_timeout
@property
def DATABASE_POOL_RECYCLE(self) -> int:
return self.database_pool_recycle
@property
def USE_IN_MEMORY_DB(self) -> bool:
return self.use_in_memory_db
@property
def AUTO_CREATE_SCHEMA(self) -> bool:
return self.auto_create_schema
@property
def REDIS_URL(self) -> str:
return self.redis_url
@property
def ENABLE_REDIS_SESSIONS(self) -> bool:
return self.enable_redis_sessions
@property
def JWT_SECRET_KEY(self) -> Optional[str]:
return self.jwt_secret_key
@property
def JWT_ALGORITHM(self) -> str:
return self.jwt_algorithm
@property
def JWT_ACCESS_TOKEN_EXPIRE_MINUTES(self) -> int:
return self.jwt_access_token_expire_minutes
@property
def JWT_REFRESH_TOKEN_EXPIRE_DAYS(self) -> int:
return self.jwt_refresh_token_expire_days
@property
def ENABLE_EMAIL_DELIVERY(self) -> bool:
return self.enable_email_delivery
@property
def SMTP_HOST(self) -> str:
return self.smtp_host
@property
def SMTP_PORT(self) -> int:
return self.smtp_port
@property
def SMTP_USER(self) -> str:
return self.smtp_user
@property
def SMTP_PASSWORD(self) -> str:
return self.smtp_password
@property
def SMTP_FROM_EMAIL(self) -> str:
return self.smtp_from_email
@property
def SMTP_FROM_NAME(self) -> str:
return self.smtp_from_name
@property
def SMTP_USE_TLS(self) -> bool:
return self.smtp_use_tls
@property
def CELERY_BROKER_URL(self) -> str:
return self.celery_broker_url
@property
def CELERY_RESULT_BACKEND(self) -> str:
return self.celery_result_backend
@property
def OSS_ENDPOINT(self) -> str:
return self.oss_endpoint
@property
def OSS_INTERNAL_ENDPOINT(self) -> str:
return self.effective_oss_internal_endpoint
@property
def OSS_ACCESS_KEY_ID(self) -> str:
return self.oss_access_key_id
@property
def OSS_ACCESS_KEY_SECRET(self) -> str:
return self.oss_access_key_secret
@property
def OSS_BUCKET_NAME(self) -> str:
return self.oss_bucket_name
@property
def OSS_DIRECT_UPLOAD_MAX_MB(self) -> int:
return self.oss_direct_upload_max_mb
@property
def OSS_DIRECT_UPLOAD_EXPIRE_SECONDS(self) -> int:
return self.oss_direct_upload_expire_seconds
@property
def CORS_ORIGINS_RAW(self) -> str:
return self.cors_origins_raw
@property
def CORS_ORIGINS(self) -> list[str]:
return self.cors_origins
@property
def RENDER_ENGINE(self) -> str:
return self.render_engine
def get_api_settings() -> APISettings:
"""获取 API 配置单例(统一入口)。"""
return get_cached_settings(APISettings)