Files
xiaoxia-saas/infra/docker/compose.yml
T
xiaoxia 01991f14d7
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 3s
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 3s
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 3s
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Check push changed paths (push) Successful in 13s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / Validate - Style (pull_request) Has been skipped
CI/CD Pipeline / Validate - Security (pull_request) Has been skipped
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Has been skipped
CI/CD Pipeline / Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 1m3s
CI/CD Pipeline / Build Staging API Image (push) Successful in 1m7s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 2m9s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 3m13s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 3m24s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 2m17s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (push) Successful in 3m33s
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Integration Tests (push) Successful in 4m17s
CI/CD Pipeline / Validate - Style (push) Successful in 4m51s
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Successful in 4m53s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 55s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 5m42s
AI Code Review / AI Code Review (pull_request) Successful in 6m57s
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 1m52s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 3m45s
CI/CD Pipeline / PR Build Web Image (pull_request) Failing after 9m5s
CI/CD Pipeline / CI Gate (pull_request) Failing after 2s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 4m58s
CI/CD Pipeline / Unit Tests (push) Successful in 11m33s
CI/CD Pipeline / Validate - Security (push) Successful in 11m53s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / CI Gate (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Canary Release to Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
fix(queue): #2073 Worker 队列分流——任务路由补全 + beat 独立 + transcode 并发独立 (#2079)
Co-authored-by: xiaoxia <dev@xiaoxiajianji.com>
Co-committed-by: xiaoxia <dev@xiaoxiajianji.com>
2026-09-28 01:09:29 +08:00

203 lines
6.8 KiB
YAML
Executable File
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# ===========================================
# 小虾剪辑 SaaS Docker Compose 配置
# ===========================================
#
# 用法:
# Staging: docker compose --env-file ../../.env -f compose.yml up -d
# Production: 设置相关环境变量后执行
#
# 环境变量说明:
# API_IMAGE - API 镜像名称 (默认: xiaoxia-saas-api:dev)
# WORKER_IMAGE - Worker 镜像名称 (默认: xiaoxia-saas-worker:dev)
# WEB_IMAGE - Web 镜像名称 (默认: xiaoxia-saas-web:dev)
# API_PORT - API 端口映射 (staging: 8000, production: 8001)
# WEB_PORT - Web 端口映射 (staging: 3001, production: 3002)
# GENERATED_FILES_HOST_DIR - 生成文件的主机目录
# GENERATION_CONCURRENCY - Generation worker 并发(用户实时任务,默认 2)
# TRANSCODE_CONCURRENCY - Transcode worker 并发(后台/转码/AI,默认 2)
# WORKER_MAX_TASKS_PER_CHILD - Worker 每个子进程最大任务数 (默认: 100)
# BEAT_ENABLED - 容器内启动 celery beat(默认 1;独立 beat 容器部署设为 0)
# WORKER_CONCURRENCY - 兼容旧变量:未显式设置上面两个并发时按此总数分配
#
# 重要:
# - 生产环境不要挂载 web-dist volume,否则会导致 403
# - 确保环境隔离网络已创建: docker network create xiaoxia-net-${ENV}
# - ENV=staging -> xiaoxia-net-staging
# - ENV=production -> xiaoxia-net-production
#
# #2073 队列分流:worker 容器内跑三个独立进程——beat(只发定时任务)、
# generation worker(只消费 generation 队列,实时高优)、transcode worker(消费
# transcode + celery 队列,后台任务)。beat 不再嵌入 generation worker,
# 不占实时任务槽位;TRANSCODE_CONCURRENCY 独立伸缩,不再依赖 WORKER_CONCURRENCY 差值。
# ===========================================
# 日志轮转配置(所有服务共享)
# ===========================================
x-logging: &default-logging
driver: json-file
options:
max-size: "50m"
max-file: "3"
services:
# =========================================
# API 服务
# =========================================
api:
image: ${API_IMAGE:-xiaoxia-saas-api:dev}
container_name: xiaoxia-api-${ENV:-staging}
restart: unless-stopped
stop_grace_period: 30s
stop_signal: SIGTERM
env_file:
- ../../.env
environment:
APP_ENV: ${APP_ENV:-staging}
GENERATED_FILES_DIR: /app/generated
GENERATED_FILES_URL_PREFIX: /generated-files
PUBLIC_API_BASE_URL: ${PUBLIC_API_BASE_URL:-https://api.xiaoxiajianji.com}
ports:
- "127.0.0.1:${API_PORT:-8000}:8000"
volumes:
- generated-files:/app/generated
- ../../deploy/configs:/app/configs:ro
networks:
- xiaoxia-net
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/health', timeout=5)"]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s
logging: *default-logging
deploy:
resources:
limits:
cpus: '2.0'
memory: 2g
reservations:
cpus: '0.5'
memory: 512M
# =========================================
# Worker 服务(#2073 队列分流:beat + generation + transcode 同容器三进程)
# =========================================
# 三个进程独立启动,任一退出则容器整体退出由 docker restart 拉起;
# 各自的并发与资源占用通过环境变量控制:
# - generation:GENERATION_CONCURRENCY(默认 2),消费 generation 队列
# - transcode: TRANSCODE_CONCURRENCY(默认 2),消费 transcode,celery 队列
# - beat: 不消费任务,只发定时任务到 celery 默认队列
worker:
image: ${WORKER_IMAGE:-xiaoxia-saas-worker:dev}
container_name: xiaoxia-worker-${ENV:-staging}
restart: unless-stopped
# 长任务(ingest HEVC 转码最长 30min、生成硬超时 11min)给足优雅关闭窗口
stop_grace_period: 300s
stop_signal: SIGTERM
env_file:
- ../../.env
environment:
APP_ENV: ${APP_ENV:-staging}
# 兼容旧变量:若两个 *_CONCURRENCY 均未显式设置,entrypoint 会按此总数分配
WORKER_CONCURRENCY: ${WORKER_CONCURRENCY:-4}
WORKER_MAX_TASKS_PER_CHILD: ${WORKER_MAX_TASKS_PER_CHILD:-100}
# #2073 队列独立伸缩:generation 默认 2,transcode 默认 2(不再差值计算)
GENERATION_CONCURRENCY: ${GENERATION_CONCURRENCY:-2}
TRANSCODE_CONCURRENCY: ${TRANSCODE_CONCURRENCY:-2}
# beat 默认在本容器启动;独立 beat 容器部署时设为 0
BEAT_ENABLED: ${BEAT_ENABLED:-1}
GENERATED_FILES_DIR: /app/generated
GENERATED_FILES_URL_PREFIX: /generated-files
PUBLIC_API_BASE_URL: ${PUBLIC_API_BASE_URL:-https://api.xiaoxiajianji.com}
volumes:
- generated-files:/app/generated
networks:
- xiaoxia-net
# 健康检查:至少有一个 celery worker 进程在跑(beat 本身不作为存活依据)
healthcheck:
test: ["CMD-SHELL", "grep -q 'celery.*worker' /proc/[0-9]*/cmdline 2>/dev/null || exit 1"]
interval: 30s
timeout: 10s
retries: 3
start_period: 40s
logging: *default-logging
# 资源限制:容器总资源 = gen + trans + beat,按 2+2 并发场景建议 4C8G;
# 后续如需独立扩容/重启,可拆为 worker-generation / worker-transcode / worker-beat 三个 service。
deploy:
resources:
limits:
cpus: '4.0'
memory: 8g
reservations:
cpus: '1.0'
memory: 2G
# =========================================
# Web 服务(Nginx + 前端静态文件)
# =========================================
web:
image: ${WEB_IMAGE:-xiaoxia-saas-web:dev}
container_name: xiaoxia-web-${ENV:-staging}
restart: unless-stopped
ports:
- "127.0.0.1:${WEB_PORT:-3001}:80"
networks:
- xiaoxia-net
environment:
- APP_ENV=${ENV:-staging}
volumes:
- ./nginx-${ENV:-staging}.conf:/etc/nginx/conf.d/default.conf:ro
healthcheck:
test: ["CMD", "wget", "--spider", "-q", "http://127.0.0.1:80"]
interval: 30s
timeout: 5s
retries: 3
logging: *default-logging
deploy:
resources:
limits:
cpus: '0.5'
memory: 512M
# ===========================================
# 共享卷配置
# ===========================================
volumes:
generated-files:
driver: local
driver_opts:
type: none
o: bind
device: ${GENERATED_FILES_HOST_DIR:?GENERATED_FILES_HOST_DIR must be set in .env}
# ===========================================
# 网络配置
# ===========================================
networks:
xiaoxia-net:
external: true
name: xiaoxia-net-${ENV:-staging}