df99305dd6
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 29s
CI/CD Pipeline / PR Build Worker Image (pull_request) Successful in 29s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 49s
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Successful in 1m39s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m44s
CI/CD Pipeline / Validate - Style (pull_request) Successful in 2m19s
AI Code Review / AI Code Review (pull_request) Failing after 2m52s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m54s
CI/CD Pipeline / Validate - Security (pull_request) Successful in 4m11s
CI/CD Pipeline / Unit Tests (pull_request) Failing after 6m23s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Failing after 1s
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 3m39s
问题:素材转码与视频生成共用 celery 默认队列、worker 单进程消费, 20+ 转码积压会把用户生成任务堵 40 分钟以上;孤儿清理把任务标 failed 后 Redis 队列消息未作废,消息被重投导致 failed→running 非法转换, worker 打印 ERROR 后继续产出半成品。 队列隔离: - 新增 packages/shared/celery_queues.py:generation/transcode/celery 三队列与 task_routes(generate_video→generation;ingest_asset/ classify_asset/duplication→transcode),apply_queue_settings() - worker 入口改双进程:generation worker 独占队列并内嵌 beat (prefetch=1, GENERATION_CONCURRENCY 默认 2),transcode worker 消费 transcode,celery(并发=总-2,最小 1),任一退出则整体终止 - compose/部署脚本/ps1 同步新增 GENERATION_CONCURRENCY 与健康检查 消息作废: - 新增 packages/shared/celery_orphan_guard.py:终态守卫 ensure_task_claimable、Redis 队列消息物理清理(JSON 信封解析, 按业务 id + celery headers.id 双匹配,未命中 rpush 保序)、 revoke_and_purge(control.revoke + 物理清队列双保险) - 入队点(生成/上传/分片/重试)send_task 后持久化 celery_task_id 到 generation_tasks/ingest_jobs(新列,067 迁移,失败仅 warning) - generate_video/ingest_asset 执行前校验 DB 状态:终态直接 discarded 不进业务逻辑;mark_processing 返回 False(非法转换)安全中止 - 孤儿/超时清理标 failed 时同时 revoke + 清队列消息 - pending 超时阈值 15→45 分钟,与 running 孤儿(20min)区分 测试:新增 22 个单测(路由表/真实 Redis 消息清理/终态守卫/ 非法转换中止/标 failed 后消息不重投/入队持久化),全量 14301 passed;067 迁移隔离 DDL 验证 upgrade/downgrade 通过。
232 lines
7.3 KiB
YAML
Executable File
232 lines
7.3 KiB
YAML
Executable File
# ===========================================
|
|
# 小虾剪辑 SaaS Docker Compose 配置
|
|
# ===========================================
|
|
#
|
|
# 用法:
|
|
# Staging: docker compose --env-file ../../.env -f compose.yml up -d
|
|
# Production: 设置相关环境变量后执行
|
|
#
|
|
# 环境变量说明:
|
|
# API_IMAGE - API 镜像名称 (默认: xiaoxia-saas-api:dev)
|
|
# WORKER_IMAGE - Worker 镜像名称 (默认: xiaoxia-saas-worker:dev)
|
|
# WEB_IMAGE - Web 镜像名称 (默认: xiaoxia-saas-web:dev)
|
|
# WEB_DOCKERFILE - Web Dockerfile 路径
|
|
# WEB_NGINX_CONF - Nginx 配置文件路径
|
|
# API_PORT - API 端口映射 (staging: 8000, production: 8001)
|
|
# WEB_PORT - Web 端口映射 (staging: 3001, production: 3002)
|
|
# GENERATED_FILES_HOST_DIR - 生成文件的主机目录
|
|
# WORKER_CONCURRENCY - Worker 并发数 (默认: 4)
|
|
# WORKER_MAX_TASKS_PER_CHILD - Worker 每个子进程最大任务数 (默认: 100)
|
|
#
|
|
# 重要:
|
|
# - 生产环境不要挂载 web-dist volume,否则会导致 403
|
|
# - 确保环境隔离网络已创建: docker network create xiaoxia-net-${ENV}
|
|
# - ENV=staging → xiaoxia-net-staging
|
|
# - ENV=production → xiaoxia-net-production
|
|
#
|
|
|
|
# ===========================================
|
|
# 日志轮转配置(所有服务共享)
|
|
# ===========================================
|
|
x-logging: &default-logging
|
|
driver: json-file
|
|
options:
|
|
max-size: "50m"
|
|
max-file: "3"
|
|
|
|
services:
|
|
# =========================================
|
|
# API 服务
|
|
# =========================================
|
|
api:
|
|
image: ${API_IMAGE:-xiaoxia-saas-api:dev}
|
|
# 不在生产环境构建镜像,使用预构建的镜像
|
|
# build:
|
|
# context: ../..
|
|
# dockerfile: infra/docker/api.Dockerfile
|
|
|
|
container_name: xiaoxia-api-${ENV:-staging}
|
|
restart: unless-stopped
|
|
stop_grace_period: 30s
|
|
stop_signal: SIGTERM
|
|
|
|
# 环境变量文件(包含数据库密码等敏感信息)
|
|
env_file:
|
|
- ../../.env
|
|
|
|
environment:
|
|
APP_ENV: ${APP_ENV:-staging}
|
|
APP_VERSION: ${APP_VERSION:-unknown}
|
|
GENERATED_FILES_DIR: /app/generated
|
|
GENERATED_FILES_URL_PREFIX: /generated-files
|
|
PUBLIC_API_BASE_URL: ${PUBLIC_API_BASE_URL:-https://api.xiaoxiajianji.com}
|
|
|
|
# 端口映射
|
|
# Staging: 8000 -> 8000
|
|
# Production: 8001 -> 8000
|
|
ports:
|
|
- "127.0.0.1:${API_PORT:-8000}:8000"
|
|
|
|
# 共享生成文件目录
|
|
volumes:
|
|
- generated-files:/app/generated
|
|
|
|
networks:
|
|
- xiaoxia-net
|
|
|
|
# 健康检查配置
|
|
healthcheck:
|
|
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://localhost:8000/health', timeout=5)"]
|
|
interval: 30s
|
|
timeout: 10s
|
|
retries: 3
|
|
start_period: 40s
|
|
|
|
logging: *default-logging
|
|
|
|
# =========================================
|
|
# 资源限制建议(生产环境建议启用)
|
|
# =========================================
|
|
deploy:
|
|
resources:
|
|
limits:
|
|
cpus: '2.0'
|
|
memory: 2g
|
|
reservations:
|
|
cpus: '0.5'
|
|
memory: 512M
|
|
|
|
# =========================================
|
|
# Worker 服务(Celery 任务队列)
|
|
# =========================================
|
|
worker:
|
|
image: ${WORKER_IMAGE:-xiaoxia-saas-worker:dev}
|
|
|
|
container_name: xiaoxia-worker-${ENV:-staging}
|
|
restart: unless-stopped
|
|
stop_grace_period: 300s
|
|
stop_signal: SIGTERM
|
|
|
|
env_file:
|
|
- ../../.env
|
|
|
|
environment:
|
|
APP_ENV: ${APP_ENV:-staging}
|
|
APP_VERSION: ${APP_VERSION:-unknown}
|
|
WORKER_CONCURRENCY: ${WORKER_CONCURRENCY:-4}
|
|
WORKER_MAX_TASKS_PER_CHILD: ${WORKER_MAX_TASKS_PER_CHILD:-100}
|
|
# #1714 队列隔离:generation 队列独占 worker(默认并发 2),其余并发给转码
|
|
GENERATION_CONCURRENCY: ${GENERATION_CONCURRENCY:-2}
|
|
GENERATED_FILES_DIR: /app/generated
|
|
GENERATED_FILES_URL_PREFIX: /generated-files
|
|
PUBLIC_API_BASE_URL: ${PUBLIC_API_BASE_URL:-https://api.xiaoxiajianji.com}
|
|
|
|
volumes:
|
|
- generated-files:/app/generated
|
|
|
|
networks:
|
|
- xiaoxia-net
|
|
|
|
# 健康检查配置
|
|
# 注:celery inspect ping 依赖 broker 连接,在容器内不可靠,改用进程检查
|
|
healthcheck:
|
|
test: ["CMD-SHELL", "pgrep -f 'celery.*worker' | head -n1 >/dev/null 2>&1 || grep -q celery /proc/1/cmdline || exit 1"]
|
|
interval: 30s
|
|
timeout: 10s
|
|
retries: 3
|
|
start_period: 40s
|
|
|
|
logging: *default-logging
|
|
|
|
# =========================================
|
|
# 资源限制建议(生产环境建议启用)
|
|
# =========================================
|
|
# 注意: Worker 需要处理视频,建议分配更多资源
|
|
# #1714 队列隔离后容器内运行 generation + transcode 两个 worker 进程,
|
|
# 总并发 = WORKER_CONCURRENCY(默认 4),4C8G 以上确保视频渲染不 OOM
|
|
deploy:
|
|
resources:
|
|
limits:
|
|
cpus: '4.0'
|
|
memory: 8g
|
|
reservations:
|
|
cpus: '1.0'
|
|
memory: 2G
|
|
|
|
# =========================================
|
|
# Web 服务(Nginx + 前端静态文件)
|
|
# =========================================
|
|
web:
|
|
image: ${WEB_IMAGE:-xiaoxia-saas-web:dev}
|
|
# 不在生产环境构建镜像,使用 web-artifact.Dockerfile
|
|
# build:
|
|
# context: ../..
|
|
# dockerfile: ${WEB_DOCKERFILE:-infra/docker/web.Dockerfile}
|
|
# args:
|
|
# NGINX_CONF: ${WEB_NGINX_CONF:-infra/docker/nginx.conf}
|
|
|
|
container_name: xiaoxia-web-${ENV:-staging}
|
|
restart: unless-stopped
|
|
|
|
# 端口映射
|
|
# Staging: 3001 -> 80
|
|
# Production: 3002 -> 80 (通过 Nginx 反向代理)
|
|
ports:
|
|
- "127.0.0.1:${WEB_PORT:-3001}:80"
|
|
|
|
networks:
|
|
- xiaoxia-net
|
|
|
|
# =========================================
|
|
# Nginx 配置运行时覆盖
|
|
# 确保容器使用正确环境的 nginx 配置,即使镜像构建时使用了默认配置
|
|
# 注意: 只覆盖 /etc/nginx/conf.d/default.conf,不挂载 /usr/share/nginx/html
|
|
# =========================================
|
|
environment:
|
|
- NGINX_ENV=${ENV:-staging}
|
|
volumes:
|
|
- ./nginx-${ENV:-staging}.conf:/etc/nginx/conf.d/default.conf:ro
|
|
|
|
healthcheck:
|
|
test: ["CMD", "wget", "--spider", "-q", "http://127.0.0.1:80"]
|
|
interval: 30s
|
|
timeout: 5s
|
|
retries: 3
|
|
|
|
logging: *default-logging
|
|
|
|
# =========================================
|
|
# 资源限制建议
|
|
# =========================================
|
|
deploy:
|
|
resources:
|
|
limits:
|
|
cpus: '0.5'
|
|
memory: 512M
|
|
|
|
# ===========================================
|
|
# 共享卷配置
|
|
# ===========================================
|
|
volumes:
|
|
generated-files:
|
|
driver: local
|
|
driver_opts:
|
|
type: none
|
|
o: bind
|
|
# 重要: 确保主机目录存在且有正确权限
|
|
# Staging: /var/lib/xiaoxia-saas-staging/generated
|
|
# Production: /var/lib/xiaoxia-saas-production/generated
|
|
device: ${GENERATED_FILES_HOST_DIR:?GENERATED_FILES_HOST_DIR must be set in .env}
|
|
|
|
# ===========================================
|
|
# 网络配置
|
|
# ===========================================
|
|
networks:
|
|
xiaoxia-net:
|
|
external: true
|
|
# 网络名根据 ENV 变量区分,实现 staging/production 环境隔离
|
|
# staging: xiaoxia-net-staging
|
|
# production: xiaoxia-net-production
|
|
name: xiaoxia-net-${ENV:-staging}
|
|
|