ci: 启用BuildKit分布式缓存+Gitea Registry,优化构建速度
Deploy / Deploy Staging (push) Failing after 138h40m13s
CI/CD Pipeline / Frontend Lint (push) Failing after 138h40m24s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 138h40m24s
Deploy / Production Browser E2E (push) Failing after 1797h15m29s
Deploy / Staging E2E Tests (push) Failing after 1797h15m31s
Deploy / Build Production Runtime Images (push) Failing after 1797h15m38s
Deploy / Deploy Production (push) Failing after 1797h46m56s
Deploy / Deploy Staging (push) Failing after 138h40m13s
CI/CD Pipeline / Frontend Lint (push) Failing after 138h40m24s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 138h40m24s
Deploy / Production Browser E2E (push) Failing after 1797h15m29s
Deploy / Staging E2E Tests (push) Failing after 1797h15m31s
Deploy / Build Production Runtime Images (push) Failing after 1797h15m38s
Deploy / Deploy Production (push) Failing after 1797h46m56s
- 拆分 requirements.txt: 稳定大包放 requirements-base.txt(numpy/scipy/opencv/Pillow/sqlalchemy 等),业务依赖留 requirements.txt - 优化 Dockerfile: 分两层安装依赖,基础层缓存命中率高 - CI 改用 docker buildx build,缓存存 Gitea Container Registry - 启用 Gitea Packages/Container Registry(git.xiaoxiajianji.com) - 镜像仍推内网 Registry(172.30.18.198:5000),缓存独立存 Gitea Registry
This commit is contained in:
@@ -64,19 +64,37 @@ jobs:
|
|||||||
|
|
||||||
- name: Build and push staging API/Worker images
|
- name: Build and push staging API/Worker images
|
||||||
shell: sh
|
shell: sh
|
||||||
|
env:
|
||||||
|
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
|
||||||
run: |
|
run: |
|
||||||
set -eu
|
set -eu
|
||||||
REGISTRY="172.30.18.198:5000"
|
REGISTRY="172.30.18.198:5000"
|
||||||
docker build --pull=false \
|
CACHE_REGISTRY="git.xiaoxiajianji.com/xiaoxia/xiaoxia-saas"
|
||||||
|
CACHE_BRANCH="${GITHUB_REF_NAME//\//-}"
|
||||||
|
|
||||||
|
# 登录 Gitea Registry(用于构建缓存)
|
||||||
|
echo "${GITHUB_TOKEN}" | docker login git.xiaoxiajianji.com -u xiaoxia --password-stdin
|
||||||
|
|
||||||
|
# 确保 buildx builder 可用
|
||||||
|
docker buildx use default || docker buildx create --use --name default-builder 2>/dev/null || true
|
||||||
|
|
||||||
|
# 构建 API(带分布式缓存,缓存存 Gitea Registry,镜像推内网 Registry)
|
||||||
|
docker buildx build \
|
||||||
|
--cache-from "type=registry,ref=${CACHE_REGISTRY}/api-cache:${CACHE_BRANCH}" \
|
||||||
|
--cache-to "type=registry,ref=${CACHE_REGISTRY}/api-cache:${CACHE_BRANCH},mode=max" \
|
||||||
-f infra/docker/api.Dockerfile \
|
-f infra/docker/api.Dockerfile \
|
||||||
-t "${REGISTRY}/xiaoxia-saas-api:dev" \
|
-t "${REGISTRY}/xiaoxia-saas-api:dev" \
|
||||||
|
--push \
|
||||||
.
|
.
|
||||||
docker build --pull=false \
|
|
||||||
|
# 构建 Worker(带分布式缓存)
|
||||||
|
docker buildx build \
|
||||||
|
--cache-from "type=registry,ref=${CACHE_REGISTRY}/worker-cache:${CACHE_BRANCH}" \
|
||||||
|
--cache-to "type=registry,ref=${CACHE_REGISTRY}/worker-cache:${CACHE_BRANCH},mode=max" \
|
||||||
-f infra/docker/worker.Dockerfile \
|
-f infra/docker/worker.Dockerfile \
|
||||||
-t "${REGISTRY}/xiaoxia-saas-worker:dev" \
|
-t "${REGISTRY}/xiaoxia-saas-worker:dev" \
|
||||||
|
--push \
|
||||||
.
|
.
|
||||||
docker push "${REGISTRY}/xiaoxia-saas-api:dev"
|
|
||||||
docker push "${REGISTRY}/xiaoxia-saas-worker:dev"
|
|
||||||
|
|
||||||
- name: Package staging release artifact
|
- name: Package staging release artifact
|
||||||
shell: sh
|
shell: sh
|
||||||
|
|||||||
Regular → Executable
+10
-7
@@ -1,6 +1,6 @@
|
|||||||
# ============================================================
|
# ============================================================
|
||||||
# API Dockerfile - 专门用于 FastAPI 应用
|
# API Dockerfile - 专门用于 FastAPI 应用
|
||||||
# 优化:仅包含 API 所需的依赖
|
# 优化:依赖分层缓存 + 多阶段构建基础层
|
||||||
# ============================================================
|
# ============================================================
|
||||||
|
|
||||||
# 基础镜像:Python 3.12
|
# 基础镜像:Python 3.12
|
||||||
@@ -18,14 +18,17 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
|
|||||||
# 设置工作目录
|
# 设置工作目录
|
||||||
WORKDIR /app
|
WORKDIR /app
|
||||||
|
|
||||||
# 复制 requirements.txt(排除 worker 专用依赖)
|
# ---- 依赖分层:基础依赖(变化少,缓存命中率高)----
|
||||||
# API 需要 psycopg2/sqlalchemy 用于数据库连接
|
COPY requirements-base.txt /tmp/requirements-base.txt
|
||||||
# 注意:opencv、scipy 等是 worker 专用依赖,不在 API 中安装
|
|
||||||
|
RUN python -m venv /opt/venv \
|
||||||
|
&& /opt/venv/bin/pip install --no-cache-dir -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com -r /tmp/requirements-base.txt \
|
||||||
|
&& rm /tmp/requirements-base.txt
|
||||||
|
|
||||||
|
# ---- 依赖分层:业务依赖(变化频繁)----
|
||||||
COPY requirements.txt /tmp/requirements.txt
|
COPY requirements.txt /tmp/requirements.txt
|
||||||
|
|
||||||
# 创建虚拟环境并安装依赖
|
RUN /opt/venv/bin/pip install --no-cache-dir -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com -r /tmp/requirements.txt \
|
||||||
RUN python -m venv /opt/venv \
|
|
||||||
&& /opt/venv/bin/pip install --no-cache-dir -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com -r /tmp/requirements.txt \
|
|
||||||
&& rm /tmp/requirements.txt
|
&& rm /tmp/requirements.txt
|
||||||
|
|
||||||
# 复制应用代码
|
# 复制应用代码
|
||||||
|
|||||||
Regular → Executable
+10
-5
@@ -1,6 +1,6 @@
|
|||||||
# ============================================================
|
# ============================================================
|
||||||
# Worker Dockerfile - 专门用于 Celery Worker
|
# Worker Dockerfile - 专门用于 Celery Worker
|
||||||
# 优化:仅包含 worker 任务所需的依赖,减小镜像体积
|
# 优化:依赖分层缓存,基础大包和业务依赖分开
|
||||||
# ============================================================
|
# ============================================================
|
||||||
|
|
||||||
# 基础镜像:Python 3.12 + ffmpeg
|
# 基础镜像:Python 3.12 + ffmpeg
|
||||||
@@ -21,12 +21,17 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
|
|||||||
# 设置工作目录
|
# 设置工作目录
|
||||||
WORKDIR /app
|
WORKDIR /app
|
||||||
|
|
||||||
# 安装 Python 依赖(优化顺序以利用 Docker 缓存)
|
# ---- 依赖分层:基础依赖(变化少,缓存命中率高)----
|
||||||
# 先安装无变化的依赖
|
COPY requirements-base.txt /tmp/requirements-base.txt
|
||||||
|
|
||||||
|
RUN pip install --no-cache-dir -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com -r /tmp/requirements-base.txt \
|
||||||
|
&& rm /tmp/requirements-base.txt
|
||||||
|
|
||||||
|
# ---- 依赖分层:业务依赖(变化频繁)----
|
||||||
COPY requirements.txt /tmp/requirements.txt
|
COPY requirements.txt /tmp/requirements.txt
|
||||||
|
|
||||||
# 安装 Python 包
|
RUN pip install --no-cache-dir -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com -r /tmp/requirements.txt \
|
||||||
RUN pip install --no-cache-dir -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com -r /tmp/requirements.txt
|
&& rm /tmp/requirements.txt
|
||||||
|
|
||||||
# 复制应用代码
|
# 复制应用代码
|
||||||
COPY apps/worker/ /app/apps/worker/
|
COPY apps/worker/ /app/apps/worker/
|
||||||
|
|||||||
Executable
+39
@@ -0,0 +1,39 @@
|
|||||||
|
# 小虾 SaaS - 基础依赖(稳定、变化少,用于 Docker 缓存分层)
|
||||||
|
# 修改此文件会触发完整重新构建,请谨慎修改
|
||||||
|
|
||||||
|
# 数据库(基础层)
|
||||||
|
psycopg2-binary==2.9.10
|
||||||
|
psycopg[binary]>=3.2.2
|
||||||
|
sqlalchemy==2.0.35
|
||||||
|
alembic==1.13.3
|
||||||
|
|
||||||
|
# Web 框架核心
|
||||||
|
fastapi==0.115.0
|
||||||
|
uvicorn[standard]==0.32.0
|
||||||
|
pydantic==2.9.0
|
||||||
|
|
||||||
|
# 认证核心
|
||||||
|
pyjwt==2.9.0
|
||||||
|
bcrypt==4.2.0
|
||||||
|
|
||||||
|
# Redis
|
||||||
|
redis==5.2.0
|
||||||
|
|
||||||
|
# 任务队列
|
||||||
|
celery==5.4.0
|
||||||
|
|
||||||
|
# 视频处理(大包,变化极少)
|
||||||
|
numpy>=1.24.0
|
||||||
|
scipy>=1.10.0
|
||||||
|
opencv-python-headless>=4.8.0
|
||||||
|
Pillow==10.4.0
|
||||||
|
ffmpeg-python==0.2.0
|
||||||
|
|
||||||
|
# 对象存储
|
||||||
|
oss2==2.18.4
|
||||||
|
|
||||||
|
# HTTP 客户端
|
||||||
|
httpx==0.27.2
|
||||||
|
|
||||||
|
# Prometheus monitoring
|
||||||
|
prometheus-client==0.21.1
|
||||||
Regular → Executable
+8
-39
@@ -1,49 +1,18 @@
|
|||||||
# 小虾 SaaS - Python 依赖
|
# 小虾 SaaS - 业务依赖(变化频繁,放在 requirements-base 之上)
|
||||||
|
# 基础依赖(大包)请放在 requirements-base.txt 中
|
||||||
|
|
||||||
# Web 框架
|
# 验证 & 设置
|
||||||
fastapi==0.115.0
|
|
||||||
uvicorn[standard]==0.32.0
|
|
||||||
pydantic==2.9.0
|
|
||||||
email-validator==2.2.0
|
|
||||||
pydantic-settings==2.6.0
|
pydantic-settings==2.6.0
|
||||||
|
email-validator==2.2.0
|
||||||
# 数据库
|
|
||||||
psycopg2-binary==2.9.10
|
|
||||||
psycopg[binary]>=3.2.2
|
|
||||||
sqlalchemy==2.0.35
|
|
||||||
alembic==1.13.3
|
|
||||||
|
|
||||||
# 认证
|
|
||||||
pyjwt==2.9.0
|
|
||||||
bcrypt==4.2.0
|
|
||||||
python-multipart==0.0.32
|
python-multipart==0.0.32
|
||||||
|
|
||||||
# Redis
|
# 邮件
|
||||||
redis==5.2.0
|
aiosmtplib==3.0.2
|
||||||
|
|
||||||
# HTTP 客户端
|
|
||||||
httpx==0.27.2
|
|
||||||
|
|
||||||
# 测试
|
# 测试
|
||||||
pytest==8.3.3
|
pytest==8.3.3
|
||||||
pytest-asyncio==0.24.0
|
pytest-asyncio==0.24.0
|
||||||
pytest-cov==6.0.0
|
pytest-cov==6.0.0
|
||||||
|
|
||||||
# 视频处理
|
# 工具
|
||||||
ffmpeg-python==0.2.0
|
python-dotenv==1.0.1
|
||||||
Pillow==10.4.0
|
|
||||||
|
|
||||||
# 对象存储(阿里云 OSS)
|
|
||||||
oss2==2.18.4
|
|
||||||
|
|
||||||
# 任务队列
|
|
||||||
celery==5.4.0
|
|
||||||
|
|
||||||
# 数据分析(用于视频质量评估)
|
|
||||||
numpy>=1.24.0
|
|
||||||
scipy>=1.10.0
|
|
||||||
|
|
||||||
opencv-python-headless>=4.8.0
|
|
||||||
|
|
||||||
# Prometheus monitoring
|
|
||||||
prometheus-client==0.21.1
|
|
||||||
|
|||||||
Reference in New Issue
Block a user