fix(worker): ffmpeg改用apt安装,彻底解决下载慢+缓存+语法+清理问题
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 20s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 42s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
AI Code Review / AI Code Review (pull_request) Successful in 1m39s
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Successful in 2m6s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 2m36s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m38s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 2m18s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m3s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 3m41s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 21s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 20s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 42s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
AI Code Review / AI Code Review (pull_request) Successful in 1m39s
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Successful in 2m6s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 2m36s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 2m38s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 2m18s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m3s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 3m41s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 21s
问题: 1. johnvansickle.com下载ffmpeg静态包需要36分钟 2. 缓存命中坏包导致反复失败 3. bash数组语法在dash下报错(#688) 4. rm -rf ffmpeg-* 误删cache挂载点导致Device or resource busy 方案: - 直接用apt install ffmpeg,从阿里云镜像下载,几秒完成 - 放在runtime阶段安装,不依赖builder阶段的下载 - 彻底消除下载、缓存、语法、清理等所有问题 - builder阶段删掉wget/xz-utils等不再需要的依赖 - 删掉builder→runtime的ffmpeg二进制拷贝 镜像体积影响:ffmpeg apt包约30MB(含依赖), 和静态编译版(~80MB)相比可能略大,但稳定性和构建速度收益远大于体积代价
This commit is contained in:
@@ -2,10 +2,9 @@
|
||||
# Worker Dockerfile - 优化版(多阶段构建 + 镜像瘦身 + cache mount加速)
|
||||
# 优化项:
|
||||
# 1. 多阶段构建:builder 阶段安装编译依赖,runtime 阶段只保留运行时
|
||||
# 2. ffmpeg 静态编译替换:从 apt 安装(457MB) 改为静态二进制(~80MB)
|
||||
# 2. ffmpeg 通过 apt 安装(阿里云镜像加速,几秒完成,稳定可靠)
|
||||
# 3. Python 依赖瘦身:strip .so 调试符号 + 清理测试文件 + 清理缓存
|
||||
# 4. pip cache mount:加速依赖下载(跨构建共享pip wheel缓存)
|
||||
# 5. ffmpeg cache mount:避免每次重新下载静态编译包
|
||||
# ============================================================
|
||||
|
||||
# ==================== Builder 阶段 ====================
|
||||
@@ -21,48 +20,8 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
g++ \
|
||||
python3-dev \
|
||||
binutils \
|
||||
wget \
|
||||
xz-utils \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# ---- 下载静态编译 ffmpeg(带缓存+完整性校验+多源重试)----
|
||||
# 注意:必须用纯sh语法(dash兼容),不能用bash数组、[[ ]]等特性
|
||||
# 修复策略:1.三源下载 2.tar完整性校验 3.损坏删缓存重试 4.二进制可执行校验
|
||||
RUN --mount=type=cache,target=/tmp/ffmpeg-cache,sharing=locked \
|
||||
cd /tmp \
|
||||
&& FFMPEG_FILE="/tmp/ffmpeg-cache/ffmpeg-release-amd64-static.tar.xz" \
|
||||
&& download_ok=0 \
|
||||
&& for src in \
|
||||
"https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz" \
|
||||
"https://mirrors.tuna.tsinghua.edu.cn/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz" \
|
||||
"https://mirrors.ustc.edu.cn/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz"; \
|
||||
do \
|
||||
if [ $download_ok -eq 0 ] && [ -f "$FFMPEG_FILE" ] && tar -tf "$FFMPEG_FILE" >/dev/null 2>&1; then \
|
||||
echo "[ffmpeg] cache valid, skip download"; \
|
||||
download_ok=1; \
|
||||
break; \
|
||||
fi; \
|
||||
rm -f "$FFMPEG_FILE"; \
|
||||
echo "[ffmpeg] downloading from $src"; \
|
||||
if wget -q --tries=3 --timeout=30 -O "$FFMPEG_FILE" "$src"; then \
|
||||
if tar -tf "$FFMPEG_FILE" >/dev/null 2>&1; then \
|
||||
echo "[ffmpeg] download OK, tar valid"; \
|
||||
download_ok=1; \
|
||||
break; \
|
||||
else \
|
||||
echo "[ffmpeg] corrupted file, removing cache"; \
|
||||
rm -f "$FFMPEG_FILE"; \
|
||||
fi; \
|
||||
fi; \
|
||||
done \
|
||||
&& if [ $download_ok -eq 0 ]; then echo "[ffmpeg] ALL SOURCES FAILED"; exit 1; fi \
|
||||
&& tar xf "$FFMPEG_FILE" \
|
||||
&& cp ffmpeg-*-amd64-static/ffmpeg /usr/local/bin/ffmpeg \
|
||||
&& cp ffmpeg-*-amd64-static/ffprobe /usr/local/bin/ffprobe \
|
||||
&& chmod +x /usr/local/bin/ffmpeg /usr/local/bin/ffprobe \
|
||||
&& ffmpeg -version >/dev/null 2>&1 && ffprobe -version >/dev/null 2>&1 \
|
||||
&& echo "[ffmpeg] binary valid" \
|
||||
&& rm -rf ffmpeg-*
|
||||
|
||||
# ---- 安装 Python 依赖 ----
|
||||
WORKDIR /tmp
|
||||
@@ -120,13 +79,10 @@ RUN sed -i 's|deb.debian.org|mirrors.aliyun.com|g' /etc/apt/sources.list.d/debia
|
||||
|
||||
# 安装最小运行时依赖(opencv-python-headless 需要 libglib2.0-0)
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
ffmpeg \
|
||||
libglib2.0-0 \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# 从 builder 复制 ffmpeg 静态二进制
|
||||
COPY --from=builder /usr/local/bin/ffmpeg /usr/local/bin/ffmpeg
|
||||
COPY --from=builder /usr/local/bin/ffprobe /usr/local/bin/ffprobe
|
||||
|
||||
# 从 builder 复制 Python 虚拟环境
|
||||
COPY --from=builder /opt/venv /opt/venv
|
||||
|
||||
|
||||
Reference in New Issue
Block a user