Files
xiaoxia-saas/scripts/ci/docker_build_push.sh
xiaoxia bc9df316dc
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (push) Successful in 1s
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 2s
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 2s
CI/CD Pipeline / Check push changed paths (push) Successful in 16s
CI/CD Pipeline / Frontend Lint (push) Has been skipped
CI/CD Pipeline / PR Build API Image (push) Has been skipped
CI/CD Pipeline / PR Build Web Image (push) Has been skipped
CI/CD Pipeline / PR Build Worker Image (push) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (push) Successful in 3m16s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Has been skipped
AI Code Review / AI Code Review (pull_request) Failing after 6m4s
CI/CD Pipeline / Validate - Style (pull_request) Has been skipped
CI/CD Pipeline / Validate - Security (pull_request) Has been skipped
CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
CI/CD Pipeline / Frontend Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Successful in 39s
CI/CD Pipeline / PR Build Web Image (pull_request) Has been skipped
CI/CD Pipeline / Validate - Style (push) Has been cancelled
CI/CD Pipeline / Validate - Security (push) Has been cancelled
CI/CD Pipeline / Validate - Python (mypy + alembic) (push) Has been cancelled
CI/CD Pipeline / Unit Tests (push) Has been cancelled
CI/CD Pipeline / Integration Tests (push) Has been cancelled
CI/CD Pipeline / Build Staging API Image (push) Has been cancelled
CI/CD Pipeline / Build Staging Web Image (push) Has been cancelled
CI/CD Pipeline / Build Staging Worker Image (push) Has been cancelled
CI/CD Pipeline / Retag skipped Staging API Image (push) Has been cancelled
CI/CD Pipeline / Retag skipped Staging Web Image (push) Has been cancelled
CI/CD Pipeline / Retag skipped Staging Worker Image (push) Has been cancelled
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been cancelled
CI/CD Pipeline / Staging E2E Tests (push) Has been cancelled
CI/CD Pipeline / Staging API Integration Tests (push) Has been cancelled
CI/CD Pipeline / Build Production API Image (push) Has been cancelled
CI/CD Pipeline / Build Production Web Image (push) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (push) Has been cancelled
CI/CD Pipeline / Deploy Production (push) Has been cancelled
CI/CD Pipeline / Production Browser E2E (push) Has been cancelled
CI/CD Pipeline / ACR Image Cleanup (push) Has been cancelled
CI/CD Pipeline / Canary Release to Production (push) Has been cancelled
CI/CD Pipeline / CI Gate (push) Has been cancelled
CI/CD Pipeline / PR Build Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been cancelled
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been cancelled
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been cancelled
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been cancelled
CI/CD Pipeline / Build Production API Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Web Image (pull_request) Has been cancelled
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been cancelled
CI/CD Pipeline / Deploy Production (pull_request) Has been cancelled
CI/CD Pipeline / Production Browser E2E (pull_request) Has been cancelled
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been cancelled
CI/CD Pipeline / Canary Release to Production (pull_request) Has been cancelled
CI/CD Pipeline / CI Gate (pull_request) Has been cancelled
PR Automation / Auto Approve on CI Green (pull_request) Has been cancelled
Preview Deploy / Deploy Preview Environment (pull_request) Has been cancelled
fix(ci): Web 镜像 buildx 幽灵缓存防护——Git Tree Hash Cache Bust
问题:ci-builder-persist 持久化 builder 的 buildkit 层缓存偶尔出现'幽灵命中':
COPY apps/web/ ./ 步骤认为自己没变(实际文件已改),导致 vite build 不执行,
打出来的 Web 镜像前端文件是旧的。

修复原理:
把 apps/web/ 目录的 git tree hash 作为 build arg 传入 Dockerfile。
buildx 把 build arg 值作为缓存键的一部分,hash 变了 → RUN 步骤缓存失效 →
vite build 必须重新执行。依赖层(npm ci)不受影响,仍然正常缓存。

改动:
1. web.Dockerfile:新增 ARG SOURCE_HASH,在构建步骤写入 .cache_bust 文件
2. docker_build_push.sh:Web 镜像构建时计算 git tree hash 并传入
3. docker_build_only.sh:同上

验证:
- 合并后触发 push 到 develop
- 查看 Build Staging Web Image 日志,确认输出 Web cache bust: SOURCE_HASH=xxxxx
- 修改 apps/web/ 下文件再次 push,确认 SOURCE_HASH 值变化且 vite build 执行
2026-08-31 22:40:57 +08:00

147 lines
5.3 KiB
Bash
Executable File
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
#!/bin/bash
# 通用Docker镜像构建+推送脚本
# 缓存策略(2026-08 起):
# - buildx 使用宿主机持久 builder (ci-builder-persist),层缓存保存在
# buildkit 容器/命名卷中,跨 job 共享、job 结束不清理
# - registry cache 仅作为冷启动兜底读取
# - 额外 tag(如分支 tag :develop)通过 EXTRA_TAGS 环境变量传入,随构建一并推送
# 用法: docker_build_push.sh [--no-cache] <Dockerfile> <image_tag> <cache_ref> [build_arg...]
# 环境变量:
# EXTRA_TAGS 空格分隔的额外 tag(完整 image:tag 引用),可选
set -eu
# 单次 build 超时时间(秒),防止 docker buildx build 无限挂起
BUILD_TIMEOUT=1500
# 持久 builder 名(宿主机级,所有 CI job 共享;由 ensure_persistent_builder.sh 维护)
BUILDER_NAME="ci-builder-persist"
NO_CACHE_FLAG=""
if [ "$1" = "--no-cache" ]; then
NO_CACHE_FLAG="--no-cache"
shift
echo "模式: --no-cache (不使用缓存,全新构建)"
fi
DOCKERFILE="$1"
IMAGE_TAG="$2"
CACHE_REF="$3"
shift 3
BUILD_ARGS=""
for arg in "$@"; do
BUILD_ARGS="$BUILD_ARGS --build-arg $arg"
done
# Web 镜像 cache bust:计算 apps/web/ 的 git tree hash
# 当源码变化时 hash 变化,buildx 的 ARG 缓存键失效 → vite build 必定重新执行
if [ "${DOCKERFILE##*/}" = "web.Dockerfile" ]; then
SOURCE_HASH=$(git rev-parse HEAD:apps/web 2>/dev/null || echo "")
if [ -n "$SOURCE_HASH" ]; then
echo "Web cache bust: SOURCE_HASH=${SOURCE_HASH}"
BUILD_ARGS="$BUILD_ARGS --build-arg SOURCE_HASH=${SOURCE_HASH}"
else
echo "⚠️ 无法计算 apps/web tree hash,跳过 cache bust"
fi
fi
# 确保持久 builder 存在并使用(幂等)
if ! docker buildx inspect "$BUILDER_NAME" > /dev/null 2>&1; then
echo "持久 builder 不存在,创建中..."
docker buildx create --name "$BUILDER_NAME" --driver docker-container \
--driver-opt network=host \
--buildkitd-flags "--allow-insecure-entitlement network.host" \
--platform linux/amd64
fi
docker buildx use "$BUILDER_NAME"
docker buildx inspect "$BUILDER_NAME" --bootstrap
# 组装额外 tag 参数
EXTRA_TAG_FLAGS=""
EXTRA_TAG_LIST=""
if [ -n "${EXTRA_TAGS:-}" ]; then
for t in $EXTRA_TAGS; do
EXTRA_TAG_FLAGS="$EXTRA_TAG_FLAGS -t $t"
EXTRA_TAG_LIST="$EXTRA_TAG_LIST $t"
done
fi
build_with_cache_retry() {
local attempt=1
local max_attempts=2
while [ $attempt -le $max_attempts ]; do
local build_output
local exit_code
set +e
build_output=$(timeout ${BUILD_TIMEOUT} docker buildx build \
$NO_CACHE_FLAG \
$BUILD_ARGS \
--cache-from "type=registry,ref=${CACHE_REF}" \
-f "${DOCKERFILE}" \
-t "${IMAGE_TAG}" \
$EXTRA_TAG_FLAGS \
--push \
. 2>&1)
exit_code=$?
set -e
if [ $exit_code -eq 0 ]; then
echo "$build_output"
return 0
fi
# 超时退出(exit code 124
if [ $exit_code -eq 124 ]; then
echo "❌ Docker build TIMEOUT after ${BUILD_TIMEOUT}s - build hung and was killed"
echo "$build_output" | tail -20
return $exit_code
fi
# 检测到缓存/快照损坏类错误,重建 builder 后重试
if echo "$build_output" | grep -qE "parent snapshot.*not found|snapshot.*does not exist|cache.*corrupt|failed to compute cache key|no such file or directory.*cache"; then
echo "$build_output"
echo ""
echo "⚠️ builder 缓存异常,重建持久 builder 后重试 (attempt $attempt/$max_attempts)..."
# 共享 builder 的重建必须串行:ci-builder-persist 被所有 build job 共用,
# 若 job A 正在构建、job B 检测到损坏直接 rm,会把 A 正在用的 buildkit 杀掉。
# 用 flock 串行化重建;拿到锁后再次检查 builder 健康度,已被别的 job 重建则直接复用。
LOCK_FILE="/tmp/ci-builder-persist-rebuild.lock"
exec 9>"$LOCK_FILE"
echo "🔒 等待重建锁(最多 120s..."
if flock -w 120 9; then
if docker buildx inspect "$BUILDER_NAME" --bootstrap >/dev/null 2>&1; then
echo "✅ builder 已被其他并发 job 重建/恢复,直接复用"
else
echo "🔨 锁内重建持久 builder..."
docker buildx rm "$BUILDER_NAME" >/dev/null 2>&1 || true
docker buildx create --name "$BUILDER_NAME" --driver docker-container \
--driver-opt network=host \
--buildkitd-flags "--allow-insecure-entitlement network.host" \
--platform linux/amd64
docker buildx use "$BUILDER_NAME"
docker buildx inspect "$BUILDER_NAME" --bootstrap
fi
else
echo "⚠️ 等待重建锁超时,直接重试 build(失败将重试/--no-cache"
docker buildx use "$BUILDER_NAME" 2>/dev/null || true
fi
attempt=$((attempt + 1))
else
echo "$build_output"
return $exit_code
fi
done
return 1
}
echo "=== Build & push image (persistent builder cache) ==="
echo "Builder: ${BUILDER_NAME} (persistent)"
echo "Registry cache(from): ${CACHE_REF}"
echo "Image tag: ${IMAGE_TAG}"
[ -n "$EXTRA_TAG_LIST" ] && echo "Extra tags: ${EXTRA_TAG_LIST}"
echo "Timeout: ${BUILD_TIMEOUT}s"
echo ""
build_with_cache_retry
echo ""
echo "Image pushed: ${IMAGE_TAG}"
[ -n "$EXTRA_TAG_LIST" ] && echo "Also pushed: ${EXTRA_TAG_LIST}"
echo ""
echo "Build completed: ${IMAGE_TAG}"