fix(ci): buildx本地缓存损坏自动检测+清理重试,根治snapshot not found连锁失败
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 21s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m13s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 1m15s
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Successful in 2m11s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 3m19s
Auto Approve CI PRs / Auto Approve on CI Green (pull_request) Successful in 3m47s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m36s
Auto Merge CI PRs / Auto Merge on CI Green + Approved (pull_request) Successful in 4m14s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 20s
AI Code Review / AI Code Review (pull_request) Successful in 7m58s
CI Build & Deploy Pipeline / Production Browser E2E (pull_request) Failing after 1439h18m8s
CI Build & Deploy Pipeline / Build Production API Image (pull_request) Failing after 1439h18m11s
CI Build & Deploy Pipeline / Deploy Production (pull_request) Failing after 1439h18m10s
CI Build & Deploy Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Failing after 1439h18m12s
CI Build & Deploy Pipeline / Staging API Integration Tests (pull_request) Failing after 1439h18m10s
CI Build & Deploy Pipeline / Build Staging Web Image (pull_request) Failing after 1439h18m13s
CI Build & Deploy Pipeline / Staging E2E Tests (pull_request) Failing after 1439h18m10s
CI Build & Deploy Pipeline / Build Production Worker Image (pull_request) Failing after 1439h18m11s
CI Build & Deploy Pipeline / Build Staging API Image (pull_request) Failing after 1439h18m13s
CI Build & Deploy Pipeline / Build Production Web Image (pull_request) Failing after 1439h50m13s
CI Build & Deploy Pipeline / Build Staging Worker Image (pull_request) Failing after 1439h50m15s
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 21s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 1m13s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 1m15s
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Successful in 2m11s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 3m19s
Auto Approve CI PRs / Auto Approve on CI Green (pull_request) Successful in 3m47s
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m36s
Auto Merge CI PRs / Auto Merge on CI Green + Approved (pull_request) Successful in 4m14s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 20s
AI Code Review / AI Code Review (pull_request) Successful in 7m58s
CI Build & Deploy Pipeline / Production Browser E2E (pull_request) Failing after 1439h18m8s
CI Build & Deploy Pipeline / Build Production API Image (pull_request) Failing after 1439h18m11s
CI Build & Deploy Pipeline / Deploy Production (pull_request) Failing after 1439h18m10s
CI Build & Deploy Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Failing after 1439h18m12s
CI Build & Deploy Pipeline / Staging API Integration Tests (pull_request) Failing after 1439h18m10s
CI Build & Deploy Pipeline / Build Staging Web Image (pull_request) Failing after 1439h18m13s
CI Build & Deploy Pipeline / Staging E2E Tests (pull_request) Failing after 1439h18m10s
CI Build & Deploy Pipeline / Build Production Worker Image (pull_request) Failing after 1439h18m11s
CI Build & Deploy Pipeline / Build Staging API Image (pull_request) Failing after 1439h18m13s
CI Build & Deploy Pipeline / Build Production Web Image (pull_request) Failing after 1439h50m13s
CI Build & Deploy Pipeline / Build Staging Worker Image (pull_request) Failing after 1439h50m15s
- 新增build_with_cache_retry()函数:检测缓存损坏类错误自动清理本地缓存并重试 - 覆盖错误模式:parent snapshot not found、cache corrupt、failed to compute cache key - 两级降级:本地缓存重试1次 → 完全不用本地缓存只用registry缓存 - 同时清理buildx builder内部snapshot状态(docker buildx prune -a) - 非缓存类错误直接透传,不误重试
This commit is contained in:
@@ -28,27 +28,64 @@ LOCAL_CACHE_DIR="/tmp/buildx-cache/${CACHE_NAME}"
|
||||
|
||||
mkdir -p "$LOCAL_CACHE_DIR"
|
||||
|
||||
# 缓存源:local优先,registry兜底
|
||||
CACHE_FROM_LOCAL="type=local,src=${LOCAL_CACHE_DIR}"
|
||||
CACHE_FROM_REGISTRY="type=registry,ref=${CACHE_REF},ignore-error=true"
|
||||
# 缓存源:local优先(带自动修复),registry兜底
|
||||
# 本地缓存损坏时自动清理后重试,避免snapshot not found导致构建全挂
|
||||
build_with_cache_retry() {
|
||||
local attempt=1
|
||||
local max_attempts=2
|
||||
while [ $attempt -le $max_attempts ]; do
|
||||
local build_output
|
||||
local exit_code
|
||||
set +e
|
||||
build_output=$(docker buildx build \
|
||||
$BUILD_ARGS \
|
||||
--cache-from "type=local,src=${LOCAL_CACHE_DIR}" \
|
||||
--cache-from "type=registry,ref=${CACHE_REF},ignore-error=true" \
|
||||
--cache-to "type=local,dest=${LOCAL_CACHE_DIR},mode=max" \
|
||||
-f "${DOCKERFILE}" \
|
||||
-t "${IMAGE_TAG}" \
|
||||
--push \
|
||||
. 2>&1)
|
||||
exit_code=$?
|
||||
set -e
|
||||
if [ $exit_code -eq 0 ]; then
|
||||
echo "$build_output"
|
||||
return 0
|
||||
fi
|
||||
# 检测到缓存损坏类错误,清掉本地缓存重试
|
||||
if echo "$build_output" | grep -qE "parent snapshot.*not found|snapshot.*does not exist|cache.*corrupt|failed to compute cache key"; then
|
||||
echo "$build_output"
|
||||
echo ""
|
||||
echo "⚠️ Local cache appears corrupted, cleaning up and retrying (attempt $attempt/$max_attempts)..."
|
||||
rm -rf "${LOCAL_CACHE_DIR}"
|
||||
mkdir -p "${LOCAL_CACHE_DIR}"
|
||||
# 清理buildx builder的内部snapshot状态
|
||||
docker buildx prune -f -a >/dev/null 2>&1 || true
|
||||
attempt=$((attempt + 1))
|
||||
else
|
||||
# 非缓存类错误,直接输出并返回
|
||||
echo "$build_output"
|
||||
return $exit_code
|
||||
fi
|
||||
done
|
||||
# 重试完还是失败,不用本地缓存最后试一次(只从registry读)
|
||||
echo "⚠️ All cached attempts failed, building without local cache..."
|
||||
docker buildx build \
|
||||
$BUILD_ARGS \
|
||||
--cache-from "type=registry,ref=${CACHE_REF},ignore-error=true" \
|
||||
--cache-to "type=local,dest=${LOCAL_CACHE_DIR},mode=max" \
|
||||
-f "${DOCKERFILE}" \
|
||||
-t "${IMAGE_TAG}" \
|
||||
--push \
|
||||
.
|
||||
}
|
||||
|
||||
# 本地缓存目标(必选,mode=max最大化命中率)
|
||||
CACHE_TO_LOCAL="type=local,dest=${LOCAL_CACHE_DIR},mode=max"
|
||||
|
||||
echo "=== Step 1: Build & push image (local cache read-write + registry read) ==="
|
||||
echo "=== Step 1: Build & push image (local cache + registry read, with auto-repair) ==="
|
||||
echo "Local cache: ${LOCAL_CACHE_DIR}"
|
||||
echo "Registry cache: ${CACHE_REF}"
|
||||
echo ""
|
||||
|
||||
docker buildx build \
|
||||
$BUILD_ARGS \
|
||||
--cache-from "${CACHE_FROM_LOCAL}" \
|
||||
--cache-from "${CACHE_FROM_REGISTRY}" \
|
||||
--cache-to "${CACHE_TO_LOCAL}" \
|
||||
-f "${DOCKERFILE}" \
|
||||
-t "${IMAGE_TAG}" \
|
||||
--push \
|
||||
.
|
||||
build_with_cache_retry
|
||||
|
||||
echo ""
|
||||
echo "Image pushed: ${IMAGE_TAG}"
|
||||
|
||||
Reference in New Issue
Block a user