Compare commits

..

6 Commits

Author SHA1 Message Date
xiaoxia 7941654224 fix(ci): 配置docker-container buildx builder以支持cache export
新CI服务器默认docker driver不支持buildx cache export功能,
在每个构建Job中自动创建/使用docker-container类型的builder实例,
确保缓存读写正常工作。
2026-07-14 15:21:52 +08:00
CI Bot c34c1634fc fix: update runner labels to match new scheme [host, ci-check] / [host, build-only]
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Successful in 42s
CI/CD Pipeline / Unit Tests (pull_request) Successful in 1m23s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 1m55s
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / Integration Tests (pull_request) Successful in 1m20s
2026-07-14 15:06:27 +08:00
xiaoxia cf881c6ba3 feat(ci): 镜像并行构建 - api/worker/web三镜像并行,构建时间从17min压缩到8-10min 2026-07-14 15:06:27 +08:00
xiaoxia 08b51ffa1d feat(ci): Runner标签分层路由 - 测试/构建分机运行 (#315)
CI/CD Pipeline / Unit Tests (push) Successful in 1m17s
CI/CD Pipeline / Frontend Lint (push) Successful in 2m11s
CI/CD Pipeline / Validate Code Quality And Tests (push) Successful in 2m21s
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Integration Tests (push) Successful in 1m35s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Has been cancelled
CI/CD Pipeline / Staging E2E Tests (push) Has been cancelled
CI/CD Pipeline / Staging API Integration Tests (push) Has been cancelled
2026-07-14 14:56:08 +08:00
xiaoxia 23d2406c27 feat(ci): 集成gitleaks + pip-audit + vulture安全扫描到Validate阶段 (#311)
CI/CD Pipeline / Validate Code Quality And Tests (push) Has been cancelled
CI/CD Pipeline / Unit Tests (push) Has been cancelled
CI/CD Pipeline / Integration Tests (push) Has been cancelled
CI/CD Pipeline / Frontend Lint (push) Has been cancelled
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Has been cancelled
CI/CD Pipeline / Staging E2E Tests (push) Has been cancelled
CI/CD Pipeline / Staging API Integration Tests (push) Has been cancelled
CI/CD Pipeline / Build Production Runtime Images (push) Has been cancelled
CI/CD Pipeline / Deploy Production (push) Has been cancelled
CI/CD Pipeline / Production Browser E2E (push) Has been cancelled
2026-07-14 14:55:58 +08:00
xiaoxia def6ee2363 feat(ci): 新增发布与灰度部署脚本
CI/CD Pipeline / Validate Code Quality And Tests (push) Successful in 46s
CI/CD Pipeline / Unit Tests (push) Successful in 1m40s
CI/CD Pipeline / Frontend Lint (push) Successful in 2m52s
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Integration Tests (push) Successful in 1m42s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Successful in 16m34s
CI/CD Pipeline / Staging E2E Tests (push) Successful in 2m52s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 3m55s
合并PR #316:新增release.sh / gray_deploy.sh / rollback_gray.sh三个脚本,适配生产环境架构(nginx upstream权重 + canary容器)
2026-07-14 13:12:41 +08:00
5 changed files with 723 additions and 82 deletions
+1 -1
View File
@@ -1 +1 @@
# CI trigger Fri Jun 26 09:53:28 PM CST 2026
trigger: 1784009947
+578 -81
View File
@@ -651,10 +651,10 @@ jobs:
echo "=== CI 失败通知 ==="
FAILED_JOB="Frontend Lint" python3 scripts/ci_notify_failure.py
deploy-staging:
name: Build & Push Staging (Watchtower auto-deploy)
build-staging-api:
name: Build Staging API Image
runs-on: saas
timeout-minutes: 30
timeout-minutes: 20
needs: [validate, frontend-lint]
if: github.event_name == 'push' && (github.ref_name == 'main' || github.ref_name == 'develop')
@@ -704,30 +704,347 @@ jobs:
if member.name:
tar.extract(member, '.')
INNERPY
- name: Build and push all images to Gitea Registry
- name: Docker login to Registry
shell: sh
env:
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
set -eu
chmod +x scripts/build_release_images.sh
ALLOW_SHARED_PRODUCTION_BUILD_HOST=true REGISTRY_TOKEN="${REGISTRY_TOKEN}" \
scripts/build_release_images.sh "${GITHUB_SHA}" staging
printf '%s' "${REGISTRY_TOKEN}" | docker login git.xiaoxiajianji.com -u xiaoxia --password-stdin
echo "Docker login successful"
- name: Setup cache strategy
shell: sh
run: |
set -eu
# develop/main 分支写回缓存,其他分支只读
if [ "${GITHUB_REF_NAME}" = "develop" ] || [ "${GITHUB_REF_NAME}" = "main" ]; then
echo "CACHE_MODE=read-write" >> $GITHUB_ENV
echo "Cache mode: read-write (will push cache)"
else
echo "CACHE_MODE=read-only" >> $GITHUB_ENV
echo "Cache mode: read-only"
fi
- name: Tag and push :staging images (Watchtower auto-update)
- name: Setup buildx builder (docker-container driver)
shell: sh
run: |
set -eu
# 确保使用 docker-container driver 以支持 cache export 功能
if ! docker buildx inspect ci-builder > /dev/null 2>&1; then
docker buildx create --use --name ci-builder --driver docker-container
echo "Created ci-builder (docker-container driver)"
else
docker buildx use ci-builder
echo "Using existing ci-builder"
fi
docker buildx inspect --bootstrap
- name: Build and push API image (buildx cache)
shell: sh
env:
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
set -eu
REGISTRY="git.xiaoxiajianji.com/xiaoxia/xiaoxia-saas"
if [ -n "${REGISTRY_TOKEN:-}" ]; then
printf '%s' "${REGISTRY_TOKEN}" | docker login git.xiaoxiajianji.com -u xiaoxia --password-stdin 2>/dev/null
IMAGE_NAME="xiaoxia-saas-api"
CACHE_REF="${REGISTRY}/api-cache:develop"
CACHE_FROM="type=registry,ref=${CACHE_REF},ignore-error=true"
if [ "${CACHE_MODE}" = "read-write" ]; then
CACHE_TO="type=registry,ref=${CACHE_REF},mode=max"
echo "Building API image with read-write cache..."
docker buildx build --build-arg APP_VERSION="${GITHUB_SHA}" --cache-from "${CACHE_FROM}" --cache-to "${CACHE_TO}" -f infra/docker/api.Dockerfile -t "${REGISTRY}/${IMAGE_NAME}:${GITHUB_SHA}" --push .
else
echo "Building API image with read-only cache..."
docker buildx build --build-arg APP_VERSION="${GITHUB_SHA}" --cache-from "${CACHE_FROM}" -f infra/docker/api.Dockerfile -t "${REGISTRY}/${IMAGE_NAME}:${GITHUB_SHA}" --push .
fi
echo "API image pushed: ${REGISTRY}/${IMAGE_NAME}:${GITHUB_SHA}"
- name: Notify CI failure
if: failure()
shell: sh
run: |
set +e
echo "=== CI 失败通知 ==="
FAILED_JOB="Build Staging API Image" python3 scripts/ci_notify_failure.py
build-staging-worker:
name: Build Staging Worker Image
runs-on: saas
timeout-minutes: 20
needs: [validate, frontend-lint]
if: github.event_name == 'push' && (github.ref_name == 'main' || github.ref_name == 'develop')
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
python3 - <<'INNERPY'
import io, os, tarfile, time, urllib.request, urllib.error
url = f"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz"
request = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['GITHUB_TOKEN']}"})
last_err = None
for attempt in range(5):
try:
with urllib.request.urlopen(request, timeout=120) as response:
archive = response.read()
break
except urllib.error.HTTPError as e:
last_err = e
if e.code >= 500 and attempt < 4:
wait = 2 ** attempt
print(f"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
except Exception as e:
last_err = e
if attempt < 4:
wait = 2 ** attempt
print(f"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
else:
raise last_err
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
INNERPY
- name: Docker login to Registry
shell: sh
env:
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
set -eu
printf '%s' "${REGISTRY_TOKEN}" | docker login git.xiaoxiajianji.com -u xiaoxia --password-stdin
echo "Docker login successful"
- name: Setup cache strategy
shell: sh
run: |
set -eu
# develop/main 分支写回缓存,其他分支只读
if [ "${GITHUB_REF_NAME}" = "develop" ] || [ "${GITHUB_REF_NAME}" = "main" ]; then
echo "CACHE_MODE=read-write" >> $GITHUB_ENV
echo "Cache mode: read-write (will push cache)"
else
echo "CACHE_MODE=read-only" >> $GITHUB_ENV
echo "Cache mode: read-only"
fi
- name: Setup buildx builder (docker-container driver)
shell: sh
run: |
set -eu
# 确保使用 docker-container driver 以支持 cache export 功能
if ! docker buildx inspect ci-builder > /dev/null 2>&1; then
docker buildx create --use --name ci-builder --driver docker-container
echo "Created ci-builder (docker-container driver)"
else
docker buildx use ci-builder
echo "Using existing ci-builder"
fi
docker buildx inspect --bootstrap
- name: Build and push Worker image (buildx cache)
shell: sh
run: |
set -eu
REGISTRY="git.xiaoxiajianji.com/xiaoxia/xiaoxia-saas"
IMAGE_NAME="xiaoxia-saas-worker"
CACHE_REF="${REGISTRY}/worker-cache:develop"
CACHE_FROM="type=registry,ref=${CACHE_REF},ignore-error=true"
if [ "${CACHE_MODE}" = "read-write" ]; then
CACHE_TO="type=registry,ref=${CACHE_REF},mode=max"
echo "Building Worker image with read-write cache..."
docker buildx build --build-arg APP_VERSION="${GITHUB_SHA}" --cache-from "${CACHE_FROM}" --cache-to "${CACHE_TO}" -f infra/docker/worker.Dockerfile -t "${REGISTRY}/${IMAGE_NAME}:${GITHUB_SHA}" --push .
else
echo "Building Worker image with read-only cache..."
docker buildx build --build-arg APP_VERSION="${GITHUB_SHA}" --cache-from "${CACHE_FROM}" -f infra/docker/worker.Dockerfile -t "${REGISTRY}/${IMAGE_NAME}:${GITHUB_SHA}" --push .
fi
echo "Worker image pushed: ${REGISTRY}/${IMAGE_NAME}:${GITHUB_SHA}"
- name: Notify CI failure
if: failure()
shell: sh
run: |
set +e
echo "=== CI 失败通知 ==="
FAILED_JOB="Build Staging Worker Image" python3 scripts/ci_notify_failure.py
build-staging-web:
name: Build Staging Web Image
runs-on: saas
timeout-minutes: 20
needs: [validate, frontend-lint]
if: github.event_name == 'push' && (github.ref_name == 'main' || github.ref_name == 'develop')
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
python3 - <<'INNERPY'
import io, os, tarfile, time, urllib.request, urllib.error
url = f"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz"
request = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['GITHUB_TOKEN']}"})
last_err = None
for attempt in range(5):
try:
with urllib.request.urlopen(request, timeout=120) as response:
archive = response.read()
break
except urllib.error.HTTPError as e:
last_err = e
if e.code >= 500 and attempt < 4:
wait = 2 ** attempt
print(f"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
except Exception as e:
last_err = e
if attempt < 4:
wait = 2 ** attempt
print(f"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
else:
raise last_err
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
INNERPY
- name: Docker login to Registry
shell: sh
env:
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
set -eu
printf '%s' "${REGISTRY_TOKEN}" | docker login git.xiaoxiajianji.com -u xiaoxia --password-stdin
echo "Docker login successful"
- name: Setup cache strategy
shell: sh
run: |
set -eu
# develop/main 分支写回缓存,其他分支只读
if [ "${GITHUB_REF_NAME}" = "develop" ] || [ "${GITHUB_REF_NAME}" = "main" ]; then
echo "CACHE_MODE=read-write" >> $GITHUB_ENV
echo "Cache mode: read-write (will push cache)"
else
echo "CACHE_MODE=read-only" >> $GITHUB_ENV
echo "Cache mode: read-only"
fi
- name: Build frontend assets (npm build)
shell: sh
run: |
set -eu
NPM_CACHE_VOLUME="xiaoxia-npm-cache"
if ! docker volume inspect "$NPM_CACHE_VOLUME" >/dev/null 2>&1; then
docker volume create "$NPM_CACHE_VOLUME" >/dev/null
echo "Created npm cache volume: $NPM_CACHE_VOLUME"
fi
docker run --rm -v "$PWD:/workspace" -v "$NPM_CACHE_VOLUME:/workspace/apps/web/node_modules" -w /workspace/apps/web docker.m.daocloud.io/library/node:20 sh -lc "npm ci && npm run build"
test -f apps/web/dist/index.html
echo "Frontend build complete: $(ls apps/web/dist/ | head -5)"
- name: Setup buildx builder (docker-container driver)
shell: sh
run: |
set -eu
# 确保使用 docker-container driver 以支持 cache export 功能
if ! docker buildx inspect ci-builder > /dev/null 2>&1; then
docker buildx create --use --name ci-builder --driver docker-container
echo "Created ci-builder (docker-container driver)"
else
docker buildx use ci-builder
echo "Using existing ci-builder"
fi
docker buildx inspect --bootstrap
- name: Build and push Web image (buildx cache)
shell: sh
run: |
set -eu
REGISTRY="git.xiaoxiajianji.com/xiaoxia/xiaoxia-saas"
IMAGE_NAME="xiaoxia-saas-web"
CACHE_REF="${REGISTRY}/web-cache:develop"
NGINX_CONF="infra/docker/nginx-staging.conf"
CACHE_FROM="type=registry,ref=${CACHE_REF},ignore-error=true"
if [ "${CACHE_MODE}" = "read-write" ]; then
CACHE_TO="type=registry,ref=${CACHE_REF},mode=max"
echo "Building Web image with read-write cache..."
docker buildx build --cache-from "${CACHE_FROM}" --cache-to "${CACHE_TO}" -f infra/docker/web-artifact.Dockerfile --build-arg "NGINX_CONF=${NGINX_CONF}" -t "${REGISTRY}/${IMAGE_NAME}:${GITHUB_SHA}" --push .
else
echo "Building Web image with read-only cache..."
docker buildx build --cache-from "${CACHE_FROM}" -f infra/docker/web-artifact.Dockerfile --build-arg "NGINX_CONF=${NGINX_CONF}" -t "${REGISTRY}/${IMAGE_NAME}:${GITHUB_SHA}" --push .
fi
echo "Web image pushed: ${REGISTRY}/${IMAGE_NAME}:${GITHUB_SHA}"
- name: Notify CI failure
if: failure()
shell: sh
run: |
set +e
echo "=== CI 失败通知 ==="
FAILED_JOB="Build Staging Web Image" python3 scripts/ci_notify_failure.py
deploy-staging:
name: Deploy Staging (Watchtower auto-deploy)
runs-on: saas
timeout-minutes: 15
needs: [build-staging-api, build-staging-worker, build-staging-web]
if: github.event_name == 'push' && (github.ref_name == 'main' || github.ref_name == 'develop')
steps:
- name: Docker login to Registry
shell: sh
env:
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
set -eu
printf '%s' "${REGISTRY_TOKEN}" | docker login git.xiaoxiajianji.com -u xiaoxia --password-stdin
echo "Docker login successful"
- name: Tag and push :staging images (Watchtower auto-update)
shell: sh
run: |
set -eu
REGISTRY="git.xiaoxiajianji.com/xiaoxia/xiaoxia-saas"
for svc in api worker web; do
echo "Pulling ${REGISTRY}/xiaoxia-saas-${svc}:${GITHUB_SHA} ..."
docker pull "${REGISTRY}/xiaoxia-saas-${svc}:${GITHUB_SHA}"
docker tag "${REGISTRY}/xiaoxia-saas-${svc}:${GITHUB_SHA}" "${REGISTRY}/xiaoxia-saas-${svc}:staging"
docker push "${REGISTRY}/xiaoxia-saas-${svc}:staging"
echo "$svc :staging tagged and pushed"
done
echo "All :staging images pushed. Watchtower will auto-deploy within 60s."
@@ -796,8 +1113,7 @@ jobs:
run: |
set +e
echo "=== CI 失败通知 ==="
FAILED_JOB="Build & Push Staging (Watchtower auto-deploy)" python3 scripts/ci_notify_failure.py
FAILED_JOB="Deploy Staging" python3 scripts/ci_notify_failure.py
staging-e2e:
@@ -951,10 +1267,10 @@ jobs:
build-production-runtime-images:
name: Build Production Runtime Images
build-production-api:
name: Build Production API Image
runs-on: saas
timeout-minutes: 30
timeout-minutes: 20
needs: [validate, frontend-lint]
if: startsWith(github.ref, 'refs/tags/v')
@@ -966,7 +1282,7 @@ jobs:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
python3 - <<'PY'
python3 - <<'INNERPY'
import io, os, tarfile, time, urllib.request, urllib.error
url = f"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz"
request = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['GITHUB_TOKEN']}"})
@@ -991,7 +1307,6 @@ jobs:
print(f"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
else:
raise last_err
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:
@@ -1004,16 +1319,251 @@ jobs:
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
PY
- name: Build and push all images (api + worker + web, with buildx cache)
INNERPY
- name: Docker login to Registry
shell: sh
env:
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
set -eu
chmod +x scripts/build_release_images.sh
REGISTRY_TOKEN="${REGISTRY_TOKEN}" scripts/build_release_images.sh "${GITHUB_REF_NAME}"
printf '%s' "${REGISTRY_TOKEN}" | docker login git.xiaoxiajianji.com -u xiaoxia --password-stdin
echo "Docker login successful"
- name: Setup buildx builder (docker-container driver)
shell: sh
run: |
set -eu
# 确保使用 docker-container driver 以支持 cache export 功能
if ! docker buildx inspect ci-builder > /dev/null 2>&1; then
docker buildx create --use --name ci-builder --driver docker-container
echo "Created ci-builder (docker-container driver)"
else
docker buildx use ci-builder
echo "Using existing ci-builder"
fi
docker buildx inspect --bootstrap
- name: Build and push API image (buildx cache)
shell: sh
run: |
set -eu
REGISTRY="git.xiaoxiajianji.com/xiaoxia/xiaoxia-saas"
IMAGE_NAME="xiaoxia-saas-api"
VERSION="${GITHUB_REF_NAME}"
CACHE_REF="${REGISTRY}/api-cache:main"
echo "Building Production API image: ${VERSION}"
docker buildx build --build-arg APP_VERSION="${VERSION}" --cache-from "type=registry,ref=${CACHE_REF},ignore-error=true" --cache-to "type=registry,ref=${CACHE_REF},mode=max" -f infra/docker/api.Dockerfile -t "${REGISTRY}/${IMAGE_NAME}:${VERSION}" --push .
echo "Production API image pushed: ${REGISTRY}/${IMAGE_NAME}:${VERSION}"
- name: Notify CI failure
if: failure()
shell: sh
run: |
set +e
echo "=== CI 失败通知 ==="
FAILED_JOB="Build Production API Image" python3 scripts/ci_notify_failure.py
build-production-worker:
name: Build Production Worker Image
runs-on: saas
timeout-minutes: 20
needs: [validate, frontend-lint]
if: startsWith(github.ref, 'refs/tags/v')
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
python3 - <<'INNERPY'
import io, os, tarfile, time, urllib.request, urllib.error
url = f"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz"
request = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['GITHUB_TOKEN']}"})
last_err = None
for attempt in range(5):
try:
with urllib.request.urlopen(request, timeout=120) as response:
archive = response.read()
break
except urllib.error.HTTPError as e:
last_err = e
if e.code >= 500 and attempt < 4:
wait = 2 ** attempt
print(f"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
except Exception as e:
last_err = e
if attempt < 4:
wait = 2 ** attempt
print(f"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
else:
raise last_err
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
INNERPY
- name: Docker login to Registry
shell: sh
env:
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
set -eu
printf '%s' "${REGISTRY_TOKEN}" | docker login git.xiaoxiajianji.com -u xiaoxia --password-stdin
echo "Docker login successful"
- name: Setup buildx builder (docker-container driver)
shell: sh
run: |
set -eu
# 确保使用 docker-container driver 以支持 cache export 功能
if ! docker buildx inspect ci-builder > /dev/null 2>&1; then
docker buildx create --use --name ci-builder --driver docker-container
echo "Created ci-builder (docker-container driver)"
else
docker buildx use ci-builder
echo "Using existing ci-builder"
fi
docker buildx inspect --bootstrap
- name: Build and push Worker image (buildx cache)
shell: sh
run: |
set -eu
REGISTRY="git.xiaoxiajianji.com/xiaoxia/xiaoxia-saas"
IMAGE_NAME="xiaoxia-saas-worker"
VERSION="${GITHUB_REF_NAME}"
CACHE_REF="${REGISTRY}/worker-cache:main"
echo "Building Production Worker image: ${VERSION}"
docker buildx build --build-arg APP_VERSION="${VERSION}" --cache-from "type=registry,ref=${CACHE_REF},ignore-error=true" --cache-to "type=registry,ref=${CACHE_REF},mode=max" -f infra/docker/worker.Dockerfile -t "${REGISTRY}/${IMAGE_NAME}:${VERSION}" --push .
echo "Production Worker image pushed: ${REGISTRY}/${IMAGE_NAME}:${VERSION}"
- name: Notify CI failure
if: failure()
shell: sh
run: |
set +e
echo "=== CI 失败通知 ==="
FAILED_JOB="Build Production Worker Image" python3 scripts/ci_notify_failure.py
build-production-web:
name: Build Production Web Image
runs-on: saas
timeout-minutes: 20
needs: [validate, frontend-lint]
if: startsWith(github.ref, 'refs/tags/v')
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
python3 - <<'INNERPY'
import io, os, tarfile, time, urllib.request, urllib.error
url = f"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz"
request = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['GITHUB_TOKEN']}"})
last_err = None
for attempt in range(5):
try:
with urllib.request.urlopen(request, timeout=120) as response:
archive = response.read()
break
except urllib.error.HTTPError as e:
last_err = e
if e.code >= 500 and attempt < 4:
wait = 2 ** attempt
print(f"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
except Exception as e:
last_err = e
if attempt < 4:
wait = 2 ** attempt
print(f"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
else:
raise last_err
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
INNERPY
- name: Docker login to Registry
shell: sh
env:
REGISTRY_TOKEN: ${{ secrets.REGISTRY_TOKEN }}
run: |
set -eu
printf '%s' "${REGISTRY_TOKEN}" | docker login git.xiaoxiajianji.com -u xiaoxia --password-stdin
echo "Docker login successful"
- name: Build frontend assets (npm build)
shell: sh
run: |
set -eu
NPM_CACHE_VOLUME="xiaoxia-npm-cache"
if ! docker volume inspect "$NPM_CACHE_VOLUME" >/dev/null 2>&1; then
docker volume create "$NPM_CACHE_VOLUME" >/dev/null
fi
docker run --rm -v "$PWD:/workspace" -v "$NPM_CACHE_VOLUME:/workspace/apps/web/node_modules" -w /workspace/apps/web docker.m.daocloud.io/library/node:20 sh -lc "npm ci && npm run build"
test -f apps/web/dist/index.html
echo "Frontend build complete"
- name: Setup buildx builder (docker-container driver)
shell: sh
run: |
set -eu
# 确保使用 docker-container driver 以支持 cache export 功能
if ! docker buildx inspect ci-builder > /dev/null 2>&1; then
docker buildx create --use --name ci-builder --driver docker-container
echo "Created ci-builder (docker-container driver)"
else
docker buildx use ci-builder
echo "Using existing ci-builder"
fi
docker buildx inspect --bootstrap
- name: Build and push Web image (buildx cache)
shell: sh
run: |
set -eu
REGISTRY="git.xiaoxiajianji.com/xiaoxia/xiaoxia-saas"
IMAGE_NAME="xiaoxia-saas-web"
VERSION="${GITHUB_REF_NAME}"
CACHE_REF="${REGISTRY}/web-cache:main"
NGINX_CONF="infra/docker/nginx-production.conf"
echo "Building Production Web image: ${VERSION}"
docker buildx build --cache-from "type=registry,ref=${CACHE_REF},ignore-error=true" --cache-to "type=registry,ref=${CACHE_REF},mode=max" -f infra/docker/web-artifact.Dockerfile --build-arg "NGINX_CONF=${NGINX_CONF}" -t "${REGISTRY}/${IMAGE_NAME}:${VERSION}" --push .
echo "Production Web image pushed: ${REGISTRY}/${IMAGE_NAME}:${VERSION}"
- name: Cleanup old Docker images
if: always()
@@ -1036,63 +1586,16 @@ jobs:
run: |
set +e
echo "=== CI 失败通知 ==="
FAILED_JOB="Build Production Runtime Images" python3 scripts/ci_notify_failure.py
FAILED_JOB="Build Production Web Image" python3 scripts/ci_notify_failure.py
deploy-production:
name: Deploy Production
runs-on: saas
timeout-minutes: 20
if: startsWith(github.ref, 'refs/tags/v')
needs: build-production-runtime-images
needs: [build-production-api, build-production-worker, build-production-web]
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
python3 - <<'PY'
import io, os, tarfile, time, urllib.request, urllib.error
url = f"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz"
request = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['GITHUB_TOKEN']}"})
last_err = None
for attempt in range(5):
try:
with urllib.request.urlopen(request, timeout=120) as response:
archive = response.read()
break
except urllib.error.HTTPError as e:
last_err = e
if e.code >= 500 and attempt < 4:
wait = 2 ** attempt
print(f"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
except Exception as e:
last_err = e
if attempt < 4:
wait = 2 ** attempt
print(f"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
else:
raise last_err
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
PY
- name: Install SSH client
shell: sh
run: |
@@ -1127,16 +1630,10 @@ jobs:
key_path="$HOME/.ssh/xiaoxia_runtime_builder"
echo "Using key: $key_path (home key)"
elif [ -n "${PRODUCTION_SSH_KEY:-}" ]; then
key_path="$HOME/.ssh/production_deploy_key"
key_path="$HOME/.ssh/id_ed25519"
printf '%s\n' "$PRODUCTION_SSH_KEY" > "$key_path"
chmod 600 "$key_path"
echo "Using key from PRODUCTION_SSH_KEY secret"
elif [ -f "$HOME/.ssh/id_ed25519" ]; then
key_path="$HOME/.ssh/id_ed25519"
echo "Using key: $key_path (default id_ed25519)"
elif [ -f /root/.ssh/id_ed25519 ]; then
key_path="/root/.ssh/id_ed25519"
echo "Using key: $key_path (root id_ed25519)"
else
echo "ERROR: No SSH key available"
ls -la ~/.ssh/ 2>/dev/null || true
+52
View File
@@ -0,0 +1,52 @@
# .gitleaks.toml - gitleaks 白名单配置
# 仓库: xiaoxia/xiaoxia-saas
# 用途: 排除已知的测试密钥、示例配置等误报
# 允许路径/文件排除
[allowlist]
description = "全局白名单 - 排除示例配置和测试文件"
paths = [
# 环境配置示例(无真实密钥)
'.env.example',
'.env.sample',
'*.env.example',
'*.env.sample',
# 测试文件
'tests/',
'test/',
'*/tests/',
'*/test/',
# 文档
'docs/',
'*.md',
'*.rst',
# 前端依赖
'node_modules/',
# Python包
'site-packages/',
# 锁定文件(自动生成)
'poetry.lock',
'Pipfile.lock',
'requirements*.txt.lock',
# CI配置本身
'.gitea/',
# Docker相关
'docker-compose*.yml',
# gitleaks配置自身
'.gitleaks.toml',
]
# 允许的密钥值/占位符正则
regexes = [
# 占位符模式
'''(?i)(your[_-]?password|your[_-]?secret|your[_-]?key|your[_-]?token|changeme|change[_-]?me|placeholder|example[_-]?key|test[_-]?key|dummy|fake|mock|xxx|none|not[_-]?set|TODO|FIXME)''',
# 数据库连接字符串中的通用密码(PostgreSQL示例配置)
'''postgresql://[^:]+:changeme@''',
'''postgresql://[^:]+:your-password@''',
'''postgresql://[^:]+:password@localhost''',
# Redis示例配置
'''redis://:changeme@''',
'''redis://:your-redis-password@''',
# JWT示例密钥
'''(?i)jwt[_-]?secret\s*[:=]\s*["']?(your[_-]?jwt|change|placeholder|secret|example)''',
]
+35
View File
@@ -0,0 +1,35 @@
# vulture.conf - 死代码检测配置
# 仓库: xiaoxia/xiaoxia-saas
# 用途: 检测未使用的函数、变量、导入、类、方法、属性
# 扫描目录(空格分隔)
path = alembic apps packages scripts
# 排除路径(每个路径一行,相对于仓库根目录)
exclude =
tests
test
*/tests
*/test
site-packages
node_modules
migrations
.gitea
docs
scripts/check_*.py
scripts/init_*.py
# 最低置信度 (%)
# 0 = 报告所有可能的未使用代码
# 100 = 只报告确定未使用的代码
# 推荐从 80% 开始,逐步调高
min-confidence = 80
# 输出格式: string, json, yaml
format = text
# 按置信度排序
sort-by-size = False
# 显示置信度
show-uncertain = True
+57
View File
@@ -0,0 +1,57 @@
# vulture_whitelist.py - vulture 白名单文件
# 用途: 列出已知被框架/动态调用的代码,避免误报
# 参考: https://vulture.readthedocs.io/en/stable/whitelists.html
# FastAPI / Starlette 框架自动调用
# FastAPI route handlers (通过装饰器注册,vulture 可能无法识别)
apps.*.main.*
apps.*.api.*
apps.*.routes.*
apps.*.views.*
# SQLAlchemy ORM
# Model 类和字段通过 ORM 框架自动使用
apps.*.models.*
apps.*.schemas.*
packages.*.models.*
# Pydantic models
# Pydantic 字段通过序列化/反序列化使用
apps.*.schemas.*
packages.*.schemas.*
# Alembic migrations
# Migration 函数由 alembic 自动调用
alembic.versions.*.upgrade
alembic.versions.*.downgrade
# Celery tasks
# Task 函数通过 celery worker 调用
apps.*.tasks.*
packages.*.tasks.*
# CLI scripts / entry points
# 脚本通过命令行调用
scripts.*
# 中间件
apps.*.middleware.*
packages.*.middleware.*
# 异常类
apps.*.exceptions.*
packages.*.exceptions.*
# 配置类
apps.*.config.*
packages.*.config.*
# 工具函数(可能被多处间接调用,先白名单,后续清理)
apps.*.utils.*
packages.*.utils.*
apps.*.helpers.*
packages.*.helpers.*
# Dependencies (FastAPI Depends)
apps.*.dependencies.*
packages.*.dependencies.*