Compare commits

...

19 Commits

Author SHA1 Message Date
xiaoxia bbfc45e5f7 ci(P2-8): 优化worker.Dockerfile镜像体积,清理Python字节码和apt缓存 2026-07-09 14:00:23 +08:00
xiaoxia 8a9ee6ab11 ci(P2-8): 优化api.Dockerfile镜像体积,清理Python字节码和apt缓存 2026-07-09 14:00:22 +08:00
xiaoxia bf2649e445 ci(P2-7): 增加Trivy镜像安全扫描(高危漏洞检测,仅告警不阻断) 2026-07-09 13:58:58 +08:00
xiaoxia 48f5f8efa5 ci(P2-5): 测试失败时上传结果产物,pytest输出改为详细模式 2026-07-09 13:58:09 +08:00
xiaoxia e14e007537 ci(P2-2): validate job增加pip依赖缓存,加速重复构建 2026-07-09 13:57:33 +08:00
xiaoxia e52b7bcaac ci(P2-6): daily-check中所有Playwright E2E测试增加重试机制 2026-07-09 13:56:59 +08:00
xiaoxia f58b4bd5d6 ci(P2-6): 所有Playwright E2E测试增加--retries=2重试机制,降低偶发失败率 2026-07-09 13:56:44 +08:00
xiaoxia f2b3e16ca1 ci(P2-4): 增加 concurrency 控制,同一分支重复push自动取消旧构建 2026-07-09 13:56:14 +08:00
xiaoxia 58107ca6ce ci(P2-3): 放宽 validate job timeout 从10分钟到20分钟,避免偶发超时 2026-07-09 13:56:01 +08:00
xiaoxia 69c4ad32dc ci(P1-4): 生产部署增加失败自动回滚+优雅停止,最小化服务中断 2026-07-09 13:55:17 +08:00
xiaoxia 5f3e034f98 ci(P1-3): staging部署增加smoke test失败自动回滚机制 2026-07-09 13:51:50 +08:00
xiaoxia c6d1136ae3 ci(P1-7): 重写 cleanup_old_images.sh:移除过时Registry清理和明文密码,仅保留本地清理
CI/CD Pipeline / Frontend Lint (push) Successful in 1m41s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 10m0s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
2026-07-09 13:50:49 +08:00
xiaoxia 90e5f47975 ci(P1-6): 增强 auto_merge_prs.sh 安全检查:CI状态+标签白名单+Review审批
CI/CD Pipeline / Frontend Lint (push) Successful in 1m34s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 10m0s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
2026-07-09 13:50:04 +08:00
xiaoxia 2a6d3f0d8b ci(P1-5): 统一 build_release_images.sh 的 :dev tag 并推送至 Registry
CI/CD Pipeline / Frontend Lint (push) Successful in 1m53s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 10m0s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
2026-07-09 13:49:20 +08:00
xiaoxia 5772de2ade ci(P1-2): 删除 performance-check 中第一个死代码 step
CI/CD Pipeline / Frontend Lint (push) Successful in 1m41s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 10m0s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
performance-check job 中有两个 step 都在做性能测试:
- 第一个 step(118行)使用 while read 子 shell 方式,变量全部丢失,结果完全没用
- 第二个 step 使用函数方式,正确统计结果

删除无用的第一个 step,减少约一半的请求次数和执行时间。
2026-07-09 13:47:07 +08:00
xiaoxia ed398b8c78 ci(P1-1): 删除重复的 tests.yml 流水线
CI/CD Pipeline / Frontend Lint (push) Successful in 1m41s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 10m0s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
ci-cd.yml 的 validate job 已经覆盖了 tests.yml 的全部功能:
- 单元测试 + 集成测试
- Black 代码格式化检查
- Flake8 静态分析
- 还有安全扫描、迁移检查等 tests.yml 没有的检查

且 ci-cd.yml 已配置 pull_request 触发,PR 时会自动运行。
删除重复流水线,减少维护成本。
2026-07-09 13:46:11 +08:00
xiaoxia bc83a361b1 ci(P0-3): 修复 daily-check.yml 中 job outputs 配置错误
CI/CD Pipeline / Frontend Lint (push) Successful in 1m49s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 10m0s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
- staging-api-tests.outputs.report: steps.smoke → steps.report
  (smoke step 输出的是 api_report,report step 才输出 report)
- staging-e2e.outputs.report: steps.smoke → steps.e2e
  (该 job 的 step id 是 e2e,不是 smoke)
- 修复后 daily-report job 才能正确获取各巡检项状态
2026-07-09 13:45:40 +08:00
xiaoxia b012a7da00 ci(P0-2): 删除调试残留的 test-ssh-secret.yml
CI/CD Pipeline / Frontend Lint (push) Successful in 1m56s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 10m0s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
该文件是用于调试 SSH secret 配置的临时 workflow,
仅在修改自身时触发,无实际业务用途,应从生产 workflow 目录移除。
2026-07-09 13:44:17 +08:00
xiaoxia f678504b97 ci(P0-1): 移除明文测试账号密码,改用 Gitea Secrets
CI/CD Pipeline / Frontend Lint (push) Successful in 2m16s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 10m1s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
- 创建 STAGING_TEST_USER / STAGING_TEST_PASSWORD 两个 secret
- 修改 daily-check.yml 中 5 处明文凭据引用
- 涉及 staging-api-tests 和 performance-check 两个 job
2026-07-09 13:44:05 +08:00
9 changed files with 273 additions and 413 deletions
File diff suppressed because one or more lines are too long
+11 -123
View File
@@ -112,7 +112,7 @@ jobs:
runs-on: saas
timeout-minutes: 10
outputs:
report: ${{ steps.smoke.outputs.report }}
report: ${{ steps.report.outputs.report }}
steps:
- name: Checkout code
@@ -171,8 +171,8 @@ jobs:
docker run --rm \
-e BASE_URL=https://staging-api.xiaoxiajianji.com \
-e WEB_URL=https://staging.xiaoxiajianji.com \
-e TEST_USER=18314979086@163.com \
-e TEST_PASSWORD=Ying1234 \
-e TEST_USER=${{ secrets.STAGING_TEST_USER }} \
-e TEST_PASSWORD=${{ secrets.STAGING_TEST_PASSWORD }} \
-e CLEANUP_ENABLED=1 \
-e PERF_CHECK_ENABLED=1 \
-e PERF_WARN_THRESHOLD_MS=500 \
@@ -213,7 +213,7 @@ jobs:
-v "$PWD:/workspace" \
-w /workspace/apps/web \
git.xiaoxiajianji.com/xiaoxia/base/playwright:v1.45.0-jammy \
sh -lc "npm ci && npx playwright test --reporter=line e2e/test_auth.spec.ts e2e/test_asset.spec.ts e2e/test_project.spec.ts" 2>&1 | tee /tmp/staging-api-e2e.log
sh -lc "npm ci && npx playwright test --reporter=line --retries=2 e2e/test_auth.spec.ts e2e/test_asset.spec.ts e2e/test_project.spec.ts" 2>&1 | tee /tmp/staging-api-e2e.log
EXIT_CODE=${PIPESTATUS[0]}
END_TIME=$(date +%s)
ELAPSED=$((END_TIME - START_TIME))
@@ -249,7 +249,7 @@ jobs:
runs-on: saas
timeout-minutes: 15
outputs:
report: ${{ steps.smoke.outputs.report }}
report: ${{ steps.e2e.outputs.report }}
steps:
- name: Checkout code
@@ -312,7 +312,7 @@ jobs:
-v "$PWD:/workspace" \
-w /workspace/apps/web \
git.xiaoxiajianji.com/xiaoxia/base/playwright:v1.45.0-jammy \
sh -lc 'npm ci && npx playwright test --reporter=line --project=chromium e2e/auth.spec.ts e2e/auth-guard.spec.ts e2e/core-upload.spec.ts e2e/core-generation.spec.ts e2e/core-titles.spec.ts' 2>&1 | tee /tmp/staging-e2e.log
sh -lc 'npm ci && npx playwright test --reporter=line --retries=2 --project=chromium e2e/auth.spec.ts e2e/auth-guard.spec.ts e2e/core-upload.spec.ts e2e/core-generation.spec.ts e2e/core-titles.spec.ts' 2>&1 | tee /tmp/staging-e2e.log
EXIT_CODE=${PIPESTATUS[0]}
END_TIME=$(date +%s)
ELAPSED=$((END_TIME - START_TIME))
@@ -341,124 +341,12 @@ jobs:
report: ${{ steps.report.outputs.report }}
steps:
- name: Run performance baseline checks
id: perf
shell: sh
run: |
set +e
START_TIME=$(date +%s)
echo "=========================================="
echo " 性能基线巡检 - Staging API"
echo " 目标: https://staging-api.xiaoxiajianji.com"
echo "=========================================="
echo ""
TOTAL=0
PASS=0
FAIL=0
WARN=0
WARN_LIST=""
FAIL_LIST=""
# 核心接口配置: 名称|路径|方法|阈值(ms)|失败阈值(ms)
# 核心接口(core): 500ms
# 普通接口(normal): 1000ms
# 重操作接口(heavy): 3000ms
ENDPOINTS="
登录|/api/v1/auth/login|POST|500|3000
获取当前用户|/api/v1/auth/me|GET|500|3000
项目列表|/api/v1/projects|GET|500|3000
素材列表|/api/v1/assets|GET|500|3000
模板列表|/api/v1/templates|GET|500|3000
剪辑计划列表|/api/v1/edit-plans|GET|500|3000
生成任务列表|/api/v1/generation/tasks|GET|500|3000
订阅信息|/api/v1/subscription/current|GET|500|3000
音色列表|/api/v1/voices|GET|1000|5000
健康检查|/health|GET|200|1000
"
# 先登录获取 token
echo "--- 准备: 获取测试 Token ---"
AUTH_RESP=$(curl -s -w "\n%{http_code}" -X POST \
-H "Content-Type: application/json" \
-d '{"email":"18314979086@163.com","password":"Ying1234"}' \
"https://staging-api.xiaoxiajianji.com/api/v1/auth/login" \
--max-time 10 2>&1)
AUTH_CODE=$(echo "$AUTH_RESP" | tail -1)
AUTH_BODY=$(echo "$AUTH_RESP" | sed '$d')
if [ "$AUTH_CODE" = "200" ]; then
TOKEN=$(echo "$AUTH_BODY" | python3 -c "import sys,json; d=json.load(sys.stdin); print(d.get('access_token',''))" 2>/dev/null)
if [ -n "$TOKEN" ]; then
echo "Token 获取成功"
else
echo "Token 解析失败,部分接口可能无法测试"
TOKEN=""
fi
else
echo "登录失败 (HTTP $AUTH_CODE),部分接口将跳过鉴权测试"
TOKEN=""
fi
echo ""
echo "--- 开始性能测试 ---"
echo ""
echo "$ENDPOINTS" | while IFS='|' read -r name path method warn_ms fail_ms; do
[ -z "$name" ] && continue
TOTAL=$((TOTAL + 1))
# 构建 curl 命令
CURL_ARGS="-s -o /dev/null -w '%{http_code} %{time_total}' --max-time 30"
if [ "$method" = "POST" ]; then
CURL_ARGS="$CURL_ARGS -X POST -H 'Content-Type: application/json' -d '{\"email\":\"18314979086@163.com\",\"password\":\"Ying1234\"}'"
fi
if [ -n "$TOKEN" ] && [ "$name" != "健康检查" ]; then
CURL_ARGS="$CURL_ARGS -H 'Authorization: Bearer $TOKEN'"
fi
# 执行请求
RESP=$(eval curl $CURL_ARGS "https://staging-api.xiaoxiajianji.com${path}" 2>&1)
HTTP_CODE=$(echo "$RESP" | awk '{print $1}')
TIME_TOTAL=$(echo "$RESP" | awk '{print $2}')
ELAPSED_MS=$(python3 -c "print(int(float('${TIME_TOTAL:-0}') * 1000))" 2>/dev/null || echo "0")
if [ "$HTTP_CODE" -ge 500 ] 2>/dev/null; then
FAIL=$((FAIL + 1))
FAIL_LIST="$FAIL_LIST\n ❌ $name - HTTP $HTTP_CODE (${ELAPSED_MS}ms)"
echo "❌ $name - HTTP $HTTP_CODE - ${ELAPSED_MS}ms (FAIL)"
elif [ "$ELAPSED_MS" -ge "$fail_ms" ] 2>/dev/null; then
FAIL=$((FAIL + 1))
FAIL_LIST="$FAIL_LIST\n ❌ $name - ${ELAPSED_MS}ms > ${fail_ms}ms"
echo "❌ $name - ${ELAPSED_MS}ms > ${fail_ms}ms (FAIL)"
elif [ "$ELAPSED_MS" -ge "$warn_ms" ] 2>/dev/null; then
WARN=$((WARN + 1))
WARN_LIST="$WARN_LIST\n ⚠️ $name - ${ELAPSED_MS}ms > ${warn_ms}ms"
echo "⚠️ $name - ${ELAPSED_MS}ms (WARN, threshold: ${warn_ms}ms)"
PASS=$((PASS + 1))
else
PASS=$((PASS + 1))
echo "✅ $name - ${ELAPSED_MS}ms (OK, threshold: ${warn_ms}ms)"
fi
done
# 由于 while 在子 shell 中执行,用文件传递结果
# 重新跑一次用文件计数方式
echo ""
echo "--- 汇总性能数据 ---"
END_TIME=$(date +%s)
ELAPSED=$((END_TIME - START_TIME))
echo ""
echo "========== 性能基线巡检报告 =========="
echo "环境: https://staging-api.xiaoxiajianji.com"
echo "耗时: ${ELAPSED}s"
echo "======================================"
- name: Generate performance report
id: report
shell: sh
env:
STAGING_TEST_USER: ${{ secrets.STAGING_TEST_USER }}
STAGING_TEST_PASSWORD: ${{ secrets.STAGING_TEST_PASSWORD }}
run: |
set +e
echo ""
@@ -476,7 +364,7 @@ jobs:
# 先登录获取 token
AUTH_RESP=$(curl -s -w "\n%{http_code}" -X POST \
-H "Content-Type: application/json" \
-d '{"email":"18314979086@163.com","password":"Ying1234"}' \
-d "{"email":"${STAGING_TEST_USER}","password":"${STAGING_TEST_PASSWORD}"}" \
"https://staging-api.xiaoxiajianji.com/api/v1/auth/login" \
--max-time 10 2>&1)
AUTH_CODE=$(echo "$AUTH_RESP" | tail -1)
@@ -492,7 +380,7 @@ jobs:
local CURL_ARGS="-s -o /dev/null -w '%{http_code} %{time_total}' --max-time 30"
if [ "$method" = "POST" ]; then
CURL_ARGS="$CURL_ARGS -X POST -H 'Content-Type: application/json' -d '{\"email\":\"18314979086@163.com\",\"password\":\"Ying1234\"}'"
CURL_ARGS="$CURL_ARGS -X POST -H 'Content-Type: application/json' -d '{\"email\":\"${STAGING_TEST_USER}\",\"password\":\"${STAGING_TEST_PASSWORD}\"}'"
fi
if [ -n "$TOKEN" ] && [ "$name" != "健康检查" ]; then
CURL_ARGS="$CURL_ARGS -H 'Authorization: Bearer $TOKEN'"
-69
View File
@@ -1,69 +0,0 @@
name: Test SSH Secret
on:
push:
branches: [develop]
paths:
- '.gitea/workflows/test-ssh-secret.yml'
jobs:
test-ssh:
runs-on: ubuntu-22.04
steps:
- name: Install SSH client
run: |
which ssh || (apt-get update && apt-get install -y openssh-client)
ssh -V
- name: Debug environment
run: |
echo "=== Environment ==="
echo "Runner hostname: $(hostname)"
echo "Runner IP: $(hostname -i || echo 'unknown')"
echo "Current user: $(whoami)"
echo "=== Secrets check ==="
if [ -n "$STAGING_SSH_HOST" ]; then
echo "STAGING_SSH_HOST: [SET] value_length=${#STAGING_SSH_HOST}"
else
echo "STAGING_SSH_HOST: [EMPTY]"
fi
if [ -n "$STAGING_SSH_USER" ]; then
echo "STAGING_SSH_USER: [SET] value_length=${#STAGING_SSH_USER}"
else
echo "STAGING_SSH_USER: [EMPTY]"
fi
if [ -n "$STAGING_SSH_KEY" ]; then
echo "STAGING_SSH_KEY: [SET] value_length=${#STAGING_SSH_KEY}"
else
echo "STAGING_SSH_KEY: [EMPTY]"
fi
env:
STAGING_SSH_HOST: ${{ secrets.STAGING_SSH_HOST }}
STAGING_SSH_USER: ${{ secrets.STAGING_SSH_USER }}
STAGING_SSH_KEY: ${{ secrets.STAGING_SSH_KEY }}
- name: Setup SSH key
run: |
mkdir -p ~/.ssh
chmod 700 ~/.ssh
echo "$STAGING_SSH_KEY" > ~/.ssh/id_ed25519
chmod 600 ~/.ssh/id_ed25519
ssh-keygen -y -f ~/.ssh/id_ed25519 > ~/.ssh/id_ed25519.pub 2>/dev/null || echo "No public key generated"
echo "=== SSH Key fingerprint ==="
ssh-keygen -lf ~/.ssh/id_ed25519 || echo "Key fingerprint failed"
env:
STAGING_SSH_KEY: ${{ secrets.STAGING_SSH_KEY }}
- name: Test SSH connection
run: |
echo "Attempting SSH connection to $STAGING_SSH_HOST..."
ssh -i ~/.ssh/id_ed25519 \
-o StrictHostKeyChecking=no \
-o UserKnownHostsFile=/dev/null \
-o ConnectTimeout=10 \
-o BatchMode=yes \
-v \
$STAGING_SSH_USER@$STAGING_SSH_HOST "echo 'SSH_CONNECTION_SUCCESS' && hostname && whoami"
echo "=== SSH Test Complete ==="
env:
STAGING_SSH_HOST: ${{ secrets.STAGING_SSH_HOST }}
STAGING_SSH_USER: ${{ secrets.STAGING_SSH_USER }}
-163
View File
@@ -1,163 +0,0 @@
name: Tests
on:
pull_request:
branches: [ main ]
jobs:
test:
runs-on: runtime-builder
steps:
- name: Checkout code
shell: sh
run: |
set -eu
python - <<'PY'
import io
import os
import tarfile
import time
import urllib.error
import urllib.request
url = f"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz"
request = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['GITHUB_TOKEN']}"})
# Retry up to 5 times with backoff for transient 5xx errors
last_err = None
for attempt in range(5):
try:
with urllib.request.urlopen(request, timeout=120) as response:
archive = response.read()
break
except urllib.error.HTTPError as e:
last_err = e
if e.code >= 500 and attempt < 4:
wait = 2 ** attempt
print(f"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
except Exception as e:
last_err = e
if attempt < 4:
wait = 2 ** attempt
print(f"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
else:
raise last_err
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
PY
- name: Show Python version
shell: sh
run: |
set -eu
python --version
python -m pip --version
- name: Install dependencies
shell: sh
run: |
set -eu
python -m pip install --upgrade pip -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
python -m pip install -r requirements.txt -r requirements-dev.txt -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
- name: Run unit tests
shell: sh
run: |
set -eu
PYTHONPATH="$PWD/apps/api:$PWD" python -m pytest tests/unit -q
- name: Run integration tests
shell: sh
run: |
set -eu
PYTHONPATH="$PWD/apps/api:$PWD" python -m pytest tests/integration -q --timeout=60 -x
lint:
runs-on: runtime-builder
steps:
- name: Checkout code
shell: sh
run: |
set -eu
python - <<'PY'
import io
import os
import tarfile
import time
import urllib.error
import urllib.request
url = f"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz"
request = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['GITHUB_TOKEN']}"})
# Retry up to 5 times with backoff for transient 5xx errors
last_err = None
for attempt in range(5):
try:
with urllib.request.urlopen(request, timeout=120) as response:
archive = response.read()
break
except urllib.error.HTTPError as e:
last_err = e
if e.code >= 500 and attempt < 4:
wait = 2 ** attempt
print(f"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
except Exception as e:
last_err = e
if attempt < 4:
wait = 2 ** attempt
print(f"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
else:
raise last_err
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
PY
- name: Install dependencies
shell: sh
run: |
set -eu
python -m pip install --upgrade pip -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
python -m pip install -r requirements.txt -r requirements-dev.txt -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
- name: Run Black (check only)
shell: sh
run: |
set -eu
python -m black --check alembic apps packages tests scripts
- name: Run Flake8
shell: sh
run: |
set -eu
python -m flake8 apps packages tests --count --statistics
+3
View File
@@ -39,6 +39,9 @@ COPY migrations/ /app/migrations/
COPY alembic/ /app/alembic/
COPY scripts/ /app/scripts/
# 清理不需要的文件,减小镜像体积
RUN find /app -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || true && find /opt/venv -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || true && find /opt/venv -name "*.pyc" -delete 2>/dev/null || true && find /opt/venv -name "*.pyo" -delete 2>/dev/null || true && rm -rf /opt/venv/share/doc /opt/venv/share/man 2>/dev/null || true && apt-get clean && rm -rf /var/lib/apt/lists/*
# 设置环境变量
ENV PATH="/opt/venv/bin:$PATH"
ENV PYTHONPATH=/app
+3
View File
@@ -48,6 +48,9 @@ COPY packages/ /app/packages/
COPY alembic.ini /app/alembic.ini
COPY migrations/ /app/migrations/
# 清理不需要的文件,减小镜像体积
RUN find /app -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || true && find /opt/venv -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || true && find /opt/venv -name "*.pyc" -delete 2>/dev/null || true && find /opt/venv -name "*.pyo" -delete 2>/dev/null || true && rm -rf /opt/venv/share/doc /opt/venv/share/man 2>/dev/null || true && apt-get clean && rm -rf /var/lib/apt/lists/*
# 设置 Python 路径
ENV PATH="/opt/venv/bin:$PATH"
ENV PYTHONPATH=/app
+118 -14
View File
@@ -1,43 +1,147 @@
#!/bin/bash
# 自动合并通过 CI 检查的 PR
# 自动合并通过 CI 检查且打了 auto-merge 标签的 PR
# 用法: ./scripts/auto_merge_prs.sh [target_branch]
# 安全规则:
# 1. PR 必须打有 auto-merge 标签(白名单)
# 2. 所有 CI 检查必须通过
# 3. 必须至少有 1 个 review approve(可通过 REQUIRE_APPROVAL=0 关闭)
# 4. mergeable 状态为 true
set -eu
GITEA_API="https://git.xiaoxiajianji.com/api/v1"
TOKEN="${GITEA_API_TOKEN:?Please set GITEA_API_TOKEN environment variable}"
REPO="xiaoxia/xiaoxia-saas"
TARGET_BRANCH="${1:-develop}"
REQUIRE_APPROVAL="${REQUIRE_APPROVAL:-1}"
AUTO_MERGE_LABEL="${AUTO_MERGE_LABEL:-auto-merge}"
echo "=== Checking open PRs targeting $TARGET_BRANCH ==="
echo "=== Auto-merge check for PRs targeting $TARGET_BRANCH ==="
echo " Require approval: $REQUIRE_APPROVAL"
echo " Required label: $AUTO_MERGE_LABEL"
echo ""
# 获取所有 open PR
# 获取所有 open PR(包含标签信息)
PRS=$(curl -s -H "Authorization: token $TOKEN" \
"$GITEA_API/repos/$REPO/pulls?state=open&labels=0" | python3 -c "
"$GITEA_API/repos/$REPO/pulls?state=open" | python3 -c "
import json, sys
data = json.load(sys.stdin)
for pr in data:
if pr.get('base', {}).get('ref') == '$TARGET_BRANCH':
if pr.get('mergeable', False):
print(f\"{pr['number']}|{pr['title']}|{pr.get('mergeable', 'unknown')}\")
if pr.get('base', {}).get('ref') != '$TARGET_BRANCH':
continue
number = pr['number']
title = pr['title']
mergeable = pr.get('mergeable', False)
labels = [l['name'] for l in pr.get('labels', [])]
head_sha = pr.get('head', {}).get('sha', '')
print(f'{number}|{title}|{mergeable}|{head_sha}|{\",\".join(labels)}')
")
if [ -z "$PRS" ]; then
echo "No mergeable PRs found for $TARGET_BRANCH"
echo "No open PRs found for $TARGET_BRANCH"
exit 0
fi
echo "$PRS" | while IFS='|' read -r number title mergeable; do
echo "Merging PR #$number: $title"
merged_count=0
skipped_count=0
echo "$PRS" | while IFS='|' read -r number title mergeable head_sha labels; do
echo "--- PR #$number: $title ---"
echo " mergeable: $mergeable"
echo " labels: $labels"
echo " head_sha: ${head_sha:0:12}"
# 检查 1: mergeable 状态
if [ "$mergeable" != "True" ] && [ "$mergeable" != "true" ]; then
echo " ⏭️ Skip: not mergeable"
skipped_count=$((skipped_count + 1))
continue
fi
# 检查 2: auto-merge 标签白名单
has_label=$(echo "$labels" | tr ',' '\n' | grep -qx "$AUTO_MERGE_LABEL" && echo "yes" || echo "no")
if [ "$has_label" != "yes" ]; then
echo " ⏭️ Skip: missing '$AUTO_MERGE_LABEL' label"
skipped_count=$((skipped_count + 1))
continue
fi
# 检查 3: CI 状态检查(所有 check 必须成功)
if [ -n "$head_sha" ]; then
ci_result=$(curl -s -H "Authorization: token $TOKEN" \
"$GITEA_API/repos/$REPO/commits/$head_sha/status" | python3 -c "
import json, sys
data = json.load(sys.stdin)
status = data.get('state', 'unknown')
statuses = data.get('statuses', [])
# 统计各状态
success = sum(1 for s in statuses if s.get('state') == 'success')
pending = sum(1 for s in statuses if s.get('state') == 'pending')
failure = sum(1 for s in statuses if s.get('state') in ('failure', 'error'))
total = len(statuses)
print(f'{status}|{total}|{success}|{pending}|{failure}')
")
ci_state=$(echo "$ci_result" | cut -d'|' -f1)
ci_total=$(echo "$ci_result" | cut -d'|' -f2)
ci_success=$(echo "$ci_result" | cut -d'|' -f3)
ci_pending=$(echo "$ci_result" | cut -d'|' -f4)
ci_failure=$(echo "$ci_result" | cut -d'|' -f5)
echo " CI status: $ci_state ($ci_success/$ci_total passed, $ci_pending pending, $ci_failure failed)"
if [ "$ci_state" != "success" ]; then
echo " ⏭️ Skip: CI not passing (state=$ci_state)"
skipped_count=$((skipped_count + 1))
continue
fi
else
echo " ⚠️ No head SHA found, skipping CI check"
fi
# 检查 4: Review approve 检查
if [ "$REQUIRE_APPROVAL" = "1" ]; then
review_result=$(curl -s -H "Authorization: token $TOKEN" \
"$GITEA_API/repos/$REPO/pulls/$number/reviews" | python3 -c "
import json, sys
data = json.load(sys.stdin)
approved = sum(1 for r in data if r.get('state') == 'APPROVED')
changes_req = sum(1 for r in data if r.get('state') == 'CHANGES_REQUESTED')
print(f'{approved}|{changes_req}')
")
approved=$(echo "$review_result" | cut -d'|' -f1)
changes_req=$(echo "$review_result" | cut -d'|' -f2)
echo " Reviews: $approved approved, $changes_req changes requested"
if [ "$approved" -lt 1 ]; then
echo " ⏭️ Skip: no approval yet"
skipped_count=$((skipped_count + 1))
continue
fi
if [ "$changes_req" -gt 0 ]; then
echo " ⏭️ Skip: has changes requested"
skipped_count=$((skipped_count + 1))
continue
fi
fi
# 全部检查通过,执行合并
echo " ✅ All checks passed, merging..."
RESULT=$(curl -s -X POST \
-H "Authorization: token $TOKEN" \
-H "Content-Type: application/json" \
"$GITEA_API/repos/$REPO/pulls/$number/merge" \
-d '{\"merge_method\": \"merge\"}')
-d '{"merge_method": "squash"}')
if echo "$RESULT" | python3 -c "import json,sys; d=json.load(sys.stdin); sys.exit(0 if 'id' in d else 1)"; then
echo " ✅ PR #$number merged successfully"
echo " ✅ PR #$number merged successfully (squash)"
merged_count=$((merged_count + 1))
else
echo " ❌ PR #$number failed: $RESULT"
error_msg=$(echo "$RESULT" | python3 -c "import json,sys; d=json.load(sys.stdin); print(d.get('message', 'unknown error'))" 2>/dev/null || echo "$RESULT")
echo " ❌ PR #$number failed: $error_msg"
fi
echo ""
done
echo ""
echo "=== Done ==="
echo " Merged: $merged_count"
echo " Skipped: $skipped_count"
+14 -3
View File
@@ -35,6 +35,7 @@ CACHE_TAG="${CACHE_TAG:-release}"
API_IMAGE="xiaoxia-saas-api:$VERSION"
WORKER_IMAGE="xiaoxia-saas-worker:$VERSION"
WEB_IMAGE="xiaoxia-saas-web:$VERSION"
WEB_LATEST="xiaoxia-saas-web:dev"
API_LATEST="xiaoxia-saas-api:dev"
WORKER_LATEST="xiaoxia-saas-worker:dev"
@@ -96,14 +97,14 @@ if [ "$USE_CACHE" -eq 1 ]; then
--cache-to "type=registry,ref=${CACHE_REGISTRY}/web-cache:${CACHE_TAG},mode=max" \
-f infra/docker/web-artifact.Dockerfile \
--build-arg "NGINX_CONF=$NGINX_CONF_FILE" \
-t "$WEB_IMAGE" \
-t "$WEB_IMAGE" -t "$WEB_LATEST" \
--load \
.
else
docker build --pull=false \
-f infra/docker/web-artifact.Dockerfile \
--build-arg "NGINX_CONF=$NGINX_CONF_FILE" \
-t "$WEB_IMAGE" \
-t "$WEB_IMAGE" -t "$WEB_LATEST" \
.
fi
@@ -116,7 +117,17 @@ if [ "$USE_PUSH" -eq 1 ]; then
docker push "$REGISTRY_API"
docker push "$REGISTRY_WORKER"
docker push "$REGISTRY_WEB"
echo "All images pushed to $REGISTRY"
# 同时推送 :dev tag(用于快速拉取最新开发版)
REGISTRY_API_DEV="${REGISTRY}/xiaoxia-saas-api:dev"
REGISTRY_WORKER_DEV="${REGISTRY}/xiaoxia-saas-worker:dev"
REGISTRY_WEB_DEV="${REGISTRY}/xiaoxia-saas-web:dev"
docker tag "$API_LATEST" "$REGISTRY_API_DEV"
docker tag "$WORKER_LATEST" "$REGISTRY_WORKER_DEV"
docker tag "$WEB_LATEST" "$REGISTRY_WEB_DEV"
docker push "$REGISTRY_API_DEV"
docker push "$REGISTRY_WORKER_DEV"
docker push "$REGISTRY_WEB_DEV"
echo "All images + :dev tag pushed to $REGISTRY"
else
echo "Registry push skipped (no auth token available)"
fi
+29 -30
View File
@@ -1,49 +1,42 @@
#!/bin/sh
# cleanup_old_images.sh
# 清理构建服务器上的旧 Docker 镜像和 Registry 旧版本
# 保留最近 KEEP_VERSIONS 个版本(默认 2
# 清理构建服务器上的旧 Docker 本地镜像
# 保留最近 KEEP_VERSIONS 个版本(默认 5
# 在 CI 构建完成后调用,防止磁盘空间耗尽
#
# 注意:Registry 侧的旧镜像清理已由 Gitea Package 清理规则接管,
# 本脚本仅负责构建服务器本地镜像清理。
set -eu
KEEP_VERSIONS="${KEEP_VERSIONS:-2}"
REGISTRY_HOST="${REGISTRY_HOST:-172.30.18.198:5000}"
KEEP_VERSIONS="${KEEP_VERSIONS:-5}"
SERVICES="xiaoxia-saas-api xiaoxia-saas-worker xiaoxia-saas-web"
ACCEPT="application/vnd.docker.distribution.manifest.v2+json, application/vnd.oci.image.manifest.v1+json"
echo "=== Docker Image Cleanup ==="
echo "=== Local Docker Image Cleanup ==="
echo "Keeping last ${KEEP_VERSIONS} versions per service"
echo ""
for svc in $SERVICES; do
# 收集所有 v0.N.N 格式的版本号(去重,按版本号排序)
versions=$(docker images --format "{{.Repository}}:{{.Tag}}" | \
# 收集所有版本标签(去重,按版本号排序)
versions=$(docker images --format "{{.Repository}}:{{.Tag}}" 2>/dev/null | \
grep "${svc}" | \
grep -E "v[0-9]+\.[0-9]+\.[0-9]+" | \
sed -E "s/.*:v([0-9]+\.[0-9]+\.[0-9]+).*/v\1/" | \
grep -E ":(v[0-9]+\.[0-9]+\.[0-9]+|[a-f0-9]{7,})$" | \
sed -E "s/.*:([^:]+)$/\1/" | \
sort -t. -k1,1V -k2,2n -k3,3n | \
uniq)
total=$(echo "$versions" | grep -c "^v" || true)
total=$(echo "$versions" | grep -c . || true)
if [ "$total" -gt "$KEEP_VERSIONS" ]; then
remove_count=$((total - KEEP_VERSIONS))
to_remove=$(echo "$versions" | head -n "$remove_count")
echo "[Registry] Cleaning old blobs for ${svc}..."
echo "[${svc}] ${total} versions found, removing ${remove_count} oldest..."
for ver in $to_remove; do
# 仓库名与服务名一致(如 xiaoxia-saas-api),不再裁剪前缀
manifest_url="http://admin:Xiaoxia2026@localhost:5000/v2/${svc}/manifests/${ver}"
digest=$(curl -s -D- -H "Accept: ${ACCEPT}" "$manifest_url" | grep -i "^docker-content-digest:" | tr -d "\r" | awk "{print \$2}")
if [ -n "$digest" ]; then
curl -s -X DELETE -H "Accept: ${ACCEPT}" "http://admin:Xiaoxia2026@localhost:5000/v2/${svc}/manifests/${digest}" > /dev/null 2>&1 || true
echo " Deleted registry tag: ${svc}:${ver}"
fi
done
echo "[Local] Removing old local images for ${svc}..."
for ver in $to_remove; do
docker rmi "${svc}:${ver}" 2>/dev/null && echo " Removed local: ${svc}:${ver}" || true
docker rmi "${REGISTRY_HOST}/${svc}:${ver}" 2>/dev/null && echo " Removed registry-ref: ${REGISTRY_HOST}/${svc}:${ver}" || true
# 清理各种前缀的镜像
docker rmi "${svc}:${ver}" 2>/dev/null && echo " Removed: ${svc}:${ver}" || true
docker rmi "git.xiaoxiajianji.com/xiaoxia/xiaoxia-saas/${svc}:${ver}" 2>/dev/null && \
echo " Removed: git.xiaoxiajianji.com/.../${svc}:${ver}" || true
done
else
echo "[${svc}] ${total} version(s) found, within keep limit (${KEEP_VERSIONS})"
@@ -53,11 +46,17 @@ done
# 清理悬空镜像(构建中间层)
echo "=== Pruning dangling images ==="
pruned=$(docker image prune -f 2>&1)
echo "$pruned" | tail -1
docker image prune -f 2>&1 | tail -1
# 清理未使用的构建缓存
echo ""
echo "=== Pruning build cache ==="
docker builder prune -f 2>&1 | tail -1 || true
echo ""
echo "=== Cleanup complete ==="
echo "Current images:"
docker images --format "table {{.Repository}}\t{{.Tag}}\t{{.Size}}" | grep -E "(xiaoxia|REPOSITORY)" || true
echo "Current xiaoxia images:"
docker images --format "table {{.Repository}}\t{{.Tag}}\t{{.Size}}" 2>/dev/null | grep -E "(xiaoxia|REPOSITORY)" || echo " (none)"
echo ""
echo "Disk usage:"
df -h / | tail -1