Compare commits

..

8 Commits

Author SHA1 Message Date
xiaoxia 8ca2ffe272 fix(ci): pr_auto_scan.py black 120列格式化
Auto Merge PRs (main) / Auto Merge on CI Green + Approved (main) (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Failing after 15s
Auto Approve CI PRs / Auto Approve on CI Green (pull_request) Successful in 35s
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Failing after 1m26s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
修复Code Quality job因脚本自身格式问题失败的问题
2026-07-23 17:53:30 +08:00
xiaoxia dfb2feef8a fix(ci): main??auto-merge/approve????Tests/test??
CI/CD Pipeline / Validate Code Quality And Tests (push) Successful in 2m0s
CI/CD Pipeline / Frontend Lint (push) Successful in 2m10s
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Failing after 3s
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
- auto-merge.yml: main????Tests/test,?3???????
- auto-approve.yml: main??????????Tests/test
- ??#464???Tests/test???????????
2026-07-17 14:04:23 +08:00
auto-approve-bot b3ef7bb041 Merge pull request 'feat(ci): main分支auto-merge从定时改为即时合并' (#464) from ci/main-auto-merge-instant into main
CI/CD Pipeline / Validate Code Quality And Tests (push) Successful in 2m10s
CI/CD Pipeline / Frontend Lint (push) Successful in 2m12s
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Failing after 4s
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
2026-07-17 13:27:18 +08:00
xiaoxia a1a272b833 feat(ci): 添加auto-approve到main分支并适配main门禁规则
Tests / lint (pull_request) Successful in 6s
Tests / test (pull_request) Failing after 29s
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Successful in 2m23s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 2m46s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
Auto Approve CI PRs / Auto Approve on CI Green (pull_request) Successful in 2m54s
Auto Merge PRs (main) / Auto Merge on CI Green + Approved (main) (pull_request) Successful in 3m0s
2026-07-17 13:24:21 +08:00
xiaoxia 728db0faf8 chore: empty commit to re-trigger CI for auto-merge verification
Tests / lint (pull_request) Successful in 9s
Tests / test (pull_request) Failing after 21s
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Successful in 1m32s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 1m44s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
Auto Merge PRs (main) / Auto Merge on CI Green + Approved (main) (pull_request) Successful in 20m36s
2026-07-17 13:06:40 +08:00
xiaoxia 7e5e412f7f feat(ci): main分支auto-merge从定时改为即时合并
Tests / lint (pull_request) Failing after 9s
Tests / test (pull_request) Failing after 28s
Auto Merge PRs (main) / Auto Merge on CI Green + Approved (main) (pull_request) Failing after 30s
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Successful in 1m53s
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 2m29s
CI/CD Pipeline / Build Production Runtime Images (pull_request) Has been skipped
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
- 触发方式:pull_request事件(CI状态变更时)
- 合并条件:2门禁全绿 + 至少1个APPROVED + 无冲突 + 非草稿
- 安全措施:幂等保护、合并失败留评论、只合main
- 新增check_ci_status.py和check_pr_approval.py辅助脚本
2026-07-17 12:37:24 +08:00
xiaoxia df08161630 fix(ci): 修复auto-merge缺少GITEA_API_TOKEN环境变量
CI/CD Pipeline / Frontend Lint (push) Successful in 58s
CI/CD Pipeline / Validate Code Quality And Tests (push) Successful in 1m36s
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Failing after 7s
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
1. 给auto-merge.yml的两个merge step添加GITEA_API_TOKEN环境变量,复用REVIEW_GITEA_TOKEN secret
2. 修复tests.yml的checkout步骤GITHUB_TOKEN和Python命令
3. 给main分支ci-cd.yml的单元测试增加Redis服务,修复Celery相关测试失败
2026-07-17 09:02:33 +08:00
xiaoxia 0c9375ff32 ci: remove temporary debug workflow - test-ssh-secret.yml
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 36s
CI/CD Pipeline / Frontend Lint (push) Failing after 0s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
Fix Flake8 E741 for PR232 / fix-flake8 (push) Failing after 3h4m59s
2026-07-12 10:25:12 +08:00
8 changed files with 812 additions and 672 deletions
+165
View File
@@ -0,0 +1,165 @@
name: Auto Approve CI PRs
on:
pull_request:
types: [synchronize, opened, ready_for_review]
jobs:
auto-approve:
name: Auto Approve on CI Green
runs-on: ci-l1
if: github.event_name == 'pull_request' && !github.event.pull_request.draft
timeout-minutes: 20
steps:
- name: Checkout code
uses: actions/checkout@v3
- name: Auto approve when CI passes
shell: bash
env:
GITHUB_TOKEN: ${{ github.token }}
REVIEW_TOKEN: ${{ secrets.REVIEW_GITEA_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
run: |
set -eu
echo "PR #${PR_NUMBER} - 检查CI状态并自动审批"
# 检查是否纯前端改动
API_URL="${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/files?limit=300"
FILES=$(curl -s -H "Authorization: token ${GITHUB_TOKEN}" "$API_URL" | python3 -c "import sys,json; [print(f['filename']) for f in json.load(sys.stdin)]")
FRONTEND_COUNT=$(echo "$FILES" | grep -c '^apps/web/' || true)
BACKEND_COUNT=$(echo "$FILES" | grep -cv '^apps/web/' || true)
TOTAL=$(echo "$FILES" | grep -cv '^$' || true)
echo "变更文件: ${TOTAL} 个 (前端: ${FRONTEND_COUNT}, 后端/公共: ${BACKEND_COUNT})"
if [ "$BACKEND_COUNT" = "0" ] && [ "$FRONTEND_COUNT" -gt "0" ]; then
SKIP_BACKEND=true
echo "✅ 纯前端改动,只检查Frontend Lint"
else
SKIP_BACKEND=false
echo "🔧 包含后端/公共变更,检查全部CI"
fi
# 定义需要检查的context
# 根据目标分支决定检查哪些门禁
TARGET_BRANCH="${GITHUB_BASE_REF}"
echo "目标分支: ${TARGET_BRANCH}"
if [ "$SKIP_BACKEND" = "true" ]; then
CONTEXTS=("CI/CD Pipeline / Frontend Lint (pull_request)")
elif [ "$TARGET_BRANCH" = "main" ]; then
# main分支只检查required statuses: Validate + Frontend Lint
# 不检查Tests/test(不是required门禁)
CONTEXTS=(
"CI/CD Pipeline / Validate Code Quality And Tests (pull_request)"
"CI/CD Pipeline / Frontend Lint (pull_request)"
"Tests / test (pull_request)"
)
else
CONTEXTS=(
"CI/CD Pipeline / Validate Code Quality And Tests (pull_request)"
"CI/CD Pipeline / Unit Tests (pull_request)"
"CI/CD Pipeline / Frontend Lint (pull_request)"
)
fi
echo "需要通过的CI检查: ${#CONTEXTS[@]} 项"
for ctx in "${CONTEXTS[@]}"; do
echo " - $ctx"
done
echo
# 轮询等待,最多20分钟(120次x10秒)
for attempt in $(seq 1 120); do
ALL_SUCCESS=true
ANY_FAILED=false
echo "--- 第${attempt}次检查 ($(date '+%H:%M:%S')) ---"
# 调用辅助脚本检查每个context状态
for ctx in "${CONTEXTS[@]}"; do
STATE=$(python3 scripts/check_ci_status.py "$GITHUB_TOKEN" "$GITHUB_REPOSITORY" "$GITHUB_SHA" "$ctx")
echo " $ctx: $STATE"
if [ "$STATE" != "success" ]; then
ALL_SUCCESS=false
fi
if [ "$STATE" = "failure" ] || [ "$STATE" = "error" ]; then
ANY_FAILED=true
fi
done
if [ "$ALL_SUCCESS" = "true" ]; then
echo
echo "✅ 所有CI检查通过,自动审批 PR #${PR_NUMBER}"
# 检查是否已有审批(任何用户的APPROVED都算,避免重复审批)
EXISTING=$(curl -s -H "Authorization: token ${REVIEW_TOKEN}" \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/reviews" \
| python3 -c "import sys,json; reviews=json.load(sys.stdin); print('yes' if any(r.get('state')=='APPROVED' for r in reviews) else 'no')")
if [ "$EXISTING" = "yes" ]; then
echo "️ PR #${PR_NUMBER} 已有审批,跳过"
exit 0
fi
# 第一步:创建PENDING reviewGitea API需要先创建再提交)
echo "创建review..."
REVIEW_CREATE=$(curl -s -X POST \
-H "Authorization: token ${REVIEW_TOKEN}" \
-H "Content-Type: application/json" \
-d '{"event": "PENDING", "body": "CI全绿,自动审批通过。"}' \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/reviews")
REVIEW_ID=$(echo "$REVIEW_CREATE" | python3 -c "import sys,json; print(json.load(sys.stdin).get('id',''))")
REVIEW_STATE=$(echo "$REVIEW_CREATE" | python3 -c "import sys,json; print(json.load(sys.stdin).get('state',''))")
echo "创建结果: id=$REVIEW_ID state=$REVIEW_STATE"
if [ -z "$REVIEW_ID" ]; then
echo "❌ 创建review失败"
echo "$REVIEW_CREATE"
exit 1
fi
# 如果已经是APPROVED就不用再submit了(兼容不同Gitea版本)
if [ "$REVIEW_STATE" = "APPROVED" ]; then
echo "✅ 自动审批成功(直接创建为APPROVED"
exit 0
fi
# 第二步:submit review为APPROVED
echo "提交review审批..."
SUBMIT_CODE=$(curl -s -o /tmp/submit_resp.json -w "%{http_code}" \
-X POST \
-H "Authorization: token ${REVIEW_TOKEN}" \
-H "Content-Type: application/json" \
-d '{"event": "APPROVED", "body": "CI全绿,自动审批通过。"}' \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/reviews/${REVIEW_ID}")
echo "提交API HTTP状态: $SUBMIT_CODE"
cat /tmp/submit_resp.json 2>/dev/null || true
echo
if [ "$SUBMIT_CODE" = "200" ] || [ "$SUBMIT_CODE" = "201" ]; then
FINAL_STATE=$(python3 -c "import json; print(json.load(open('/tmp/submit_resp.json')).get('state',''))" 2>/dev/null || echo "?")
echo "✅ 自动审批成功 (state: $FINAL_STATE)"
exit 0
else
echo "❌ 提交审批失败"
exit 1
fi
fi
if [ "$ANY_FAILED" = "true" ]; then
echo
echo "❌ CI检查有失败项,不自动审批"
exit 0
fi
sleep 10
done
echo
echo "⏰ 等待超时(20分钟),CI尚未全部完成"
exit 0
+122 -12
View File
@@ -1,21 +1,131 @@
name: Auto Merge PRs
name: Auto Merge PRs (main)
on:
schedule:
- cron: '0 */6 * * *'
workflow_dispatch:
pull_request:
types: [synchronize, opened, ready_for_review, review_requested]
jobs:
auto-merge:
name: Auto Merge on CI Green + Approved (main)
runs-on: ubuntu-latest
if: github.event_name == 'pull_request' && !github.event.pull_request.draft && github.event.pull_request.base.ref == 'main'
timeout-minutes: 30
steps:
- name: Checkout
- name: Checkout code
uses: actions/checkout@v3
- name: Auto merge develop PRs
- name: Auto merge when CI passes and approved
shell: bash
env:
GITHUB_TOKEN: ${{ github.token }}
MERGE_TOKEN: ${{ secrets.REVIEW_GITEA_TOKEN }}
PR_NUMBER: ${{ github.event.pull_request.number }}
BASE_REF: ${{ github.event.pull_request.base.ref }}
run: |
bash scripts/auto_merge_prs.sh develop
- name: Auto merge main PRs (release only)
run: |
bash scripts/auto_merge_prs.sh main
set -eu
echo "PR #${PR_NUMBER} - 检查CI状态+审批并自动合并到${BASE_REF}"
echo
# 只合main分支
if [ "$BASE_REF" != "main" ]; then
echo "Skip: 目标分支不是main"
exit 0
fi
# main分支门禁:Validate + Frontend Lint
CONTEXTS=(
"CI/CD Pipeline / Validate Code Quality And Tests (pull_request)"
"Tests / test (pull_request)"
"CI/CD Pipeline / Frontend Lint (pull_request)"
)
echo "检查门禁: ${#CONTEXTS[@]} 项"
echo
# 轮询等待,最多30分钟(180次x10秒)
for attempt in $(seq 1 180); do
ALL_SUCCESS=true
ANY_FAILED=false
echo "--- 第${attempt}次检查 ($(date '+%H:%M:%S')) ---"
# 检查CI状态
for ctx in "${CONTEXTS[@]}"; do
STATE=$(python3 scripts/check_ci_status.py "$GITHUB_TOKEN" "$GITHUB_REPOSITORY" "$GITHUB_SHA" "$ctx")
echo " CI: ${ctx##*/}: $STATE"
if [ "$STATE" != "success" ]; then
ALL_SUCCESS=false
fi
if [ "$STATE" = "failure" ] || [ "$STATE" = "error" ]; then
ANY_FAILED=true
fi
done
# 检查审批状态
APPROVAL_RESULT=$(python3 scripts/check_pr_approval.py "$MERGE_TOKEN" "$GITHUB_REPOSITORY" "$PR_NUMBER" 1)
echo " 审批: $APPROVAL_RESULT"
HAS_APPROVAL=false
if echo "$APPROVAL_RESULT" | grep -q '^approved'; then
HAS_APPROVAL=true
fi
# 全部满足 → 合并
if [ "$ALL_SUCCESS" = "true" ] && [ "$HAS_APPROVAL" = "true" ]; then
echo
echo "CI全绿 + 审批通过,执行自动合并"
# 幂等检查:PR是否还是open
PR_STATE=$(curl -s -H "Authorization: token ${MERGE_TOKEN}" \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}" \
| python3 -c "import sys,json; print(json.load(sys.stdin).get('state',''))")
if [ "$PR_STATE" != "open" ]; then
echo "PR状态为 ${PR_STATE},无需合并"
exit 0
fi
# 执行mergemain分支用merge,保留历史)
HTTP_CODE=$(curl -s -o /tmp/merge_resp.json -w "%{http_code}" \
-X POST \
-H "Authorization: token ${MERGE_TOKEN}" \
-H "Content-Type: application/json" \
-d '{"do":"merge","merge_title_field":"","merge_message_field":"","delete_branch_after_merge":true,"force_merge":false}' \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}/merge")
echo "合并API HTTP状态: $HTTP_CODE"
if [ "$HTTP_CODE" = "200" ]; then
echo "自动合并成功"
exit 0
elif [ "$HTTP_CODE" = "405" ]; then
echo "合并失败(405),可能有冲突或门禁未通过"
curl -s -X POST \
-H "Authorization: token ${MERGE_TOKEN}" \
-H "Content-Type: application/json" \
-d '{"body": "Auto merge failed: PR may have conflicts or unresolved checks. Please review manually."}' \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/comments" > /dev/null 2>&1 || true
exit 0
else
echo "自动合并失败 (HTTP $HTTP_CODE)"
cat /tmp/merge_resp.json 2>/dev/null || true
curl -s -X POST \
-H "Authorization: token ${MERGE_TOKEN}" \
-H "Content-Type: application/json" \
-d "{\"body\": \"Auto merge failed (HTTP ${HTTP_CODE}), please check manually.\"}" \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/issues/${PR_NUMBER}/comments" > /dev/null 2>&1 || true
exit 1
fi
fi
if [ "$ANY_FAILED" = "true" ]; then
echo
echo "CI有失败项,不自动合并"
exit 0
fi
sleep 10
done
echo
echo "等待超时(30分钟)"
exit 0
+21
View File
@@ -128,6 +128,27 @@ jobs:
grep -q "Running upgrade" /tmp/alembic-upgrade.sql
python3 scripts/check_schema_metadata.py
- name: Start Redis for unit tests
shell: sh
run: |
set -eu
docker rm -f ci-redis-validate 2>/dev/null || true
docker run -d --name ci-redis-validate \
-p 6379:6379 \
--health-cmd "redis-cli ping" \
--health-interval 2s \
--health-timeout 2s \
--health-retries 15 \
redis:7-alpine
for i in $(seq 1 20); do
if docker inspect --format='{{.State.Health.Status}}' ci-redis-validate 2>/dev/null | grep -q healthy; then
echo "Redis is ready"
break
fi
echo "Waiting for Redis... ($i/20)"
sleep 1
done
docker inspect --format='{{.State.Health.Status}}' ci-redis-validate | grep -q healthy
- name: Run unit tests
shell: sh
env:
-658
View File
@@ -1,658 +0,0 @@
name: Daily Health Check
on:
schedule:
- cron: '0 19 * * *' # UTC 19:00 = 北京时间凌晨 3:00
workflow_dispatch:
permissions:
contents: read
jobs:
# ── 1. 生产环境冒烟测试 ─────────────────────────────────────────────
production-smoke:
name: Production Smoke Test
runs-on: saas
timeout-minutes: 8
outputs:
report: ${{ steps.smoke.outputs.report }}
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
python3 - <<'PY'
import io, os, tarfile, time, urllib.request, urllib.error
url = f"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz"
request = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['GITHUB_TOKEN']}"})
last_err = None
for attempt in range(5):
try:
with urllib.request.urlopen(request, timeout=120) as response:
archive = response.read()
break
except urllib.error.HTTPError as e:
last_err = e
if e.code >= 500 and attempt < 4:
wait = 2 ** attempt
print(f"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
except Exception as e:
last_err = e
if attempt < 4:
wait = 2 ** attempt
print(f"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
else:
raise last_err
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
PY
- name: Production health check & smoke test
id: smoke
shell: sh
env:
SMOKE_ENV: production
EXISTING_TOKEN: ${{ secrets.PROD_E2E_TOKEN }}
MODULES: health,assets,generation,subscription,nginx
run: |
set +e
START_TIME=$(date +%s)
chmod +x tests/e2e/api_smoke_test.sh
BASE_URL="https://api.xiaoxiajianji.com" \
WEB_URL="https://saas.xiaoxiajianji.com" \
SMOKE_ENV="${SMOKE_ENV}" \
EXISTING_TOKEN="${EXISTING_TOKEN}" \
MODULES="${MODULES}" \
CLEANUP_ENABLED=0 \
PERF_CHECK_ENABLED=1 \
PERF_WARN_THRESHOLD_MS=500 \
PERF_FAIL_THRESHOLD_MS=5000 \
bash tests/e2e/api_smoke_test.sh 2>&1 | tee /tmp/prod-smoke.log
SMOKE_EXIT=${PIPESTATUS[0]}
END_TIME=$(date +%s)
ELAPSED=$((END_TIME - START_TIME))
echo ""
echo "========== 生产冒烟测试报告 =========="
echo "环境: https://api.xiaoxiajianji.com"
echo "耗时: ${ELAPSED}s"
# 提取通过/失败数
grep "测试完成:" /tmp/prod-smoke.log || true
if [ "$SMOKE_EXIT" -eq 0 ]; then
echo "结果: PASS"
echo "report=PASS" >> "${GITHUB_OUTPUT}"
else
echo "结果: FAIL"
grep "失败用例:" /tmp/prod-smoke.log || true
echo "report=FAIL" >> "${GITHUB_OUTPUT}"
fi
echo "======================================"
exit $SMOKE_EXIT
# ── 2. Staging API 集成测试 ─────────────────────────────────────────
staging-api-tests:
name: Staging API Integration Tests
runs-on: saas
timeout-minutes: 10
outputs:
report: ${{ steps.smoke.outputs.report }}
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
python3 - <<'PY'
import io, os, tarfile, time, urllib.request, urllib.error
url = f"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz"
request = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['GITHUB_TOKEN']}"})
last_err = None
for attempt in range(5):
try:
with urllib.request.urlopen(request, timeout=120) as response:
archive = response.read()
break
except urllib.error.HTTPError as e:
last_err = e
if e.code >= 500 and attempt < 4:
wait = 2 ** attempt
print(f"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
except Exception as e:
last_err = e
if attempt < 4:
wait = 2 ** attempt
print(f"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
else:
raise last_err
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
PY
- name: Run API smoke test on staging
id: smoke
shell: sh
run: |
set +e
START_TIME=$(date +%s)
chmod +x tests/e2e/api_smoke_test.sh
docker run --rm \
-e BASE_URL=https://staging-api.xiaoxiajianji.com \
-e WEB_URL=https://staging.xiaoxiajianji.com \
-e TEST_USER=18314979086@163.com \
-e TEST_PASSWORD=Ying1234 \
-e CLEANUP_ENABLED=1 \
-e PERF_CHECK_ENABLED=1 \
-e PERF_WARN_THRESHOLD_MS=500 \
-e PERF_FAIL_THRESHOLD_MS=3000 \
-v "$PWD:/workspace" \
-w /workspace \
git.xiaoxiajianji.com/xiaoxia/base/playwright:v1.45.0-jammy \
bash tests/e2e/api_smoke_test.sh 2>&1 | tee /tmp/staging-api-smoke.log
SMOKE_EXIT=${PIPESTATUS[0]}
END_TIME=$(date +%s)
ELAPSED=$((END_TIME - START_TIME))
echo ""
echo "========== Staging API 冒烟测试报告 =========="
echo "环境: https://staging-api.xiaoxiajianji.com"
echo "耗时: ${ELAPSED}s"
grep "测试完成:" /tmp/staging-api-smoke.log || true
if [ "$SMOKE_EXIT" -eq 0 ]; then
echo "结果: PASS"
echo "api_report=PASS" >> "${GITHUB_OUTPUT}"
else
echo "结果: FAIL"
grep "失败用例:" /tmp/staging-api-smoke.log || true
echo "api_report=FAIL" >> "${GITHUB_OUTPUT}"
fi
echo "=============================================="
exit $SMOKE_EXIT
- name: Run Staging API Integration Tests (Playwright)
id: e2e_api
shell: sh
run: |
set +e
START_TIME=$(date +%s)
docker run --rm \
-e E2E_BASE_URL=https://staging.xiaoxiajianji.com \
-e E2E_API_BASE=https://staging-api.xiaoxiajianji.com/api/v1 \
-v "$PWD:/workspace" \
-w /workspace/apps/web \
git.xiaoxiajianji.com/xiaoxia/base/playwright:v1.45.0-jammy \
sh -lc "npm ci && npx playwright test --reporter=line e2e/test_auth.spec.ts e2e/test_asset.spec.ts e2e/test_project.spec.ts" 2>&1 | tee /tmp/staging-api-e2e.log
EXIT_CODE=${PIPESTATUS[0]}
END_TIME=$(date +%s)
ELAPSED=$((END_TIME - START_TIME))
echo ""
echo "========== Staging API 集成测试报告 =========="
echo "环境: https://staging-api.xiaoxiajianji.com"
echo "耗时: ${ELAPSED}s"
grep -E "passed|failed|timed out" /tmp/staging-api-e2e.log || true
if [ "$EXIT_CODE" -eq 0 ]; then
echo "结果: PASS"
echo "int_report=PASS" >> "${GITHUB_OUTPUT}"
else
echo "结果: FAIL"
echo "int_report=FAIL" >> "${GITHUB_OUTPUT}"
fi
echo "=============================================="
exit $EXIT_CODE
- name: Set report output
id: report
shell: sh
run: |
if [ "${{ steps.smoke.outputs.api_report }}" = "PASS" ] && [ "${{ steps.e2e_api.outputs.int_report }}" = "PASS" ]; then
echo "report=PASS" >> "${GITHUB_OUTPUT}"
else
echo "report=FAIL" >> "${GITHUB_OUTPUT}"
fi
# ── 3. Staging 浏览器 E2E ──────────────────────────────────────────
staging-e2e:
name: Staging Browser E2E
runs-on: saas
timeout-minutes: 15
outputs:
report: ${{ steps.smoke.outputs.report }}
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
python3 - <<'PY'
import io, os, tarfile, time, urllib.request, urllib.error
url = f"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz"
request = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['GITHUB_TOKEN']}"})
last_err = None
for attempt in range(5):
try:
with urllib.request.urlopen(request, timeout=120) as response:
archive = response.read()
break
except urllib.error.HTTPError as e:
last_err = e
if e.code >= 500 and attempt < 4:
wait = 2 ** attempt
print(f"Checkout HTTP {e.code}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
except Exception as e:
last_err = e
if attempt < 4:
wait = 2 ** attempt
print(f"Checkout error: {e}, retrying in {wait}s (attempt {attempt+1}/5)...")
time.sleep(wait)
continue
raise
else:
raise last_err
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
PY
- name: Run Playwright E2E on staging
id: e2e
shell: sh
run: |
set +e
START_TIME=$(date +%s)
docker run --rm --ipc=host \
-e E2E_BASE_URL=https://staging.xiaoxiajianji.com \
-e E2E_API_BASE=https://staging-api.xiaoxiajianji.com/api/v1 \
-e E2E_BROWSER_CHANNEL=chromium \
-e PLAYWRIGHT_HEADLESS=1 \
-v "$PWD:/workspace" \
-w /workspace/apps/web \
git.xiaoxiajianji.com/xiaoxia/base/playwright:v1.45.0-jammy \
sh -lc 'npm ci && npx playwright test --reporter=line --project=chromium e2e/auth.spec.ts e2e/auth-guard.spec.ts e2e/core-upload.spec.ts e2e/core-generation.spec.ts e2e/core-titles.spec.ts' 2>&1 | tee /tmp/staging-e2e.log
EXIT_CODE=${PIPESTATUS[0]}
END_TIME=$(date +%s)
ELAPSED=$((END_TIME - START_TIME))
echo ""
echo "========== Staging E2E 测试报告 =========="
echo "环境: https://staging.xiaoxiajianji.com"
echo "耗时: ${ELAPSED}s"
grep -E "passed|failed|timed out" /tmp/staging-e2e.log || true
if [ "$EXIT_CODE" -eq 0 ]; then
echo "结果: PASS"
echo "report=PASS" >> "${GITHUB_OUTPUT}"
else
echo "结果: FAIL"
echo "report=FAIL" >> "${GITHUB_OUTPUT}"
fi
echo "=========================================="
exit $EXIT_CODE
# ── 4. 性能基线巡检 ────────────────────────────────────────────────
performance-check:
name: Performance Baseline Check
runs-on: saas
timeout-minutes: 8
outputs:
report: ${{ steps.report.outputs.report }}
steps:
- name: Run performance baseline checks
id: perf
shell: sh
run: |
set +e
START_TIME=$(date +%s)
echo "=========================================="
echo " 性能基线巡检 - Staging API"
echo " 目标: https://staging-api.xiaoxiajianji.com"
echo "=========================================="
echo ""
TOTAL=0
PASS=0
FAIL=0
WARN=0
WARN_LIST=""
FAIL_LIST=""
# 核心接口配置: 名称|路径|方法|阈值(ms)|失败阈值(ms)
# 核心接口(core): 500ms
# 普通接口(normal): 1000ms
# 重操作接口(heavy): 3000ms
ENDPOINTS="
登录|/api/v1/auth/login|POST|500|3000
获取当前用户|/api/v1/auth/me|GET|500|3000
项目列表|/api/v1/projects|GET|500|3000
素材列表|/api/v1/assets|GET|500|3000
模板列表|/api/v1/templates|GET|500|3000
剪辑计划列表|/api/v1/edit-plans|GET|500|3000
生成任务列表|/api/v1/generation/tasks|GET|500|3000
订阅信息|/api/v1/subscription/current|GET|500|3000
音色列表|/api/v1/voices|GET|1000|5000
健康检查|/health|GET|200|1000
"
# 先登录获取 token
echo "--- 准备: 获取测试 Token ---"
AUTH_RESP=$(curl -s -w "\n%{http_code}" -X POST \
-H "Content-Type: application/json" \
-d '{"email":"18314979086@163.com","password":"Ying1234"}' \
"https://staging-api.xiaoxiajianji.com/api/v1/auth/login" \
--max-time 10 2>&1)
AUTH_CODE=$(echo "$AUTH_RESP" | tail -1)
AUTH_BODY=$(echo "$AUTH_RESP" | sed '$d')
if [ "$AUTH_CODE" = "200" ]; then
TOKEN=$(echo "$AUTH_BODY" | python3 -c "import sys,json; d=json.load(sys.stdin); print(d.get('access_token',''))" 2>/dev/null)
if [ -n "$TOKEN" ]; then
echo "Token 获取成功"
else
echo "Token 解析失败,部分接口可能无法测试"
TOKEN=""
fi
else
echo "登录失败 (HTTP $AUTH_CODE),部分接口将跳过鉴权测试"
TOKEN=""
fi
echo ""
echo "--- 开始性能测试 ---"
echo ""
echo "$ENDPOINTS" | while IFS='|' read -r name path method warn_ms fail_ms; do
[ -z "$name" ] && continue
TOTAL=$((TOTAL + 1))
# 构建 curl 命令
CURL_ARGS="-s -o /dev/null -w '%{http_code} %{time_total}' --max-time 30"
if [ "$method" = "POST" ]; then
CURL_ARGS="$CURL_ARGS -X POST -H 'Content-Type: application/json' -d '{\"email\":\"18314979086@163.com\",\"password\":\"Ying1234\"}'"
fi
if [ -n "$TOKEN" ] && [ "$name" != "健康检查" ]; then
CURL_ARGS="$CURL_ARGS -H 'Authorization: Bearer $TOKEN'"
fi
# 执行请求
RESP=$(eval curl $CURL_ARGS "https://staging-api.xiaoxiajianji.com${path}" 2>&1)
HTTP_CODE=$(echo "$RESP" | awk '{print $1}')
TIME_TOTAL=$(echo "$RESP" | awk '{print $2}')
ELAPSED_MS=$(python3 -c "print(int(float('${TIME_TOTAL:-0}') * 1000))" 2>/dev/null || echo "0")
if [ "$HTTP_CODE" -ge 500 ] 2>/dev/null; then
FAIL=$((FAIL + 1))
FAIL_LIST="$FAIL_LIST\n ❌ $name - HTTP $HTTP_CODE (${ELAPSED_MS}ms)"
echo "❌ $name - HTTP $HTTP_CODE - ${ELAPSED_MS}ms (FAIL)"
elif [ "$ELAPSED_MS" -ge "$fail_ms" ] 2>/dev/null; then
FAIL=$((FAIL + 1))
FAIL_LIST="$FAIL_LIST\n ❌ $name - ${ELAPSED_MS}ms > ${fail_ms}ms"
echo "❌ $name - ${ELAPSED_MS}ms > ${fail_ms}ms (FAIL)"
elif [ "$ELAPSED_MS" -ge "$warn_ms" ] 2>/dev/null; then
WARN=$((WARN + 1))
WARN_LIST="$WARN_LIST\n ⚠️ $name - ${ELAPSED_MS}ms > ${warn_ms}ms"
echo "⚠️ $name - ${ELAPSED_MS}ms (WARN, threshold: ${warn_ms}ms)"
PASS=$((PASS + 1))
else
PASS=$((PASS + 1))
echo "✅ $name - ${ELAPSED_MS}ms (OK, threshold: ${warn_ms}ms)"
fi
done
# 由于 while 在子 shell 中执行,用文件传递结果
# 重新跑一次用文件计数方式
echo ""
echo "--- 汇总性能数据 ---"
END_TIME=$(date +%s)
ELAPSED=$((END_TIME - START_TIME))
echo ""
echo "========== 性能基线巡检报告 =========="
echo "环境: https://staging-api.xiaoxiajianji.com"
echo "耗时: ${ELAPSED}s"
echo "======================================"
- name: Generate performance report
id: report
shell: sh
run: |
set +e
echo ""
echo "=========================================="
echo " 性能基线巡检 - 详细报告"
echo "=========================================="
TOTAL=0
PASS=0
FAIL=0
WARN=0
RESULTS=""
START_TIME=$(date +%s)
# 先登录获取 token
AUTH_RESP=$(curl -s -w "\n%{http_code}" -X POST \
-H "Content-Type: application/json" \
-d '{"email":"18314979086@163.com","password":"Ying1234"}' \
"https://staging-api.xiaoxiajianji.com/api/v1/auth/login" \
--max-time 10 2>&1)
AUTH_CODE=$(echo "$AUTH_RESP" | tail -1)
AUTH_BODY=$(echo "$AUTH_RESP" | sed '$d')
TOKEN=""
if [ "$AUTH_CODE" = "200" ]; then
TOKEN=$(echo "$AUTH_BODY" | python3 -c "import sys,json; d=json.load(sys.stdin); print(d.get('access_token',''))" 2>/dev/null || echo "")
fi
run_perf_test() {
local name="$1" path="$2" method="$3" warn_ms="$4" fail_ms="$5"
TOTAL=$((TOTAL + 1))
local CURL_ARGS="-s -o /dev/null -w '%{http_code} %{time_total}' --max-time 30"
if [ "$method" = "POST" ]; then
CURL_ARGS="$CURL_ARGS -X POST -H 'Content-Type: application/json' -d '{\"email\":\"18314979086@163.com\",\"password\":\"Ying1234\"}'"
fi
if [ -n "$TOKEN" ] && [ "$name" != "健康检查" ]; then
CURL_ARGS="$CURL_ARGS -H 'Authorization: Bearer $TOKEN'"
fi
local RESP=$(eval curl $CURL_ARGS "https://staging-api.xiaoxiajianji.com${path}" 2>&1)
local HTTP_CODE=$(echo "$RESP" | awk '{print $1}')
local TIME_TOTAL=$(echo "$RESP" | awk '{print $2}')
local ELAPSED_MS=$(python3 -c "print(int(float('${TIME_TOTAL:-0}') * 1000))" 2>/dev/null || echo "0")
if echo "$HTTP_CODE" | grep -q "^[5]"; then
FAIL=$((FAIL + 1))
RESULTS="$RESULTS\n ❌ $name - HTTP $HTTP_CODE (${ELAPSED_MS}ms)"
echo "❌ $name - HTTP $HTTP_CODE - ${ELAPSED_MS}ms [FAIL]"
return 1
elif [ "$ELAPSED_MS" -ge "$fail_ms" ] 2>/dev/null; then
FAIL=$((FAIL + 1))
RESULTS="$RESULTS\n ❌ $name - ${ELAPSED_MS}ms > ${fail_ms}ms [FAIL]"
echo "❌ $name - ${ELAPSED_MS}ms > ${fail_ms}ms [FAIL]"
return 1
elif [ "$ELAPSED_MS" -ge "$warn_ms" ] 2>/dev/null; then
WARN=$((WARN + 1))
PASS=$((PASS + 1))
RESULTS="$RESULTS\n ⚠️ $name - ${ELAPSED_MS}ms (阈值: ${warn_ms}ms) [WARN]"
echo "⚠️ $name - ${ELAPSED_MS}ms > 阈值 ${warn_ms}ms [WARN]"
return 0
else
PASS=$((PASS + 1))
RESULTS="$RESULTS\n ✅ $name - ${ELAPSED_MS}ms (阈值: ${warn_ms}ms) [OK]"
echo "✅ $name - ${ELAPSED_MS}ms (阈值: ${warn_ms}ms) [OK]"
return 0
fi
}
echo ""
echo "=== 核心接口 (阈值: 500ms / 3000ms) ==="
run_perf_test "登录" "/api/v1/auth/login" "POST" 500 3000 || true
run_perf_test "获取当前用户" "/api/v1/auth/me" "GET" 500 3000 || true
run_perf_test "项目列表" "/api/v1/projects" "GET" 500 3000 || true
run_perf_test "素材列表" "/api/v1/assets" "GET" 500 3000 || true
run_perf_test "模板列表" "/api/v1/templates" "GET" 500 3000 || true
run_perf_test "剪辑计划列表" "/api/v1/edit-plans" "GET" 500 3000 || true
run_perf_test "生成任务列表" "/api/v1/generation/tasks" "GET" 500 3000 || true
run_perf_test "订阅信息" "/api/v1/subscription/current" "GET" 500 3000 || true
echo ""
echo "=== 普通接口 (阈值: 1000ms / 5000ms) ==="
run_perf_test "音色列表" "/api/v1/voices" "GET" 1000 5000 || true
echo ""
echo "=== 基础接口 (阈值: 200ms / 1000ms) ==="
run_perf_test "健康检查" "/health" "GET" 200 1000 || true
END_TIME=$(date +%s)
ELAPSED=$((END_TIME - START_TIME))
echo ""
echo "========== 性能基线巡检报告 =========="
echo "环境: https://staging-api.xiaoxiajianji.com"
echo "总接口: ${TOTAL}"
echo "通过: ${PASS}"
echo "失败: ${FAIL}"
echo "警告: ${WARN}"
echo "耗时: ${ELAPSED}s"
echo "======================================"
# 写入结果文件供 report job 使用
echo "${TOTAL}" > /tmp/perf_total
echo "${PASS}" > /tmp/perf_pass
echo "${FAIL}" > /tmp/perf_fail
echo "${WARN}" > /tmp/perf_warn
echo "${ELAPSED}" > /tmp/perf_elapsed
if [ "$FAIL" -gt 0 ]; then
echo "report=FAIL" >> "${GITHUB_OUTPUT}"
echo "perf_detail=fail:${FAIL}:warn:${WARN}" >> "${GITHUB_OUTPUT}"
exit 1
else
echo "report=PASS" >> "${GITHUB_OUTPUT}"
if [ "$WARN" -gt 0 ]; then
echo "perf_detail=pass:warn:${WARN}" >> "${GITHUB_OUTPUT}"
else
echo "perf_detail=pass" >> "${GITHUB_OUTPUT}"
fi
exit 0
fi
# ── 5. 每日巡检汇总报告 ────────────────────────────────────────────
daily-report:
name: Daily Check Report
runs-on: saas
timeout-minutes: 2
if: always()
needs:
- production-smoke
- staging-api-tests
- staging-e2e
- performance-check
steps:
- name: Print summary report
shell: sh
run: |
echo ""
echo "╔══════════════════════════════════════════════════════╗"
echo "║ 每日巡检报告 ║"
echo "╠══════════════════════════════════════════════════════╣"
# 获取各 job 状态
PROD_STATUS="${{ needs.production-smoke.result }}"
STAGING_API_STATUS="${{ needs.staging-api-tests.result }}"
STAGING_E2E_STATUS="${{ needs.staging-e2e.result }}"
PERF_STATUS="${{ needs.performance-check.result }}"
format_result() {
if [ "$1" = "success" ]; then
echo "✅ PASS"
elif [ "$1" = "failure" ]; then
echo "❌ FAIL"
elif [ "$1" = "skipped" ]; then
echo "⏭️ SKIP"
else
echo "❓ UNKNOWN ($1)"
fi
}
echo "║"
echo "║ 生产冒烟测试: $(format_result "$PROD_STATUS")"
echo "║ Staging API: $(format_result "$STAGING_API_STATUS")"
echo "║ Staging E2E: $(format_result "$STAGING_E2E_STATUS")"
echo "║ 性能基线巡检: $(format_result "$PERF_STATUS")"
echo "║"
echo "║ 巡检时间: $(date '+%Y-%m-%d %H:%M:%S UTC')"
echo "║"
# 判断整体状态
ALL_PASS=true
FAILED_ITEMS=""
for status_name in "$PROD_STATUS:生产冒烟" "$STAGING_API_STATUS:Staging API" "$STAGING_E2E_STATUS:Staging E2E" "$PERF_STATUS:性能基线"; do
STATUS=$(echo "$status_name" | cut -d: -f1)
NAME=$(echo "$status_name" | cut -d: -f2)
if [ "$STATUS" != "success" ] && [ "$STATUS" != "skipped" ]; then
ALL_PASS=false
FAILED_ITEMS="$FAILED_ITEMS $NAME"
fi
done
echo "╠══════════════════════════════════════════════════════╣"
if [ "$ALL_PASS" = "true" ]; then
echo "║ 整体状态: ✅ 全部通过 ║"
else
echo "║ 整体状态: ❌ 存在失败 ║"
echo "║ 失败项: ${FAILED_ITEMS} ║"
fi
echo "╚══════════════════════════════════════════════════════╝"
echo ""
# 如果有失败项,以非零退出码结束(方便 Gitea 标记流水线失败)
if [ "$ALL_PASS" = "false" ]; then
echo "⚠️ 部分巡检项失败,请检查上方日志获取详细信息。"
# 不 exit 1,因为我们用了 always(),保持 report job 成功,
# 但其他失败的 job 已经让整体流水线标记为失败
fi
+6 -2
View File
@@ -11,9 +11,11 @@ jobs:
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
python - <<'PY'
python3 - <<'PY'
import io
import os
import tarfile
@@ -93,9 +95,11 @@ jobs:
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
set -eu
python - <<'PY'
python3 - <<'PY'
import io
import os
import tarfile
+44
View File
@@ -0,0 +1,44 @@
#!/usr/bin/env python3
"""检查指定commit的CI status状态。
用法: python3 check_ci_status.py <token> <repo> <sha> <context>
返回: 打印状态 (success/failure/pending/error)
"""
import json
import sys
import urllib.error
import urllib.request
def main():
if len(sys.argv) != 5:
print("pending")
return
token = sys.argv[1]
repo = sys.argv[2]
sha = sys.argv[3]
target_context = sys.argv[4]
api_url = f"https://git.xiaoxiajianji.com/api/v1/repos/{repo}/commits/{sha}/statuses?per_page=100"
req = urllib.request.Request(api_url, headers={"Authorization": f"token {token}"})
try:
with urllib.request.urlopen(req, timeout=30) as resp:
statuses = json.loads(resp.read().decode())
except Exception:
print("pending")
return
# API返回按时间倒序,第一个就是最新的
for s in statuses:
if s.get("context") == target_context:
print(s.get("status", "pending"))
return
print("pending")
if __name__ == "__main__":
main()
+46
View File
@@ -0,0 +1,46 @@
#!/usr/bin/env python3
"""检查PR是否有至少N个APPROVED审批。
用法: python3 check_pr_approval.py <token> <repo> <pr_number> <min_approval>
返回: 打印 "approved""pending"
"""
import json
import sys
import urllib.request
def main():
if len(sys.argv) != 5:
print("pending")
return
token = sys.argv[1]
repo = sys.argv[2]
pr_number = sys.argv[3]
min_approval = int(sys.argv[4])
api_url = f"https://git.xiaoxiajianji.com/api/v1/repos/{repo}/pulls/{pr_number}/reviews"
req = urllib.request.Request(api_url, headers={"Authorization": f"token {token}"})
try:
with urllib.request.urlopen(req, timeout=30) as resp:
reviews = json.loads(resp.read().decode())
except Exception:
print("pending")
return
# 统计APPROVED的人数(去重,同一人多次审批只算一次)
approvers = set()
for r in reviews:
if r.get("state") == "APPROVED":
approvers.add(r.get("user", {}).get("login", ""))
if len(approvers) >= min_approval:
print(f"approved ({len(approvers)})")
else:
print(f"pending ({len(approvers)})")
if __name__ == "__main__":
main()
+408
View File
@@ -0,0 +1,408 @@
#!/usr/bin/env python3
"""
PR自动扫描器:扫描所有open PR,对CI全绿的进行自动审批/合并
作为短作业模式的兜底机制,每5分钟运行一次
新增:AI审查联动 - AI代码审查发现严重问题时,不自动审批
"""
import argparse
import json
import os
import re
import sys
import time
import urllib.error
import urllib.request
def api_request(token, repo, endpoint, method="GET", data=None):
"""Gitea API请求"""
url = f"https://git.xiaoxiajianji.com/api/v1/repos/{repo}/{endpoint}"
headers = {"Authorization": f"token {token}", "Content-Type": "application/json"}
body = json.dumps(data).encode() if data else None
req = urllib.request.Request(url, data=body, headers=headers, method=method)
# 跳过SSL验证
import ssl
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
try:
resp = urllib.request.urlopen(req, context=ctx)
return json.loads(resp.read().decode()), resp.status
except urllib.error.HTTPError as e:
return json.loads(e.read().decode()) if e.read() else {"error": str(e)}, e.code
def get_open_prs(token, repo, base="develop"):
"""获取所有open的PR"""
prs = []
page = 1
while True:
data, code = api_request(token, repo, f"pulls?state=open&base={base}&sort=recentupdate&per_page=50&page={page}")
if code != 200 or not isinstance(data, list) or len(data) == 0:
break
prs.extend(data)
if len(data) < 50:
break
page += 1
return prs
def get_commit_status(token, repo, sha):
"""获取commit的CI状态汇总"""
data, code = api_request(token, repo, f"commits/{sha}/status")
if code != 200:
return {}, "error"
return data, data.get("state", "unknown")
def check_required_contexts(token, repo, sha, contexts):
"""检查指定的context是否都通过"""
data, _ = get_commit_status(token, repo, sha)
statuses = {s["context"]: s["status"] for s in data.get("statuses", [])}
all_success = True
any_pending = False
any_failed = False
for ctx in contexts:
state = statuses.get(ctx, "pending")
if state != "success":
all_success = False
if state == "pending":
any_pending = True
if state in ("failure", "error"):
any_failed = True
return all_success, any_pending, any_failed, statuses
def get_pr_files(token, repo, pr_number):
"""获取PR变更文件"""
files = []
page = 1
while True:
data, code = api_request(token, repo, f"pulls/{pr_number}/files?per_page=300&page={page}")
if code != 200 or not isinstance(data, list) or len(data) == 0:
break
files.extend(data)
if len(data) < 300:
break
page += 1
return [f["filename"] for f in files]
def is_frontend_only(files):
"""判断是否纯前端改动"""
if not files:
return False
frontend_count = sum(1 for f in files if f.startswith("apps/web/"))
backend_count = len(files) - frontend_count
return backend_count == 0 and frontend_count > 0
def has_approval(token, repo, pr_number):
"""检查PR是否已有审批"""
reviews, code = api_request(token, repo, f"pulls/{pr_number}/reviews")
if code != 200:
return False
return any(r.get("state") == "APPROVED" for r in reviews if isinstance(r, dict))
def get_ai_review_result(token, repo, pr_number):
"""
检查AI代码审查结果,返回 (has_critical, review_body)
has_critical: 是否有严重问题(需修改的问题 > 0)
review_body: 最新的AI审查评论文本
"""
# AI审查评论标记
AI_REVIEW_MARKER = "AI_CODE_REVIEW_AUTO_COMMENT"
comments, code = api_request(token, repo, f"issues/{pr_number}/comments")
if code != 200:
return False, None
# 找最新的AI审查评论
ai_comments = [c for c in comments if isinstance(c, dict) and AI_REVIEW_MARKER in c.get("body", "")]
if not ai_comments:
return False, None
# 按时间排序,取最新的
latest = max(ai_comments, key=lambda c: c.get("created_at", ""))
body = latest.get("body", "")
# 解析严重问题数量
# 匹配 "严重问题数量:X 个" 或 "需修改的问题(严重)" 下的列表
critical_count = 0
# 方式1:直接匹配数字
match = re.search(r"严重问题数量[:]\s*(\d+)\s*个", body)
if match:
critical_count = int(match.group(1))
else:
# 方式2:数 "需修改的问题" 章节下的条目数
critical_section = re.search(
r"###\s*[❌⚠️].*?(?:需修改|问题).*?\n(.*?)(?=\n###|\Z)",
body,
re.DOTALL,
)
if critical_section:
section_text = critical_section.group(1)
# 数编号条目 1. 2. 3.
items = re.findall(r"^\d+\.\s+\*\*", section_text, re.MULTILINE)
critical_count = len(items)
has_critical = critical_count > 0
return has_critical, body
def approve_pr(token, repo, pr_number, reason="CI全绿,自动审批通过。"):
"""审批PR"""
# 创建review
data, code = api_request(
token,
repo,
f"pulls/{pr_number}/reviews",
method="POST",
data={"event": "PENDING", "body": reason},
)
if code not in (200, 201):
return False, f"创建review失败: HTTP {code}"
review_id = data.get("id")
if data.get("state") == "APPROVED":
return True, "直接创建APPROVED成功"
if not review_id:
return False, "未获取到review ID"
# submit为APPROVED
data2, code2 = api_request(
token,
repo,
f"pulls/{pr_number}/reviews/{review_id}/events",
method="POST",
data={"event": "APPROVED", "body": reason},
)
if code2 in (200, 201):
return True, "审批提交成功"
else:
# 尝试另一个端点
data3, code3 = api_request(
token,
repo,
f"pulls/{pr_number}/reviews/{review_id}",
method="POST",
data={"event": "APPROVED", "body": reason},
)
if code3 in (200, 201):
return True, "审批提交成功(备用端点)"
return False, f"审批提交失败: HTTP {code2}/{code3}"
def add_pr_label(token, repo, pr_number, label):
"""给PR添加标签"""
data, code = api_request(
token,
repo,
f"issues/{pr_number}/labels",
method="POST",
data={"labels": [label]},
)
return code in (200, 201)
def merge_pr(token, repo, pr_number):
"""合并PRsquash merge"""
# 等待几秒让状态同步
time.sleep(30)
# 检查PR状态
pr_data, code = api_request(token, repo, f"pulls/{pr_number}")
if code != 200:
return False, f"获取PR状态失败: HTTP {code}"
if pr_data.get("state") != "open":
return False, f"PR状态不是open: {pr_data.get('state')}"
# 执行squash merge
data, code = api_request(
token,
repo,
f"pulls/{pr_number}/merge",
method="POST",
data={
"do": "squash",
"merge_title_field": "",
"merge_message_field": "",
"delete_branch_after_merge": True,
"force_merge": False,
},
)
if code == 200:
return True, "合并成功"
elif code == 405:
return False, "合并返回405(门禁未满足或冲突)"
else:
return False, f"合并失败: HTTP {code}"
def main():
parser = argparse.ArgumentParser(description="PR自动扫描器")
parser.add_argument("--token", required=True, help="Gitea API token")
parser.add_argument("--repo", default="xiaoxia/xiaoxia-saas", help="仓库")
parser.add_argument("--base", default="develop", help="目标分支")
parser.add_argument("--approve", action="store_true", help="执行自动审批")
parser.add_argument("--merge", action="store_true", help="执行自动合并")
parser.add_argument("--dry-run", default="false", help="试运行模式")
parser.add_argument("--max-prs", type=int, default=20, help="最多处理的PR数")
parser.add_argument("--skip-ai-review", action="store_true", help="跳过AI审查检查(强制审批)")
args = parser.parse_args()
dry_run = args.dry_run.lower() == "true"
# required contexts(与分支保护一致)
REQUIRED_CONTEXTS_FULL = [
"CI/CD Pipeline / Validate - Code Quality (pull_request)",
"CI/CD Pipeline / Validate - Type Check (mypy) (pull_request)",
"CI/CD Pipeline / Validate - Migration (alembic) (pull_request)",
"CI/CD Pipeline / Frontend Lint (pull_request)",
"CI/CD Pipeline / PR Build API Image (pull_request)",
"CI/CD Pipeline / PR Build Worker Image (pull_request)",
"CI/CD Pipeline / PR Build Web Image (pull_request)",
]
REQUIRED_CONTEXTS_APPROVE = [
"CI/CD Pipeline / Validate - Code Quality (pull_request)",
"CI/CD Pipeline / Validate - Type Check (mypy) (pull_request)",
"CI/CD Pipeline / Validate - Migration (alembic) (pull_request)",
"CI/CD Pipeline / Frontend Lint (pull_request)",
]
FRONTEND_ONLY_CONTEXT = [
"CI/CD Pipeline / Frontend Lint (pull_request)",
]
# 获取所有open PR
print(f"获取 {args.base} 分支的open PR...")
prs = get_open_prs(args.token, args.repo, args.base)
print(f"找到 {len(prs)} 个open PR")
approved_count = 0
merged_count = 0
skipped_count = 0
ai_blocked_count = 0
for pr in prs[: args.max_prs]:
pr_num = pr["number"]
pr_title = pr["title"]
head_sha = pr["head"]["sha"]
base_ref = pr.get("base", {}).get("re", "")
# 跳过draft
if pr.get("draft"):
print(f"\n⏭️ #{pr_num} {pr_title[:50]} - draft,跳过")
skipped_count += 1
continue
# 跳过目标分支不对的
if base_ref != args.base:
skipped_count += 1
continue
print(f"\n--- #{pr_num} {pr_title[:60]} ---")
# 判断是否纯前端
files = get_pr_files(args.token, args.repo, pr_num)
frontend_only = is_frontend_only(files)
if frontend_only:
approve_contexts = FRONTEND_ONLY_CONTEXT
merge_contexts = FRONTEND_ONLY_CONTEXT
print(f" 类型: 纯前端改动 ({len(files)}个文件)")
else:
approve_contexts = REQUIRED_CONTEXTS_APPROVE
merge_contexts = REQUIRED_CONTEXTS_FULL
print(f" 类型: 全栈/后端改动 ({len(files)}个文件)")
# 检查审批用的CI状态
all_ok, pending, failed, _ = check_required_contexts(args.token, args.repo, head_sha, approve_contexts)
# === AI审查检查 ===
ai_has_critical = False
if not args.skip_ai_review and all_ok and not failed and args.approve:
ai_has_critical, ai_body = get_ai_review_result(args.token, args.repo, pr_num)
if ai_has_critical:
print(" ⚠️ AI审查发现严重问题,阻止自动审批")
ai_blocked_count += 1
# 给PR打标签便于人工识别
if not dry_run:
add_pr_label(args.token, args.repo, pr_num, "ai-review/需修改")
# === 自动审批 ===
if args.approve and all_ok and not failed and not ai_has_critical:
if has_approval(args.token, args.repo, pr_num):
print(" ✅ 已有审批,跳过")
else:
if dry_run:
print(" 🎯 [DRY-RUN] 将自动审批")
else:
print(" 🎯 执行自动审批...")
ok, msg = approve_pr(args.token, args.repo, pr_num)
if ok:
print(f" ✅ 审批成功: {msg}")
approved_count += 1
else:
print(f" ❌ 审批失败: {msg}")
elif ai_has_critical:
print(" 🚫 AI审查阻止审批(人工可手动审批覆盖)")
elif failed:
print(" ❌ CI有失败项,跳过审批")
elif pending:
print(" ⏳ CI仍在运行,跳过")
# === 自动合并 ===
if args.merge:
# 检查合并用的CI状态
merge_ok, merge_pending, merge_failed, _ = check_required_contexts(
args.token, args.repo, head_sha, merge_contexts
)
# 检查审批
approved = has_approval(args.token, args.repo, pr_num)
if merge_ok and approved and not merge_failed:
if dry_run:
print(" 🎯 [DRY-RUN] 将自动合并")
else:
print(" 🎯 执行自动合并...")
ok, msg = merge_pr(args.token, args.repo, pr_num)
if ok:
print(f" ✅ 合并成功: {msg}")
merged_count += 1
else:
print(f" ⚠️ 合并失败: {msg}")
elif merge_pending:
print(" ⏳ 合并条件未满足: CI运行中")
elif merge_failed:
print(" ❌ 合并条件未满足: CI有失败")
elif not approved:
print(" ⏳ 合并条件未满足: 无审批")
print("\n=== 扫描结果 ===")
print(f" 处理PR数: {min(len(prs), args.max_prs)}")
print(f" 自动审批: {approved_count}")
print(f" 自动合并: {merged_count}")
print(f" AI审查阻止: {ai_blocked_count}")
print(f" 跳过: {skipped_count}")
print(" 模式: {'DRY-RUN' if dry_run else '正式执行'}")
if __name__ == "__main__":
main()