xiaoxia/xiaoxia-saas
1
0
Fork 0
Code Issues 16 Pull Requests 12 Actions 34.2k Packages Projects Releases 7 Wiki Activity

feat(ci): integrate gitleaks (P0) and pip-audit (P1) into validate job #256

Closed
xiaoxia wants to merge 9 commits from feat/ci-security-scanning into develop
pull from: feat/ci-security-scanning
merge into: xiaoxia:develop
Conversation 1 Commits 9 Files Changed 4
+443
xiaoxia commented 2026-07-13 15:12:15 +08:00
Owner
Copy Link
Copy Source

概述

在 validate Job 中接入两个安全扫描工具：

gitleaks 密钥检测（P0 - 阻断模式）

  • PR 触发时：增量扫描，只扫描 PR 改动的文件（origin/base..HEAD）
  • 主分支 push 时：全量扫描，扫描整个仓库
  • 发现密钥直接阻断合并，确保敏感信息不会流入代码库
  • 配置 .gitleaks.toml 白名单，排除示例配置、测试文件、文档等已知无密钥区域

pip-audit Python 依赖漏洞扫描（P1 - 告警模式）

  • 扫描所有 requirements 文件（requirements.txt / requirements-base.txt / requirements-dev.txt / requirements-worker.txt）
  • 当前为告警模式，不阻断 CI（先跑 1-2 周摸清存量漏洞）
  • 后续根据漏洞严重程度设置阻断阈值
  • 数据源使用 OSV（PyPA 官方推荐）

validate Job 步骤顺序（更新后）

  1. Checkout code
  2. Verify CI environment
  3. Secret detection (gitleaks) ← 新增（最优先，快速发现问题）
  4. Install dependencies
  5. Python dependency vulnerability scan (pip-audit) ← 新增
  6. Run code quality checks
  7. Run security scan (bandit)
  8. Validate release scripts syntax
  9. Validate Alembic migrations
  10. Prepare git for migration diff
  11. Check migration safety

相关文档

  • 接入方案：docs/ci/代码安全扫描CI集成方案_report.md
  • 白名单配置：.gitleaks.toml
  • 实施指南：docs/ci/IMPLEMENTATION_GUIDE.md
## 概述 在 validate Job 中接入两个安全扫描工具： ### gitleaks 密钥检测（P0 - 阻断模式） - PR 触发时：**增量扫描**，只扫描 PR 改动的文件（origin/base..HEAD） - 主分支 push 时：**全量扫描**，扫描整个仓库 - 发现密钥直接阻断合并，确保敏感信息不会流入代码库 - 配置 `.gitleaks.toml` 白名单，排除示例配置、测试文件、文档等已知无密钥区域 ### pip-audit Python 依赖漏洞扫描（P1 - 告警模式） - 扫描所有 requirements 文件（requirements.txt / requirements-base.txt / requirements-dev.txt / requirements-worker.txt） - 当前为**告警模式**，不阻断 CI（先跑 1-2 周摸清存量漏洞） - 后续根据漏洞严重程度设置阻断阈值 - 数据源使用 OSV（PyPA 官方推荐） ## validate Job 步骤顺序（更新后） 1. Checkout code 2. Verify CI environment 3. **Secret detection (gitleaks)** ← 新增（最优先，快速发现问题） 4. Install dependencies 5. **Python dependency vulnerability scan (pip-audit)** ← 新增 6. Run code quality checks 7. Run security scan (bandit) 8. Validate release scripts syntax 9. Validate Alembic migrations 10. Prepare git for migration diff 11. Check migration safety ## 相关文档 - 接入方案：`docs/ci/代码安全扫描CI集成方案_report.md` - 白名单配置：`.gitleaks.toml` - 实施指南：`docs/ci/IMPLEMENTATION_GUIDE.md`
xiaoxia added 2 commits 2026-07-13 15:12:15 +08:00
ci: add gitleaks allowlist configuration c29297a032 
- Exclude env examples, test files, docs, node_modules, site-packages
- Add regex patterns for common placeholder values
- Exclude generated lock files and CI config
ci: integrate gitleaks and pip-audit into validate job
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Failing after 1m36s
Details
CI/CD Pipeline / Unit Tests (pull_request) Failing after 40s
Details
CI/CD Pipeline / Integration Tests (pull_request) Failing after 31s
Details
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
Details
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Failing after 1560h0m54s
Details
CI/CD Pipeline / Production Browser E2E (pull_request) Failing after 1560h0m49s
Details
CI/CD Pipeline / Deploy Production (pull_request) Failing after 1560h32m26s
Details
CI/CD Pipeline / Build Production Runtime Images (pull_request) Failing after 1560h32m29s
Details
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 1560h0m51s
Details
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 1560h0m51s
Details
cef402f88d 
- Add gitleaks secret detection step (P0 - blocking)
  - PR mode: incremental scan (only changed files)
  - Push mode: full repository scan
  - Block merge if secrets detected
- Add pip-audit dependency vulnerability scan (P1 - advisory)
  - Scan all requirements files
  - Advisory mode only, no blocking
- Steps placed early in validate job for fast feedback
xiaoxia added 1 commit 2026-07-13 15:12:16 +08:00
ci: integrate gitleaks and pip-audit into validate job
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Failing after 1m36s
Details
CI/CD Pipeline / Unit Tests (pull_request) Failing after 40s
Details
CI/CD Pipeline / Integration Tests (pull_request) Failing after 31s
Details
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
Details
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Failing after 1560h0m54s
Details
CI/CD Pipeline / Production Browser E2E (pull_request) Failing after 1560h0m49s
Details
CI/CD Pipeline / Deploy Production (pull_request) Failing after 1560h32m26s
Details
CI/CD Pipeline / Build Production Runtime Images (pull_request) Failing after 1560h32m29s
Details
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 1560h0m51s
Details
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 1560h0m51s
Details
cef402f88d 
- Add gitleaks secret detection step (P0 - blocking)
  - PR mode: incremental scan (only changed files)
  - Push mode: full repository scan
  - Block merge if secrets detected
- Add pip-audit dependency vulnerability scan (P1 - advisory)
  - Scan all requirements files
  - Advisory mode only, no blocking
- Steps placed early in validate job for fast feedback
xiaoxia reviewed 2026-07-13 15:20:45 +08:00
xiaoxia left a comment
Author
Owner
Copy Link
Copy Source

Approved by 灵应 - CI优化项目组审批

Approved by 灵应 - CI优化项目组审批
xiaoxia added 1 commit 2026-07-13 16:25:09 +08:00
fix(ci): add Chinese mirror for gitleaks download
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Failing after 2m12s
Details
CI/CD Pipeline / Integration Tests (pull_request) Failing after 24s
Details
CI/CD Pipeline / Unit Tests (pull_request) Failing after 21m1s
Details
CI/CD Pipeline / Production Browser E2E (pull_request) Failing after 1559h2m7s
Details
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 1559h2m10s
Details
CI/CD Pipeline / Build Production Runtime Images (pull_request) Failing after 1559h2m12s
Details
CI/CD Pipeline / Deploy Production (pull_request) Failing after 1559h2m10s
Details
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Failing after 1559h2m12s
Details
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
Details
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 1559h33m44s
Details
e557da4d98 
- Gitea runner servers cannot reach GitHub directly (connection timeout)
- Add ghproxy mirror as primary download source
- Add multiple fallback URLs for reliability
- Fix pip-audit exit code handling
- Improve error messages
xiaoxia added 1 commit 2026-07-13 16:26:28 +08:00
docs(ci): add security scanning roadmap document
CI/CD Pipeline / Unit Tests (pull_request) Failing after 18s
Details
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Failing after 21m53s
Details
CI/CD Pipeline / Integration Tests (pull_request) Failing after 16s
Details
CI/CD Pipeline / Production Browser E2E (pull_request) Failing after 1558h41m56s
Details
CI/CD Pipeline / Deploy Production (pull_request) Failing after 1558h41m58s
Details
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 1558h41m59s
Details
CI/CD Pipeline / Build Production Runtime Images (pull_request) Failing after 1558h42m1s
Details
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Failing after 1558h42m1s
Details
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
Details
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 1559h13m34s
Details
e9ad12e072 
- Phase 0: baseline (bandit, black, isort, flake8) - existing
- Phase 1: security foundation (gitleaks P0, pip-audit P1) - in progress
- Phase 2: quality & frontend (npm audit P1, vulture P2) - planned
- Phase 3: deep quality (mypy, semgrep, dep auto-update) - future
- Access principles, progress tracking table
xiaoxia added 1 commit 2026-07-13 16:39:45 +08:00
docs(ci): add first security scan briefing report
CI/CD Pipeline / Unit Tests (pull_request) Failing after 16s
Details
CI/CD Pipeline / Frontend Lint (pull_request) Failing after 17s
Details
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Failing after 2m37s
Details
CI/CD Pipeline / Integration Tests (pull_request) Failing after 25s
Details
CI/CD Pipeline / Production Browser E2E (pull_request) Failing after 1558h38m45s
Details
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 1558h38m45s
Details
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 1558h38m45s
Details
CI/CD Pipeline / Build Production Runtime Images (pull_request) Failing after 1558h38m46s
Details
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Failing after 1558h38m46s
Details
CI/CD Pipeline / Deploy Production (pull_request) Failing after 1559h10m20s
Details
cd66aace5b 
- Overview of gitleaks, pip-audit, vulture status
- Known issues: new runner stuck, black format failure
- Next steps and action items
xiaoxia added 1 commit 2026-07-13 16:50:47 +08:00
fix(ci): add more Chinese mirrors for gitleaks download + graceful degradation
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Failing after 9s
Details
CI/CD Pipeline / Unit Tests (pull_request) Failing after 5s
Details
CI/CD Pipeline / Frontend Lint (pull_request) Failing after 7s
Details
CI/CD Pipeline / Integration Tests (pull_request) Failing after 7s
Details
CI/CD Pipeline / Production Browser E2E (pull_request) Failing after 1558h39m20s
Details
CI/CD Pipeline / Build Production Runtime Images (pull_request) Failing after 1558h39m22s
Details
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Failing after 1558h39m22s
Details
CI/CD Pipeline / Deploy Production (pull_request) Failing after 1558h39m21s
Details
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 1558h39m21s
Details
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 1559h10m56s
Details
10cb082560
xiaoxia added 1 commit 2026-07-13 16:51:47 +08:00
docs(ci): update security scan briefing with gitleaks fix details + PR264 review
CI/CD Pipeline / Unit Tests (pull_request) Failing after 9s
Details
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Failing after 10s
Details
CI/CD Pipeline / Integration Tests (pull_request) Failing after 26s
Details
CI/CD Pipeline / Production Browser E2E (pull_request) Failing after 1558h36m40s
Details
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 1558h36m41s
Details
CI/CD Pipeline / Build Production Runtime Images (pull_request) Failing after 1558h36m41s
Details
CI/CD Pipeline / Deploy Production (pull_request) Failing after 1558h36m41s
Details
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Failing after 1558h36m41s
Details
CI/CD Pipeline / Frontend Lint (pull_request) Has been skipped
Details
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 1559h8m16s
Details
90f3c0694c
xiaoxia added 1 commit 2026-07-13 16:54:22 +08:00
fix(ci): fix gitleaks.toml regex syntax - glob to regex patterns
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Failing after 10s
Details
CI/CD Pipeline / Frontend Lint (pull_request) Failing after 10s
Details
CI/CD Pipeline / Unit Tests (pull_request) Failing after 18s
Details
CI/CD Pipeline / Integration Tests (pull_request) Failing after 25s
Details
CI/CD Pipeline / Production Browser E2E (pull_request) Failing after 1558h35m52s
Details
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 1558h35m53s
Details
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 1558h35m53s
Details
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Failing after 1558h35m54s
Details
CI/CD Pipeline / Deploy Production (pull_request) Failing after 1559h7m28s
Details
CI/CD Pipeline / Build Production Runtime Images (pull_request) Failing after 1559h7m29s
Details
a141cd56ff
xiaoxia added 1 commit 2026-07-13 16:56:42 +08:00
fix(ci): use triple-quoted strings in gitleaks.toml to avoid quote parsing errors
CI/CD Pipeline / Unit Tests (pull_request) Failing after 15s
Details
CI/CD Pipeline / Frontend Lint (pull_request) Failing after 14s
Details
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Failing after 17s
Details
CI/CD Pipeline / Integration Tests (pull_request) Failing after 8s
Details
CI/CD Pipeline / Production Browser E2E (pull_request) Failing after 1558h33m24s
Details
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 1558h33m24s
Details
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 1558h33m24s
Details
CI/CD Pipeline / Build Production Runtime Images (pull_request) Failing after 1558h33m25s
Details
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Failing after 1558h33m25s
Details
CI/CD Pipeline / Deploy Production (pull_request) Failing after 1559h4m59s
Details
b4890da58c
xiaoxia closed this pull request 2026-07-15 07:39:14 +08:00
Some checks are pending
CI/CD Pipeline / Unit Tests (pull_request) Failing after 15s
Details
CI/CD Pipeline / Frontend Lint (pull_request) Failing after 14s
Details
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Failing after 17s
Details
CI/CD Pipeline / Integration Tests (pull_request) Failing after 8s
Details
CI/CD Pipeline / Production Browser E2E (pull_request) Failing after 1558h33m24s
Details
CI/CD Pipeline / Staging API Integration Tests (pull_request) Failing after 1558h33m24s
Details
CI/CD Pipeline / Staging E2E Tests (pull_request) Failing after 1558h33m24s
Details
CI/CD Pipeline / Build Production Runtime Images (pull_request) Failing after 1558h33m25s
Details
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (pull_request) Failing after 1558h33m25s
Details
CI/CD Pipeline / Deploy Production (pull_request) Failing after 1559h4m59s
Details

Pull request closed

Please reopen this pull request to perform a merge.
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
architecture
架构设计
 backend
后端相关
 blocked
阻塞中
 bug
Bug修复
 ci-bug
CI本身的bug
 ci-optimization
CI优化类任务
 ci-pipeline
CI流水线/workflow相关
 cleanup
代码清理
 devops
运维/部署/CI
 documentation
文档相关
 feature
新功能
 frontend
前端相关
 P0
阻塞性，必须立即处理
 P1
高优先级
 P2
中优先级
 P3
低优先级/后考虑
 question
需要讨论确认
 ready
PR CI通过，可合并
 refactor
代码重构
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
auto-approve-bot frontend-dev xiaoxia
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: xiaoxia/xiaoxia-saas#256
Delete Branch "feat/ci-security-scanning"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?