Commit Graph

25 Commits

Author SHA1 Message Date
xiaoxia 44402f01d6 fix(worker): ffmpeg改用apt安装,彻底解决下载慢+缓存+语法+清理问题 (#689)
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Build Staging Web Image (push) Successful in 29s
CI/CD Pipeline / Validate Code Quality And Tests (push) Successful in 2m20s
CI/CD Pipeline / Frontend Unit Tests (push) Successful in 2m31s
CI/CD Pipeline / Build Staging API Image (push) Successful in 2m42s
CI/CD Pipeline / Unit Tests (push) Successful in 2m40s
CI/CD Pipeline / Frontend Lint (push) Successful in 2m57s
CI/CD Pipeline / Integration Tests (push) Successful in 1m5s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 28m58s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 4m12s
CI/CD Pipeline / ACR Image Cleanup (push) Successful in 46s
CI/CD Pipeline / Staging E2E Tests (push) Successful in 20m19s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 22m33s
Co-authored-by: xiaoxia <dev@xiaoxiajianji.com>
Co-committed-by: xiaoxia <dev@xiaoxiajianji.com>
2026-07-21 22:19:55 +08:00
xiaoxia c424ffcc66 fix(worker): ffmpeg下载脚本改为纯sh兼容,修复dash数组语法错误 (#688)
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Build Staging Web Image (push) Successful in 38s
CI/CD Pipeline / Frontend Unit Tests (push) Successful in 2m11s
CI/CD Pipeline / Validate Code Quality And Tests (push) Successful in 2m17s
CI/CD Pipeline / Build Staging API Image (push) Successful in 2m19s
CI/CD Pipeline / Unit Tests (push) Successful in 2m36s
CI/CD Pipeline / Frontend Lint (push) Successful in 3m1s
CI/CD Pipeline / Integration Tests (push) Successful in 1m3s
CI/CD Pipeline / Build Staging Worker Image (push) Has been cancelled
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been cancelled
CI/CD Pipeline / Staging E2E Tests (push) Has been cancelled
CI/CD Pipeline / Staging API Integration Tests (push) Has been cancelled
CI/CD Pipeline / ACR Image Cleanup (push) Has been cancelled
Co-authored-by: xiaoxia <dev@xiaoxiajianji.com>
Co-committed-by: xiaoxia <dev@xiaoxiajianji.com>
2026-07-21 20:20:55 +08:00
xiaoxia cd27bd087f fix(worker): ffmpeg下载增加完整性校验+多源重试+坏包自动清缓存 (#686)
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Build Staging API Image (push) Successful in 25s
CI/CD Pipeline / Build Staging Web Image (push) Successful in 57s
CI/CD Pipeline / Frontend Unit Tests (push) Successful in 1m48s
CI/CD Pipeline / Frontend Lint (push) Successful in 2m1s
CI/CD Pipeline / Unit Tests (push) Successful in 2m36s
CI/CD Pipeline / Validate Code Quality And Tests (push) Successful in 2m48s
CI/CD Pipeline / Build Staging Worker Image (push) Failing after 3m33s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (push) Has been skipped
CI/CD Pipeline / Integration Tests (push) Successful in 1m4s
Co-authored-by: xiaoxia <dev@xiaoxiajianji.com>
Co-committed-by: xiaoxia <dev@xiaoxiajianji.com>
2026-07-21 19:56:53 +08:00
xiaoxia faf018030f perf(ci): Dockerfile优化 - cache mount加速 + API多阶段瘦身 (#679)
CI/CD Pipeline / Check if frontend-only change (push) Has been skipped
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Build Staging Web Image (push) Successful in 45s
CI/CD Pipeline / Unit Tests (push) Successful in 2m22s
CI/CD Pipeline / Frontend Lint (push) Successful in 2m45s
CI/CD Pipeline / Build Staging API Image (push) Successful in 2m51s
CI/CD Pipeline / Frontend Unit Tests (push) Failing after 2m21s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 3m2s
CI/CD Pipeline / Integration Tests (push) Failing after 1m3s
CI/CD Pipeline / Build Staging Worker Image (push) Has been cancelled
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been cancelled
CI/CD Pipeline / Staging E2E Tests (push) Has been cancelled
CI/CD Pipeline / Staging API Integration Tests (push) Has been cancelled
CI/CD Pipeline / ACR Image Cleanup (push) Has been cancelled
Co-authored-by: xiaoxia <dev@xiaoxiajianji.com>
Co-committed-by: xiaoxia <dev@xiaoxiajianji.com>
2026-07-21 14:00:29 +08:00
xiaoxia 535e525a0e fix(worker): 修复瘦身后shared模块无法导入的PYTHONPATH问题
CI/CD Pipeline / Validate Code Quality And Tests (push) Successful in 1m48s
CI/CD Pipeline / Unit Tests (push) Successful in 1m50s
CI/CD Pipeline / Integration Tests (push) Successful in 1m9s
CI/CD Pipeline / Frontend Lint (push) Failing after 16s
CI/CD Pipeline / Build Staging API Image (push) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (push) Has been skipped
CI/CD Pipeline / Build Staging Web Image (push) Has been skipped
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
PR #343: 修复Worker镜像瘦身后shared模块无法导入的PYTHONPATH问题

将ENV PYTHONPATH=/app改为ENV PYTHONPATH=/app:/app/packages,解决ModuleNotFoundError: No module named "shared"问题。
2026-07-15 09:16:35 +08:00
xiaoxia ca8b945054 perf(ci): Worker镜像瘦身 - 多阶段构建+ffmpeg静态替换+Python依赖瘦身
CI/CD Pipeline / Frontend Lint (push) Successful in 1m4s
CI/CD Pipeline / Unit Tests (push) Successful in 1m36s
CI/CD Pipeline / Validate Code Quality And Tests (push) Successful in 1m38s
CI/CD Pipeline / Build Production API Image (push) Has been skipped
CI/CD Pipeline / Build Production Worker Image (push) Has been skipped
CI/CD Pipeline / Build Production Web Image (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Build Staging Web Image (push) Successful in 44s
CI/CD Pipeline / Integration Tests (push) Successful in 1m13s
CI/CD Pipeline / Build Staging API Image (push) Successful in 10m20s
CI/CD Pipeline / Build Staging Worker Image (push) Successful in 21m55s
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (push) Successful in 9m14s
CI/CD Pipeline / Staging E2E Tests (push) Failing after 1m0s
CI/CD Pipeline / Staging API Integration Tests (push) Successful in 2m52s
2026-07-15 07:49:15 +08:00
xiaoxia fecc786d4a feat(worker): 支持 WORKER_CONCURRENCY 环境变量控制 Celery 并发数 (#208)
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 9s
CI/CD Pipeline / Frontend Lint (push) Successful in 2m30s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
2026-07-10 22:07:41 +08:00
xiaoxia 4ab641f765 feat(docker): 将 APP_VERSION 构建进镜像,解决 Watchtower 更新后版本号显示不准的问题 (#200)
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 9s
CI/CD Pipeline / Frontend Lint (push) Successful in 2m16s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Has been skipped
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
2026-07-10 22:07:25 +08:00
lingying 1bc3ff855f fix(docker): 修复 worker 镜像在 Debian Trixie 下构建失败
CI/CD Pipeline / Build Production Runtime Images (push) Has been skipped
CI/CD Pipeline / Deploy Production (push) Has been skipped
CI/CD Pipeline / Production Browser E2E (push) Has been skipped
CI/CD Pipeline / Staging E2E Tests (push) Failing after 14h42m31s
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Failing after 14h51m59s
CI/CD Pipeline / Frontend Lint (push) Failing after 14h55m35s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 14h55m35s
- libgl1-mesa-glx 在 Debian 13 (Trixie) 中已被移除
- 替换为 libgl1,兼容 Bookworm 和 Trixie
2026-07-08 21:58:53 +08:00
lingying 92c71fa57b ci(foundation): 所有Dockerfile基础镜像替换为私有Registry
CI/CD Pipeline / Validate Code Quality And Tests (push) Has been cancelled
CI/CD Pipeline / Frontend Lint (push) Has been cancelled
CI/CD Pipeline / Build & Push Staging (Watchtower auto-deploy) (push) Has been cancelled
CI/CD Pipeline / Staging E2E Tests (push) Has been cancelled
CI/CD Pipeline / Build Production Runtime Images (push) Has been cancelled
CI/CD Pipeline / Deploy Production (push) Has been cancelled
CI/CD Pipeline / Production Browser E2E (push) Has been cancelled
- api/worker/ci: python:3.12-slim -> 私有Registry
- web/node: node:20 -> 私有Registry
- web/nginx: nginx:alpine -> 私有Registry
- 彻底摆脱公网镜像依赖,构建更快更稳
2026-07-08 20:34:28 +08:00
灵应 bf6ae9c430 refactor: API/Worker 依赖拆分 - 移除 API 镜像中的重依赖
Deploy / Build Production Runtime Images (push) Has been skipped
Deploy / Deploy Production (push) Has been skipped
Deploy / Production Browser E2E (push) Has been skipped
Deploy / Staging E2E Tests (push) Failing after 93h39m59s
Deploy / Deploy Staging (push) Failing after 94h0m21s
CI/CD Pipeline / Frontend Lint (push) Failing after 94h0m28s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 94h0m28s
- 新建 requirements-worker.txt,包含 numpy/scipy/opencv/Pillow/ffmpeg-python
- requirements-base.txt 移除视频处理大包,仅保留公共基础依赖
- worker.Dockerfile 新增 requirements-worker.txt 安装层
- API Dockerfile 无需修改,自动不再包含重包
- 验证:API/packages 代码无 numpy/scipy/cv2 导入
- 全量测试 994 passed, 0 failed

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-05 14:54:04 +08:00
xiaoxia bd18acd040 fix: Deploy阶段全面修复 - E2E服务器错位/Worker venv统一/内网Registry登录/生产构建缓存
Deploy / Staging E2E Tests (push) Has been skipped
Deploy / Build Production Runtime Images (push) Has been skipped
Deploy / Deploy Production (push) Has been skipped
Deploy / Production Browser E2E (push) Has been skipped
Deploy / Deploy Staging (push) Failing after 135h54m40s
CI/CD Pipeline / Frontend Lint (push) Failing after 135h54m55s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 135h54m55s
2026-07-03 20:59:29 +08:00
xiaoxia bcb1663de4 ci: 启用BuildKit分布式缓存+Gitea Registry,优化构建速度
Deploy / Staging E2E Tests (push) Has been skipped
Deploy / Build Production Runtime Images (push) Has been skipped
Deploy / Deploy Production (push) Has been skipped
Deploy / Production Browser E2E (push) Has been skipped
Deploy / Deploy Staging (push) Failing after 138h40m13s
CI/CD Pipeline / Frontend Lint (push) Failing after 138h40m24s
CI/CD Pipeline / Validate Code Quality And Tests (push) Failing after 138h40m24s
- 拆分 requirements.txt: 稳定大包放 requirements-base.txt(numpy/scipy/opencv/Pillow/sqlalchemy 等),业务依赖留 requirements.txt
- 优化 Dockerfile: 分两层安装依赖,基础层缓存命中率高
- CI 改用 docker buildx build,缓存存 Gitea Container Registry
- 启用 Gitea Packages/Container Registry(git.xiaoxiajianji.com)
- 镜像仍推内网 Registry(172.30.18.198:5000),缓存独立存 Gitea Registry
2026-07-03 18:13:49 +08:00
CI Test 46d9b9edfe fix: add Chinese apt mirrors to both Dockerfiles for faster builds
Tests / test (push) Failing after 2s
Tests / lint (push) Failing after 0s
CI/CD Pipeline / Validate Code Quality And Tests (push) Has been cancelled
CI/CD Pipeline / Frontend Lint (push) Has been cancelled
Deploy / Deploy Staging (push) Has been cancelled
Deploy / Build Production Runtime Images (push) Has been cancelled
Deploy / Deploy Production (push) Has been cancelled
Deploy / Production Browser E2E (push) Has been cancelled
2026-06-27 20:38:37 +08:00
Celery Worker Fix Agent 8210806632 fix: 修复 Celery Worker 任务注册和导入问题
CI/CD Pipeline / Validate Code Quality And Tests (pull_request) Has been cancelled
CI/CD Pipeline / Frontend Lint (pull_request) Has been cancelled
1. 修复 Worker 启动入口: celery_app → worker_app.celery_app
   - 旧 celery_app.py 不导入任何任务模块,导致 Worker 注册 0 个任务
   - 删除旧版 apps/worker/celery_app.py,统一使用 worker_app/celery_app.py

2. 为缺少装饰器的任务补充 @celery_app.task:
   - classification.py: classify_asset() 添加装饰器
   - generation.py: generate_video() 添加装饰器,修正签名匹配 API 调用方式

3. 确保 voice_extraction 任务被正确注册:
   - 添加 voice_extraction 到 celery_app imports
   - 修复 voice_extraction.py 中错误的相对导入 (.celery_app → worker_app.celery_app)
   - 修复 dedup.py 中指向已删除模块的导入

4. 修复 worker_app/celery_app.py:
   - 添加 broker_connection_retry_on_startup=True
   - imports 中添加 voice_extraction 和 dedup 模块

5. 修复 Dockerfile:
   - CMD 改为 celery -A worker_app.celery_app
   - 添加非 root 用户 celery 运行 Worker

6. 新建 packages/shared/config.py 和 storage.py 兼容层
   - 为 worker 任务模块提供统一的 config/storage 访问入口
2026-06-27 19:31:07 +08:00
CI Bot 11494918eb fix: resolve all workspace removal breakage - stub permissions, quota, ports, adapters
- Remove Workspace from domain/__init__.py exports
- Stub domain/permissions.py (all checks pass)
- Stub domain/quota.py (all checks pass)
- Create missing port stubs: workspace_member_repository, workspace_repository, workspace_invitation_repository
- Create missing adapter stubs: workspace_repository, workspace_invitation_repository
- Create routes/permissions.py stub (require_workspace_member no-op)
- Fix .env Redis URL encoding on production server
2026-06-27 09:16:53 +08:00
xiaoxia e915c0d6df fix: 创建 API 和 Worker Dockerfiles
- 创建 infra/docker/api.Dockerfile 用于 FastAPI 应用
- 创建 infra/docker/worker.Dockerfile 用于 Celery Worker
- 使用 python:3.12-slim-bookworm 作为基础镜像
- Worker 包含 ffmpeg 和 opencv 依赖(用于视频处理)
- API 包含 PostgreSQL 开发库(用于 psycopg2 编译)
- 分离镜像减少不必要的依赖
2026-06-27 00:04:15 +08:00
Xiaoxia AI 816d98175a fix(ops): expose release version and limit worker concurrency 2026-06-23 07:52:19 +08:00
Xiaoxia AI 46bd774742 build(docker): include maintenance scripts 2026-06-21 11:32:54 +08:00
Xiaoxia AI 0809a079c5 refactor(db): introduce alembic schema baseline 2026-06-21 06:48:16 +08:00
Xiaoxia AI 1fb8463bab fix(worker): speed up ffmpeg package install 2026-06-20 22:52:23 +08:00
Xiaoxia AI 3afe9689f7 feat(worker): generate real mp4 outputs 2026-06-20 22:42:07 +08:00
Xiaoxia AI 2c86106c08 fix: switch docker pip mirror to aliyun 2026-06-15 18:53:37 +08:00
Xiaoxia AI ba4e672bf8 fix: use domestic pip mirror for docker builds 2026-06-15 18:52:33 +08:00
Xiaoxia AI b5a62ee9a3 feat: initial SaaS scaffold
- domain: User, Workspace, Project, AssetLibrary, Asset, IngestJob
- ports: repository interfaces
- application: use cases
- adapters: in-memory
- API: FastAPI 5 routes
- worker: Celery ingest_asset
- tests: 4 passing
2026-06-15 15:17:40 +08:00