Compare commits

..

1 Commits

Author SHA1 Message Date
xiaoxia 8437d4060b feat(products): 成片库视频内联播放,去掉弹窗 + 自适应比例
CI/CD Pipeline / Check push changed paths (pull_request) Has been skipped
CI/CD Pipeline / Dedup Check - skip PR tests when covered by push pipeline (pull_request) Successful in 1s
CI/CD Pipeline / Build Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Build Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging API Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Web Image (pull_request) Has been skipped
CI/CD Pipeline / Retag skipped Staging Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Staging (Watchtower auto-deploy) (pull_request) Has been skipped
CI/CD Pipeline / Staging E2E Tests (pull_request) Has been skipped
CI/CD Pipeline / Staging API Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / ACR Image Cleanup (pull_request) Has been skipped
CI/CD Pipeline / Check if frontend-only change (pull_request) Successful in 3m35s
Preview Deploy / Deploy Preview Environment (pull_request) Successful in 4m7s
CI/CD Pipeline / Validate - Type Check (mypy) (pull_request) Successful in 4m5s
PR Automation / Auto Merge on CI Green + Approved (pull_request) Successful in 4m40s
CI/CD Pipeline / Validate - Migration (alembic) (pull_request) Successful in 5m18s
PR Automation / Auto Approve on CI Green (pull_request) Successful in 6m13s
CI/CD Pipeline / Unit Tests (pull_request) Has been skipped
CI/CD Pipeline / PR Build API Image (pull_request) Has been skipped
CI/CD Pipeline / PR Build Worker Image (pull_request) Has been skipped
AI Code Review / AI Code Review (pull_request) Successful in 10m1s
CI/CD Pipeline / Frontend Unit Tests (pull_request) Successful in 2m9s
CI/CD Pipeline / PR Build Web Image (pull_request) Successful in 2m16s
CI/CD Pipeline / Validate - Code Quality (pull_request) Successful in 10m52s
CI/CD Pipeline / Integration Tests (pull_request) Has been skipped
CI/CD Pipeline / Frontend Lint (pull_request) Successful in 3m7s
CI/CD Pipeline / Build Production API Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Worker Image (pull_request) Has been skipped
CI/CD Pipeline / Build Production Web Image (pull_request) Has been skipped
CI/CD Pipeline / Deploy Production (pull_request) Has been skipped
CI/CD Pipeline / Canary Release to Production (pull_request) Has been skipped
CI/CD Pipeline / Production Browser E2E (pull_request) Has been skipped
CI/CD Pipeline / CI Gate (pull_request) Successful in 43s
Preview Cleanup / Cleanup Preview Environment (pull_request) Successful in 4m1s
ACR Cleanup / ACR Image Cleanup (pull_request_target) Waiting to run
- ProductCard: 缩略图区域改为 <video> 内联播放,点击播放按钮原生 controls
- 视频比例通过 onLoadedMetadata 动态获取,获取不到时默认 16:9
- ProductLibrary: 移除 playingProduct 状态和 VideoPlayer 弹窗组件
- products.css: .xx-product-thumb 去掉固定 9:16 aspect-ratio,
  新增 .xx-product-thumb-video 样式(object-fit: contain)
- 卡片网格保持 4 列,卡片高度自适应视频内容
2026-08-31 16:40:15 +08:00
51 changed files with 362 additions and 2168 deletions
+151 -153
View File
@@ -81,6 +81,12 @@ jobs:
skip_backend: ${{ steps.check.outputs.skip_backend }}
skip_frontend: ${{ steps.check.outputs.skip_frontend }}
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sfH "Authorization: token $GITHUB_TOKEN" -o /tmp/_ci_checkout.sh "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" && bash /tmp/_ci_checkout.sh
- name: Check changed files
id: check
shell: bash
@@ -119,14 +125,14 @@ jobs:
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
curl -sfH "Authorization: token ${GITHUB_TOKEN:-$GITEA_TOKEN}" -o /tmp/_ci_trace.py "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/ci_trace_report.py?ref=${GITHUB_SHA}" 2>/dev/null && python3 /tmp/_ci_trace.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
validate-style:
validate-code-quality:
needs: dedupe-check
if: always() && needs.dedupe-check.outputs.skip_tests != 'true'
name: Validate - Style
name: Validate - Code Quality
runs-on: ci-l2
timeout-minutes: 6
timeout-minutes: 8
env:
PIP_CACHE_DIR: /root/.cache/pip
PIP_NO_CACHE_DIR: ''
@@ -150,12 +156,11 @@ jobs:
run: bash scripts/ci/step_timer_start.sh
- name: Cache pip dependencies
uses: actions/cache@v4
continue-on-error: true
with:
path: /root/.cache/pip
key: ${{ runner.os }}-pip-style-${{ hashFiles('requirements*.txt') }}
key: ${{ runner.os }}-pip-codequality-${{ hashFiles('requirements*.txt') }}
restore-keys: |
${{ runner.os }}-pip-style-
${{ runner.os }}-pip-codequality-
${{ runner.os }}-pip-
- name: Install dependencies
shell: sh
@@ -179,9 +184,17 @@ jobs:
[ $i -eq 3 ] && exit 1
sleep 5
done
- name: Run style checks
for i in 1 2 3; do
python3 -m pip install --no-binary :all: black==26.5.1 isort==8.0.1 && break
echo "pip install black/isort 失败,重试 $i/3..."
[ $i -eq 3 ] && exit 1
sleep 5
done
- name: Run code quality and security checks
shell: bash
run: bash scripts/ci/validate_style.sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: bash scripts/ci/validate_code_quality.sh
- name: Auto-fix formatting (black + isort)
if: failure()
shell: sh
@@ -197,7 +210,7 @@ jobs:
CI_WEBHOOK_URL: ${{ secrets.CI_WEBHOOK_URL }}
run: |
set +e
FAILED_JOB="Validate - Style" python3 scripts/ci_notify_failure.py
FAILED_JOB="Validate - Code Quality" python3 scripts/ci_notify_failure.py
- name: Job duration summary
if: always()
shell: sh
@@ -210,7 +223,7 @@ jobs:
CI_NOTIFY_WEBHOOK: ${{ secrets.CI_NOTIFY_WEBHOOK }}
run: |
set +e
NOTIFY_MODE=failure JOB_NAME="Validate - Style" python3 scripts/ci_notify.py
NOTIFY_MODE=failure JOB_NAME="Validate - Code Quality" python3 scripts/ci_notify.py
- name: Report CI trace
if: always()
shell: sh
@@ -223,16 +236,12 @@ jobs:
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
validate-security:
validate-type-check:
needs: dedupe-check
if: always() && needs.dedupe-check.outputs.skip_tests != 'true'
name: Validate - Security
name: Validate - Type Check (mypy)
runs-on: ci-l2
timeout-minutes: 8
env:
PIP_CACHE_DIR: /root/.cache/pip
PIP_NO_CACHE_DIR: ''
permissions:
contents: read
steps:
@@ -251,116 +260,6 @@ jobs:
- name: Record job start time
shell: sh
run: bash scripts/ci/step_timer_start.sh
- name: Cache pip dependencies
uses: actions/cache@v4
continue-on-error: true
with:
path: /root/.cache/pip
key: ${{ runner.os }}-pip-security-${{ hashFiles('requirements*.txt') }}
restore-keys: |
${{ runner.os }}-pip-security-
${{ runner.os }}-pip-
- name: Install dependencies
shell: sh
run: |
set -eu
for i in 1 2 3; do
python3 -m pip install -q -r requirements-base.txt && break
echo "pip install requirements-base.txt 失败,重试 $i/3..."
[ $i -eq 3 ] && exit 1
sleep 5
done
for i in 1 2 3; do
python3 -m pip install -q -r requirements.txt && break
echo "pip install requirements.txt 失败,重试 $i/3..."
[ $i -eq 3 ] && exit 1
sleep 5
done
for i in 1 2 3; do
python3 -m pip install -q -r requirements-dev.txt && break
echo "pip install requirements-dev.txt 失败,重试 $i/3..."
[ $i -eq 3 ] && exit 1
sleep 5
done
- name: Run security checks
shell: bash
env:
GITHUB_TOKEN: ${{ github.token }}
run: bash scripts/ci/validate_security.sh
- name: CI failure notification
if: failure()
shell: sh
env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
CI_WEBHOOK_URL: ${{ secrets.CI_WEBHOOK_URL }}
run: |
set +e
FAILED_JOB="Validate - Security" python3 scripts/ci_notify_failure.py
- name: Job duration summary
if: always()
shell: sh
run: bash scripts/ci/step_timer_end.sh
- name: Notify on failure
continue-on-error: true
if: failure()
shell: sh
env:
CI_NOTIFY_WEBHOOK: ${{ secrets.CI_NOTIFY_WEBHOOK }}
run: |
set +e
NOTIFY_MODE=failure JOB_NAME="Validate - Security" python3 scripts/ci_notify.py
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
validate-python:
needs: dedupe-check
if: always() && needs.dedupe-check.outputs.skip_tests != 'true'
name: Validate - Python (mypy + alembic)
runs-on: ci-l2
timeout-minutes: 10
env:
PIP_CACHE_DIR: /root/.cache/pip
PIP_NO_CACHE_DIR: ''
DATABASE_URL: postgresql+psycopg://postgres:postgres@host.docker.internal:5432/xiaoxia_saas
USE_IN_MEMORY_DB: 'false'
CI_USE_SHARED_PG: 'true'
permissions:
contents: read
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sfH "Authorization: token $GITHUB_TOKEN" -o /tmp/_ci_checkout.sh "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" && bash /tmp/_ci_checkout.sh
- name: Zombie run selfcheck
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
GITHUB_RUN_ID: ${{ github.run_id }}
run: bash scripts/ci/ci_run_selfcheck.sh
- name: Record job start time
shell: sh
run: bash scripts/ci/step_timer_start.sh
- name: Cache pip dependencies
uses: actions/cache@v4
continue-on-error: true
with:
path: /root/.cache/pip
key: ${{ runner.os }}-pip-python-${{ hashFiles('requirements*.txt') }}
restore-keys: |
${{ runner.os }}-pip-python-
${{ runner.os }}-pip-
- name: Install dependencies
shell: sh
run: |
@@ -386,9 +285,6 @@ jobs:
- name: Run mypy type check
shell: bash
run: bash scripts/ci/validate_mypy.sh
- name: Run alembic migration validation
shell: bash
run: bash scripts/ci/validate_migration.sh
- name: CI failure notification
if: failure()
shell: sh
@@ -397,7 +293,7 @@ jobs:
CI_WEBHOOK_URL: ${{ secrets.CI_WEBHOOK_URL }}
run: |
set +e
FAILED_JOB="Validate - Python (mypy + alembic)" python3 scripts/ci_notify_failure.py
FAILED_JOB="Validate - Type Check (mypy)" python3 scripts/ci_notify_failure.py
- name: Job duration summary
if: always()
shell: sh
@@ -410,7 +306,7 @@ jobs:
CI_NOTIFY_WEBHOOK: ${{ secrets.CI_NOTIFY_WEBHOOK }}
run: |
set +e
NOTIFY_MODE=failure JOB_NAME="Validate - Python (mypy + alembic)" python3 scripts/ci_notify.py
NOTIFY_MODE=failure JOB_NAME="Validate - Type Check (mypy)" python3 scripts/ci_notify.py
- name: Report CI trace
if: always()
shell: sh
@@ -423,6 +319,92 @@ jobs:
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
validate-migration:
needs: dedupe-check
if: always() && needs.dedupe-check.outputs.skip_tests != 'true'
name: Validate - Migration (alembic)
runs-on: ci-l2
timeout-minutes: 8
permissions:
contents: read
env:
DATABASE_URL: postgresql+psycopg://postgres:postgres@host.docker.internal:5432/xiaoxia_saas
USE_IN_MEMORY_DB: 'false'
CI_USE_SHARED_PG: 'true'
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sfH "Authorization: token $GITHUB_TOKEN" -o /tmp/_ci_checkout.sh "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" && bash /tmp/_ci_checkout.sh
- name: Zombie run selfcheck
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
GITHUB_RUN_ID: ${{ github.run_id }}
run: bash scripts/ci/ci_run_selfcheck.sh
- name: Record job start time
shell: sh
run: bash scripts/ci/step_timer_start.sh
- name: Install dependencies
shell: sh
run: |
set -eu
for i in 1 2 3; do
python3 -m pip install -q -r requirements-base.txt && break
echo "pip install requirements-base.txt 失败,重试 $i/3..."
[ $i -eq 3 ] && exit 1
sleep 5
done
for i in 1 2 3; do
python3 -m pip install -q -r requirements.txt && break
echo "pip install requirements.txt 失败,重试 $i/3..."
[ $i -eq 3 ] && exit 1
sleep 5
done
for i in 1 2 3; do
python3 -m pip install -q -r requirements-dev.txt && break
echo "pip install requirements-dev.txt 失败,重试 $i/3..."
[ $i -eq 3 ] && exit 1
sleep 5
done
- name: Run alembic migration validation
shell: bash
run: bash scripts/ci/validate_migration.sh
- name: CI failure notification
if: failure()
shell: sh
env:
GITEA_TOKEN: ${{ secrets.GITEA_TOKEN }}
CI_WEBHOOK_URL: ${{ secrets.CI_WEBHOOK_URL }}
run: |
set +e
FAILED_JOB="Validate - Migration (alembic)" python3 scripts/ci_notify_failure.py
- name: Job duration summary
if: always()
shell: sh
run: bash scripts/ci/step_timer_end.sh
- name: Notify on failure
continue-on-error: true
if: failure()
shell: sh
env:
CI_NOTIFY_WEBHOOK: ${{ secrets.CI_NOTIFY_WEBHOOK }}
run: |
set +e
NOTIFY_MODE=failure JOB_NAME="Validate - Migration (alembic)" python3 scripts/ci_notify.py
- name: Report CI trace
if: always()
shell: sh
env:
AGENTLOOP_LICENSE_KEY: ${{ secrets.AGENTLOOP_LICENSE_KEY }}
run: |
STATUS="ok"
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
unit-tests:
needs: [check-frontend-only, dedupe-check]
@@ -459,7 +441,6 @@ jobs:
run: bash scripts/ci/step_install_ffmpeg.sh
- name: Cache pip dependencies
uses: actions/cache@v4
continue-on-error: true
with:
path: /root/.cache/pip
key: ${{ runner.os }}-pip-unittests-${{ hashFiles('requirements*.txt') }}
@@ -514,6 +495,9 @@ jobs:
needs:
- check-frontend-only
- dedupe-check
- validate-code-quality
- validate-type-check
- validate-migration
env:
DATABASE_URL: postgresql+psycopg://postgres:postgres@host.docker.internal:5432/xiaoxia_saas
USE_IN_MEMORY_DB: 'false'
@@ -668,7 +652,6 @@ jobs:
run: bash scripts/ci/step_timer_start.sh
- name: Cache npm dependencies
uses: actions/cache@v4
continue-on-error: true
with:
path: /root/.npm
key: ${{ runner.os }}-npm-vitest-${{ hashFiles('apps/web/package-lock.json') }}
@@ -875,13 +858,19 @@ jobs:
skip_backend: ${{ steps.check.outputs.skip_backend }}
skip_frontend: ${{ steps.check.outputs.skip_frontend }}
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sfH "Authorization: token $GITHUB_TOKEN" -o /tmp/_ci_checkout.sh "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" && bash /tmp/_ci_checkout.sh
- name: Check changed paths
id: check
shell: bash
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sfH "Authorization: token $GITHUB_TOKEN" -o /tmp/_ci_push_paths.sh "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/ci_push_paths.sh?ref=${GITHUB_SHA}" && bash /tmp/_ci_push_paths.sh
bash scripts/ci/ci_push_paths.sh
- name: Report CI trace
if: always()
shell: sh
@@ -892,7 +881,7 @@ jobs:
[ ${{ job.status }} = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
curl -sfH "Authorization: token ${GITHUB_TOKEN:-$GITEA_TOKEN}" -o /tmp/_ci_trace.py "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/ci_trace_report.py?ref=${GITHUB_SHA}" 2>/dev/null && python3 /tmp/_ci_trace.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
build-staging:
name: Build Staging ${{ matrix.service_display }} Image
@@ -1150,7 +1139,9 @@ jobs:
- check-push-paths
- build-staging
- retag-staging-skipped
if: (!cancelled()) && github.event_name == 'push' && (github.ref_name == 'main' || github.ref_name == 'develop')
# 显式 success() 状态检查:上游 build/retag 被路径过滤 if 跳过(skipped)时不阻塞本 job
# 上游真正失败时仍然阻断(act_runner 对无状态函数的 if 隐式包 success(),纯 skipped 也会连带跳过)
if: success() && github.event_name == 'push' && (github.ref_name == 'main' || github.ref_name == 'develop')
steps:
- name: Checkout code
shell: sh
@@ -1408,9 +1399,8 @@ jobs:
runs-on: runtime-builder
timeout-minutes: ${{ matrix.timeout }}
needs:
- validate-style
- validate-security
- validate-python
- validate-code-quality
- validate-type-check
- unit-tests
- frontend-lint
- frontend-unit-test
@@ -1896,9 +1886,9 @@ jobs:
if: always() && github.event_name == 'pull_request'
needs:
- check-frontend-only
- validate-style
- validate-security
- validate-python
- validate-code-quality
- validate-type-check
- validate-migration
- unit-tests
- integration-tests
- frontend-lint
@@ -1906,6 +1896,14 @@ jobs:
- build-pr
timeout-minutes: 3
steps:
- name: Checkout code
shell: sh
env:
GITHUB_TOKEN: ${{ github.token }}
run: |
curl -sfH "Authorization: token $GITHUB_TOKEN" -o /tmp/_ci_checkout.sh \
"${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/step_checkout.sh?ref=${GITHUB_SHA}" && bash /tmp/_ci_checkout.sh
- name: Evaluate CI Gate
id: gate
shell: bash
@@ -1914,9 +1912,9 @@ jobs:
PR_NUMBER: ${{ github.event.pull_request.number }}
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
RESULT_CHECK_FRONTEND: ${{ needs.check-frontend-only.result }}
RESULT_STYLE: ${{ needs.validate-style.result }}
RESULT_SECURITY: ${{ needs.validate-security.result }}
RESULT_PYTHON: ${{ needs.validate-python.result }}
RESULT_CODE_QUALITY: ${{ needs.validate-code-quality.result }}
RESULT_TYPE_CHECK: ${{ needs.validate-type-check.result }}
RESULT_MIGRATION: ${{ needs.validate-migration.result }}
RESULT_UNIT_TESTS: ${{ needs.unit-tests.result }}
RESULT_INTEGRATION: ${{ needs.integration-tests.result }}
RESULT_FRONTEND_LINT: ${{ needs.frontend-lint.result }}
@@ -1928,9 +1926,9 @@ jobs:
echo ""
echo "各job结果:"
echo " check-frontend-only: $RESULT_CHECK_FRONTEND"
echo " validate-style: $RESULT_STYLE"
echo " validate-security: $RESULT_SECURITY"
echo " validate-python: $RESULT_PYTHON"
echo " validate-code-quality: $RESULT_CODE_QUALITY"
echo " validate-type-check: $RESULT_TYPE_CHECK"
echo " validate-migration: $RESULT_MIGRATION"
echo " unit-tests: $RESULT_UNIT_TESTS"
echo " integration-tests: $RESULT_INTEGRATION"
echo " frontend-lint: $RESULT_FRONTEND_LINT"
@@ -1970,9 +1968,9 @@ jobs:
# 必填检查项(根据PR类型决定)
# 通用检查(所有PR都必须过)
REQUIRED_GENERAL=(
"validate-style:$RESULT_STYLE"
"validate-security:$RESULT_SECURITY"
"validate-python:$RESULT_PYTHON"
"validate-code-quality:$RESULT_CODE_QUALITY"
"validate-type-check:$RESULT_TYPE_CHECK"
"validate-migration:$RESULT_MIGRATION"
"frontend-lint:$RESULT_FRONTEND_LINT"
"build-pr:$RESULT_BUILD_PR"
"ai-code-review:$AI_REVIEW_STATUS"
@@ -2067,4 +2065,4 @@ jobs:
[ "${{ steps.gate.outputs.gate_result }}" = "success" ] || STATUS="error"
START_TIME=""
[ -f /tmp/ci_job_start_time ] && START_TIME=$(cat /tmp/ci_job_start_time)
curl -sfH "Authorization: token ${GITHUB_TOKEN:-$GITEA_TOKEN}" -o /tmp/_ci_trace.py "${GITHUB_API_URL}/repos/${GITHUB_REPOSITORY}/raw/scripts/ci/ci_trace_report.py?ref=${GITHUB_SHA}" 2>/dev/null && python3 /tmp/_ci_trace.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
python3 scripts/ci/ci_trace_report.py --service xiaoxia-saas-ci --status $STATUS --start-time "$START_TIME" || true
+1 -1
View File
@@ -3,7 +3,7 @@ name: PR Auto Scan
# 作为短作业模式的兜底,防止事件驱动遗漏
on:
schedule:
- cron: "*/15 * * * *" # 每10分钟扫描一次(脚本自带240s墙钟上限,降频减负)
- cron: "*/10 * * * *" # 每10分钟扫描一次(脚本自带240s墙钟上限,降频减负)
workflow_dispatch:
permissions:
@@ -1,23 +0,0 @@
"""add duplicate_rate to generated_videos
Revision ID: 059_duplicate_rate
Revises: 058_uq_asset_lib_project_kind
Create Date: 2026-08-31
"""
import sqlalchemy as sa
from alembic import op
revision = "059_duplicate_rate"
down_revision = "058_uq_asset_lib_project_kind"
branch_labels = None
depends_on = None
def upgrade() -> None:
op.add_column("generated_videos", sa.Column("duplicate_rate", sa.Float(), nullable=True))
def downgrade() -> None:
op.drop_column("generated_videos", "duplicate_rate")
@@ -1,57 +0,0 @@
"""migrate template_segments data to template_clip_configs
Revision ID: 060_migrate_segments
Revises: 059_duplicate_rate
Create Date: 2026-08-31
"""
import sqlalchemy as sa
from alembic import op
revision = "060_migrate_segments"
down_revision = "059_duplicate_rate"
branch_labels = None
depends_on = None
def upgrade() -> None:
dialect = op.get_bind().dialect.name
if dialect == "postgresql":
config_expr = (
"CASE WHEN s.material_type IS NOT NULL AND s.material_type != '' "
"THEN json_build_object('material_type', s.material_type)::jsonb "
"ELSE '{}'::jsonb END"
)
empty_json = "'{}'::jsonb"
else:
config_expr = (
"CASE WHEN s.material_type IS NOT NULL AND s.material_type != '' "
"THEN JSON_OBJECT('material_type', s.material_type) "
"ELSE '{}' END"
)
empty_json = "'{}'"
sql_str = (
"INSERT INTO template_clip_configs "
'(id, template_id, clip_type, "order", min_duration, max_duration, '
"text_template, material_requirements, transition_effect, config, "
"created_at, updated_at) "
"SELECT "
"s.id, s.template_id, 'main', s.segment_order, "
"s.duration_min, s.duration_max, "
"'', " + empty_json + ", "
"'cut', " + config_expr + ", "
"s.created_at, s.updated_at "
"FROM template_segments s "
"WHERE NOT EXISTS ("
" SELECT 1 FROM template_clip_configs c "
" WHERE c.template_id = s.template_id"
")"
)
op.execute(sa.text(sql_str))
def downgrade() -> None:
pass
+21 -67
View File
@@ -26,7 +26,7 @@ from app.schemas.asset import (
UpdateAssetReviewRequest,
)
from app.schemas.tag import TagAssetsRequest
from app.services.asset_segment_tracker import compute_asset_availability, get_asset_recent_use_counts
from app.services.asset_segment_tracker import compute_asset_availability
from fastapi import APIRouter, Depends, HTTPException, Query, Response
from packages.domain.smart_match import smart_select_assets
@@ -579,80 +579,34 @@ def smart_match_assets(
filtered_assets = asset_repository.find_by_library(request.library_id, status=["ready"], limit=10000)
total_candidates = len(filtered_assets)
# ── 过滤前置:余量 + 高频使用,过滤在评分/截取 limit 之前完成 ──────────
# 旧实现先 smart_select_assets(limit=N) 再对这 N 条做过滤,过滤后不回补,
# 当排名靠前的素材恰好都被排除时返回空 items(前端回退全选,smart-match 名存实亡)。
# 现在先过滤全量候选,每级过滤后为空/不足则回退上一级,最后才评分截取。
# 调用统一智能选素材算法(kind 已在 DB 层过滤,无需重复过滤)
results = smart_select_assets(
filtered_assets,
limit=request.limit,
kind=None,
)
# 1) 余量过滤:usable=False(零重复可切区间耗尽且历史区间均达复用上限)的素材排除
usable_assets = []
exhausted_assets = []
for a in filtered_assets:
# 结果层过滤:usable=false(零重复可切区间耗尽且历史区间均达复用上限)的素材
# 不返回给前端;不动 smart_select_assets 评分逻辑本身
filtered_results = []
for r in results:
try:
avail = compute_asset_availability(a)
avail = compute_asset_availability(r.asset)
except Exception:
logger.warning(
"smart-match 余量计算失败,按可用处理: asset_id=%s",
getattr(a, "id", "?"),
getattr(r.asset, "id", "?"),
exc_info=True,
)
avail = None
if avail is not None and not avail["usable"]:
exhausted_assets.append(a)
else:
usable_assets.append(a)
if exhausted_assets:
logger.info(
"smart-match 余量过滤: 候选 %d,可切区间耗尽 %d",
len(filtered_assets), len(exhausted_assets),
)
# 回退策略:余量过滤后为空(全部耗尽)时,保留全部候选,不返回空结果。
# 宁可让用户在已耗尽素材上复用,也比 smart-match 空结果回退全选更可控
# (全选同样会选到这些素材,且不经过评分排序)。
pool = usable_assets if usable_assets else filtered_assets
# 2) 高频使用排除:同一素材在最近 5 个视频中出现超过 3 次则排除
MAX_RECENT_USE_COUNT = 3
high_freq_assets = set()
if pool:
asset_ids = [getattr(a, "id", "") for a in pool if getattr(a, "id", "")]
if asset_ids:
try:
use_counts = get_asset_recent_use_counts(
db=asset_repository.session,
asset_ids=asset_ids,
recent_video_count=5,
)
for a in pool:
aid = getattr(a, "id", "")
count = use_counts.get(aid, 0)
if count > MAX_RECENT_USE_COUNT:
high_freq_assets.add(aid)
logger.info(
"smart-match 排除高频使用素材: asset_id=%s use_count=%d limit=%d",
aid, count, MAX_RECENT_USE_COUNT,
)
# 回退策略:排除后剩余素材不足(为空或不够 limit)时,
# 不再全部排除,保留全部可用素材
if high_freq_assets:
remaining_count = len(pool) - len(high_freq_assets)
enough = request.limit is None or remaining_count >= request.limit
if remaining_count > 0 and enough:
pool = [a for a in pool if getattr(a, "id", "") not in high_freq_assets]
else:
logger.info(
"smart-match 高频排除后素材不足(%d<%s),保留全部 %d",
remaining_count,
request.limit if request.limit is not None else "不限",
len(pool),
)
except Exception:
logger.warning("smart-match 高频使用查询失败,跳过排除", exc_info=True)
# 3) 调用统一智能选素材算法(kind 已在 DB 层过滤,无需重复过滤)
results = smart_select_assets(pool, limit=request.limit, kind=None)
logger.info(
"smart-match 排除已用尽素材: asset_id=%s name=%s",
getattr(r.asset, "id", "?"),
getattr(r.asset, "name", ""),
)
continue
filtered_results.append(r)
# 扁平结构:SmartMatchItem 继承 AssetResponse,素材字段直接在条目顶层,
# 前端无需解析 item.asset 包装层,item.id / item.usable / 余量字段直接可读
@@ -662,7 +616,7 @@ def smart_match_assets(
score=r.score,
breakdown=r.breakdown,
)
for r in results
for r in filtered_results
]
return SmartMatchResponse(items=items, total_candidates=total_candidates)
@@ -37,9 +37,6 @@ from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException, Query, s
from sqlalchemy.orm import Session
from packages.adapters.sqlalchemy_impl.asset_repository import SQLAlchemyAssetRepository
from packages.adapters.sqlalchemy_impl.template_clip_config_repository import (
SQLAlchemyTemplateClipConfigRepository,
)
from packages.adapters.sqlalchemy_impl.template_repository import (
SQLAlchemyTemplateRepository,
)
@@ -414,25 +411,7 @@ def _get_template_segments(
result.append((cc.order, dur_min, dur_max))
return sorted(result, key=lambda x: x[0])
except Exception:
logger.warning("新模板系统查询clip_configs失败(主表可能不存在),直接查clip_configs表", exc_info=True)
# 兜底:直接查 template_clip_configs 表(片段表有 template_id 外键,不依赖模板主表)
try:
direct_repo = SQLAlchemyTemplateClipConfigRepository(db)
direct_configs = direct_repo.list_by_template(template_id)
if direct_configs:
result = []
for cc in direct_configs:
dur_min = _safe_segment_duration(cc.min_duration, _DEFAULT_EDITOR_CLIP_DURATION)
dur_max = _safe_segment_duration(
cc.max_duration or cc.min_duration,
_DEFAULT_EDITOR_CLIP_DURATION,
)
dur_min, dur_max = min(dur_min, dur_max), max(dur_min, dur_max)
result.append((cc.order, dur_min, dur_max))
return sorted(result, key=lambda x: x[0])
except Exception:
logger.warning("直接查clip_configs表也失败,继续回退旧系统", exc_info=True)
logger.warning("新模板系统查询clip_configs失败,回退到旧系统", exc_info=True)
# 回退到旧模板系统(template_segments表)
try:
-1
View File
@@ -52,7 +52,6 @@ def _to_video_response(item, storage: OSSStorageService | None = None) -> VideoI
generation_params=item.generation_params,
download_url=download_url,
generated_at=format_utc_datetime(item.generated_at) if hasattr(item, "generated_at") else "",
duplicate_rate=getattr(item, "duplicate_rate", None),
)
-1
View File
@@ -22,7 +22,6 @@ class VideoItemResponse(BaseModel):
generation_params: dict = Field(default_factory=dict)
download_url: str | None = None
generated_at: str = ""
duplicate_rate: float | None = None
class ListVideosResponse(BaseModel):
@@ -438,57 +438,3 @@ def make_reuse_callback(
return result
return _reuse
def get_asset_recent_use_counts(
db: Session,
asset_ids: list[str],
recent_video_count: int = 5,
) -> dict[str, int]:
"""统计每个素材在最近 N 个不同 plan_id 中的使用次数。
遍历素材 metadata 中的 used_time_ranges,统计有多少个不同的 plan_id(去重),
返回 {asset_id: count}。只统计最近 recent_video_count 个不同 plan_id 的使用次数。
Args:
db: 数据库会话
asset_ids: 素材 ID 列表
recent_video_count: 统计最近多少个不同 plan_id
Returns:
{asset_id: 在最近 recent_video_count 个 plan 中的使用次数}
"""
if not asset_ids:
return {}
result: dict[str, int] = {}
models = db.query(AssetModel).filter(AssetModel.id.in_(asset_ids)).all()
for model in models:
meta = _read_meta(model)
ranges = meta.get(USED_RANGES_KEY) or []
if not ranges:
result[model.id] = 0
continue
# 按 created_at 倒序收集不同 plan_id
sorted_ranges = sorted(
ranges,
key=lambda r: r.get("created_at") or "",
reverse=True,
)
recent_plan_ids: set[str] = set()
for r in sorted_ranges:
plan_id = r.get("plan_id")
if plan_id:
recent_plan_ids.add(plan_id)
if len(recent_plan_ids) >= recent_video_count:
break
result[model.id] = len(recent_plan_ids)
# 未找到的素材计为 0
for aid in asset_ids:
if aid not in result:
result[aid] = 0
return result
File diff suppressed because one or more lines are too long
+1 -4
View File
@@ -338,15 +338,12 @@ const GeneratePage: React.FC = () => {
size: titleSettings.size,
font: titleSettings.font,
color: titleSettings.color,
position: titleSettings.position as "top" | "center" | "bottom" | "custom",
position: titleSettings.position as "top" | "center" | "bottom",
bold: titleSettings.bold,
italic: titleSettings.italic,
stroke: titleSettings.stroke,
shadow: titleSettings.shadow,
posX: titleSettings.posX,
posY: titleSettings.posY,
}}
onTitlePositionChange={styleUpdaters.updateTitlePosition}
/>
)}
{currentStep >= 5 && generated && finalVideo && (
@@ -32,15 +32,12 @@ interface FrontendPreviewPlayerProps {
size: number
font: string
color: string
position: "top" | "center" | "bottom" | "custom"
position: "top" | "center" | "bottom"
bold?: boolean
italic?: boolean
stroke?: boolean
shadow?: boolean
posX?: number | null
posY?: number | null
}
onTitlePositionChange?: (posX: number, posY: number) => void
}
function formatTime(seconds: number): string {
@@ -108,7 +105,6 @@ const FrontendPreviewPlayer: React.FC<FrontendPreviewPlayerProps> = ({
serverClips,
voiceAudioUrl,
titleSettings,
onTitlePositionChange,
}) => {
const segments = useMemo(
() => buildPlaybackSegments(assets, template, serverClips),
@@ -130,55 +126,6 @@ const FrontendPreviewPlayer: React.FC<FrontendPreviewPlayerProps> = ({
return { width: 1080, height: 1920 }
}
})()
const customTitleXPct =
titleSettings?.posX != null && playRes.width > 0
? (titleSettings.posX / playRes.width) * 100
: null
const customTitleYPct =
titleSettings?.posY != null && playRes.height > 0
? (titleSettings.posY / playRes.height) * 100
: null
// ── 拖拽状态(用 ref 避免在每帧渲染中触发重渲染)──
const draggingTitleRef = useRef(false)
const handleTitlePointerDown = useCallback(
(e: React.PointerEvent<HTMLDivElement>) => {
if (!onTitlePositionChange || !playerContainerRef.current) return
e.preventDefault()
e.stopPropagation()
;(e.target as Element).setPointerCapture(e.pointerId)
draggingTitleRef.current = true
;(e.currentTarget as HTMLDivElement).style.cursor = "grabbing"
},
[onTitlePositionChange],
)
const handleTitlePointerMove = useCallback(
(e: React.PointerEvent<HTMLDivElement>) => {
if (!draggingTitleRef.current || !onTitlePositionChange || !playerContainerRef.current) return
e.preventDefault()
e.stopPropagation()
const rect = playerContainerRef.current.getBoundingClientRect()
const relX = Math.max(0, Math.min(rect.width, e.clientX - rect.left))
const relY = Math.max(0, Math.min(rect.height, e.clientY - rect.top))
const posX = Math.round((relX / rect.width) * playRes.width)
const posY = Math.round((relY / rect.height) * playRes.height)
onTitlePositionChange(posX, posY)
},
[onTitlePositionChange, playRes.width, playRes.height],
)
const handleTitlePointerUp = useCallback((e: React.PointerEvent<HTMLDivElement>) => {
if (!draggingTitleRef.current) return
draggingTitleRef.current = false
;(e.currentTarget as HTMLDivElement).style.cursor = "grab"
try {
if ((e.currentTarget as Element).hasPointerCapture(e.pointerId)) {
;(e.currentTarget as Element).releasePointerCapture(e.pointerId)
}
} catch {
/* ignore */
}
}, [])
const playerContainerRef = useRef<HTMLDivElement>(null)
const [containerHeight, setContainerHeight] = useState(0)
useEffect(() => {
@@ -577,34 +524,15 @@ const FrontendPreviewPlayer: React.FC<FrontendPreviewPlayerProps> = ({
<div
style={{
position: "absolute",
...(customTitleXPct != null && customTitleYPct != null
? {
left: `${customTitleXPct}%`,
top: `${customTitleYPct}%`,
transform: "translate(-50%, -50%)",
textAlign: "center" as const,
}
: {
left: `${titleSidePct}%`,
right: `${titleSidePct}%`,
textAlign: "center" as const,
...(titleSettings.position === "top"
? { top: `${titleTopPct}%` }
: titleSettings.position === "center"
? { top: "50%", transform: "translateY(-50%)" }
: { bottom: `${titleBottomPct}%` }),
}),
cursor: onTitlePositionChange ? "grab" : "default",
touchAction: "none",
userSelect: "none",
WebkitUserSelect: "none",
padding: "8px 12px",
boxShadow: "inset 0 0 0 16px transparent",
left: `${titleSidePct}%`,
right: `${titleSidePct}%`,
textAlign: "center",
...(titleSettings.position === "top"
? { top: `${titleTopPct}%` }
: titleSettings.position === "center"
? { top: "50%", transform: "translateY(-50%)" }
: { bottom: `${titleBottomPct}%` }),
}}
onPointerDown={handleTitlePointerDown}
onPointerMove={handleTitlePointerMove}
onPointerUp={handleTitlePointerUp}
onPointerCancel={handleTitlePointerUp}
>
<span
style={{
@@ -23,7 +23,7 @@ const MaterialModeTabs: React.FC<MaterialModeTabsProps> = ({ mode, onModeChange
onClick={() => onModeChange("auto")}
type="button"
>
AI智能匹配
</button>
</div>
)
@@ -38,8 +38,9 @@ const TitlePresetsGrid: React.FC<TitlePresetsGridProps> = ({
className="xx-title-preset-preview-text"
style={{ ...p.previewStyle, fontFamily: getFontFamily(fontFamily || "思源黑体") }}
>
T
</span>
<span className="xx-title-preset-card-label">{p.label}</span>
</button>
)
})}
-1
View File
@@ -42,7 +42,6 @@ export const POSITION_OPTIONS = [
{ value: "top", label: "顶部" },
{ value: "center", label: "居中" },
{ value: "bottom", label: "底部" },
{ value: "custom", label: "自定义" },
]
/* ── 标题字体选项 ── */
+15 -5
View File
@@ -1733,8 +1733,8 @@
/* 标题预设卡片网格 */
.xx-title-presets-grid {
display: grid;
grid-template-columns: repeat(6, 1fr);
gap: 8px;
grid-template-columns: repeat(4, 1fr);
gap: 10px;
}
.xx-title-preset-card {
@@ -1742,8 +1742,7 @@
flex-direction: column;
align-items: center;
justify-content: center;
aspect-ratio: 1 / 1;
padding: 4px;
padding: 14px 8px;
background: var(--bg-secondary);
border: 2px solid transparent;
border-radius: var(--radius-sm);
@@ -1763,10 +1762,21 @@
}
.xx-title-preset-preview-text {
line-height: 1;
line-height: 1.4;
margin-bottom: 6px;
user-select: none;
}
.xx-title-preset-card-label {
font-size: 11px;
color: var(--text-secondary);
}
.xx-title-preset-card.active .xx-title-preset-card-label {
color: var(--primary-color);
font-weight: 500;
}
/* 样式按钮组 */
.xx-style-btns {
display: flex;
@@ -20,14 +20,13 @@ const SECONDS_PER_ASSET = 15
/**
* 根据模板 segments 计算所需素材数量上限。
* 取每个 segment 的 duration_min 之和作为目标视频总时长,
* 再按 15 秒/素材估算需要多少个素材,且保证不少于片段数(每个片段至少 1 个素材);
* 结果钳制到 [1, 200] 区间(后端 limit 上限 200)。
* 再按 15 秒/素材估算需要多少个素材;结果钳制到 [1, 200] 区间(后端 limit 上限 200)。
*/
function computeLimitFromSegments(segments?: TemplateSegment[]): number {
if (!segments || segments.length === 0) return DEFAULT_LIMIT
const totalSeconds = segments.reduce((sum, seg) => sum + (seg.duration_min || 0), 0)
if (totalSeconds <= 0) return DEFAULT_LIMIT
const limit = Math.max(segments.length, Math.ceil(totalSeconds / SECONDS_PER_ASSET))
const limit = Math.ceil(totalSeconds / SECONDS_PER_ASSET)
return Math.max(1, Math.min(limit, 200))
}
@@ -233,7 +233,7 @@ export function useCanvasPlayer(
fontSize: number
fontFamily: string
color: string
position: "top" | "center" | "bottom" | "custom"
position: "top" | "center" | "bottom"
bold?: boolean
stroke?: boolean
shadow?: boolean
@@ -27,8 +27,6 @@ const DEFAULT_TITLE_SETTINGS: TitleSettings = {
stroke: true,
shadow: false,
color: "#ffffff",
posX: null,
posY: null,
}
export interface GenerateFormState {
@@ -117,14 +117,6 @@ export function useGenerateVideo(props: UseGenerateVideoProps) {
font_size: props.titleSettings.size,
font_color: props.titleSettings.color,
position: props.titleSettings.position,
...(props.titleSettings.position === "custom" &&
props.titleSettings.posX != null &&
props.titleSettings.posY != null
? {
pos_x: Math.round(props.titleSettings.posX),
pos_y: Math.round(props.titleSettings.posY),
}
: {}),
bold: props.titleSettings.bold,
stroke: props.titleSettings.stroke,
shadow: props.titleSettings.shadow,
@@ -46,16 +46,7 @@ export function useTitleStyleUpdaters({
const updatePosition = useCallback(
(position: string) => {
// 切回三档预设时清掉自定义坐标
onTitleSettingsChange({ ...titleSettings, position, posX: null, posY: null })
},
[titleSettings, onTitleSettingsChange],
)
/** 拖拽更新自定义位置(由预览播放器调用) */
const updateTitlePosition = useCallback(
(posX: number, posY: number) => {
onTitleSettingsChange({ ...titleSettings, position: "custom", posX, posY })
onTitleSettingsChange({ ...titleSettings, position })
},
[titleSettings, onTitleSettingsChange],
)
@@ -120,7 +111,6 @@ export function useTitleStyleUpdaters({
updateTitle,
toggleAiAutoSelect,
updatePosition,
updateTitlePosition,
updateFont,
updateSize,
updateColor,
-3
View File
@@ -16,9 +16,6 @@ export interface TitleSettings {
stroke: boolean
shadow: boolean
color: string
/** 自由位置坐标(PlayRes 像素),仅当 position="custom" 时有效 */
posX: number | null
posY: number | null
}
/* ── 智能匹配结果 ── */
@@ -11,7 +11,7 @@ import {
} from "@ant-design/icons"
import type { ProductItem } from "../types"
import { statusConfig, reviewStatusConfig } from "../constants"
import { formatTime, formatSize } from "../utils"
import { formatTime } from "../utils"
interface ProductCardProps {
product: ProductItem
@@ -210,20 +210,11 @@ export const ProductCard: React.FC<ProductCardProps> = ({
<span className={`xx-product-status ${st.className}`}>{st.text}</span>
<span className="xx-product-date">{product.date}</span>
</div>
<div className="xx-product-meta-grid">
<span className="xx-product-meta-item">{product.resolution || "-"}</span>
<span className="xx-product-meta-item">
{product.duration > 0 ? formatTime(product.duration) : "-"}
{product.duplicateRate > 0 && (
<span className={`xx-product-dup-rate ${dupClass}`}>
{product.duplicateRate.toFixed(1)}%
</span>
<span className="xx-product-meta-item">{formatSize(product.fileSize)}</span>
<span
className={`xx-product-meta-item xx-product-dup-rate${
product.duplicateRate > 0 ? ` ${dupClass}` : ""
}`}
>
{product.duplicateRate > 0 ? `${product.duplicateRate.toFixed(1)}%` : "-"}
</span>
</div>
)}
</div>
{/* 操作按钮 */}
+2 -17
View File
@@ -276,11 +276,11 @@
background-position: center;
}
/* 内联视频播放器(cover 填满容器,竖屏视频不留左右空白) */
/* 内联视频播放器 */
.xx-product-thumb-video {
width: 100%;
height: 100%;
object-fit: cover;
object-fit: contain;
display: block;
}
@@ -347,21 +347,6 @@
gap: var(--space-xs);
}
/* 卡片信息网格:分辨率/时长 一行,大小/查重率 一行 */
.xx-product-meta-grid {
display: grid;
grid-template-columns: 1fr 1fr;
gap: 4px var(--space-sm);
}
.xx-product-meta-item {
font-size: var(--font-size-xs);
color: var(--text-secondary);
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
}
/* 状态标签 */
.xx-product-status {
padding: 2px 10px;
-65
View File
@@ -308,71 +308,6 @@ class VideoDeduplicator:
return sum(similarities) / len(similarities) if similarities else 0.0
def compute_duplicate_rate(
self,
fingerprint: VideoFingerprint,
project_id: str,
current_video_id: str | None,
session: Session,
) -> float:
"""计算当前视频与项目内已有视频的最高相似度百分比。
遍历项目内所有其他有指纹的视频,对每个计算相似度:
- MD5 精确匹配 → 100%
- pHash 相似度 → (1.0 - avg_distance / 64) * 100
取最高值作为 duplicate_rate0~100)。
如果没有其他视频可比较,返回 0.0。
Args:
fingerprint: 当前视频的指纹
project_id: 项目 ID
current_video_id: 当前视频 ID(排除自身,可为 None)
session: 数据库会话
Returns:
duplicate_rate: 0~100 的浮点数
"""
# 限制查询最近 100 个视频,避免大项目内存溢出
from packages.adapters.sqlalchemy_impl.models import GeneratedVideoModel
recent_models = (
session.query(GeneratedVideoModel)
.filter(GeneratedVideoModel.project_id == project_id)
.order_by(GeneratedVideoModel.generated_at.desc())
.limit(100)
.all()
)
video_repo = SQLAlchemyGeneratedVideoRepository(session)
existing_videos = [video_repo._to_domain(m) for m in recent_models]
max_similarity = 0.0
for existing in existing_videos:
if current_video_id and existing.id == current_video_id:
continue
if not existing.video_fingerprint:
continue
ef = existing.video_fingerprint
# MD5 精确匹配 → 100%
if fingerprint.md5 == ef.get("md5"):
return 100.0
# pHash 相似度
existing_phashes = ef.get("keyframe_phashes", [])
if not existing_phashes or not fingerprint.keyframe_phashes:
continue
min_distances = []
for phash in fingerprint.keyframe_phashes:
distances = [hamming_distance(phash, ep) for ep in existing_phashes]
min_distances.append(min(distances))
avg_distance = sum(min_distances) / len(min_distances) if min_distances else 64
similarity = (1.0 - avg_distance / 64) * 100
max_similarity = max(max_similarity, similarity)
return round(max(max_similarity, 0.0), 2)
@celery_app.task(bind=True, max_retries=3, name="worker.check_duplicate")
def check_duplicate_task(self: Task, generated_video_id: str) -> dict:
@@ -121,15 +121,6 @@ def create_video_record_and_dedup(
generated_video.is_duplicate = False
generated_video.duplicate_of = None
# 计算重复率百分比(与项目内所有已有视频对比取最高相似度)
try:
dup_rate = deduplicator.compute_duplicate_rate(fingerprint, project_id, video_id, session)
generated_video.duplicate_rate = dup_rate
logger.info("Duplicate rate for %s: %.2f%%", video_id, dup_rate)
except Exception as rate_err:
logger.warning("Failed to compute duplicate_rate for %s: %s", video_id, rate_err)
generated_video.duplicate_rate = None
video_repo.update(generated_video)
session.commit()
logger.info(
-8
View File
@@ -30,14 +30,6 @@ RUN apt-get update \
&& apt-get install -y --no-install-recommends docker-ce-cli docker-buildx-plugin \
&& rm -rf /var/lib/apt/lists/*
# Pre-bake ffmpeg: unit-tests run in fresh containers each time; installing ffmpeg
# on every job cost ~24 min (apt update + hundreds of codec deps). Bake it into the
# image so step_install_ffmpeg.sh detects it and exits instantly.
RUN apt-get update \
&& apt-get install -y --no-install-recommends ffmpeg \
&& ffmpeg -version | head -1 \
&& rm -rf /var/lib/apt/lists/*
# Pre-install base deps (layer cache)
COPY requirements-base.txt ./
RUN python -m venv "$VIRTUAL_ENV" \
-3
View File
@@ -1,6 +1,5 @@
# Build stage
FROM git.xiaoxiajianji.com/xiaoxia/base/node:20 AS builder
ARG SOURCE_HASH=""
WORKDIR /app
ARG VITE_API_URL=https://saas-api.xiaoxiajianji.com
ENV VITE_API_URL=$VITE_API_URL
@@ -19,10 +18,8 @@ COPY apps/web/ ./
# 构建:TS增量编译 + Vite构建,tsbuildinfo用cache mount持久化
# node_modules直接使用镜像中已安装的(layer缓存保证完整性)
# SOURCE_HASH 变化时强制重新执行(防止 buildkit 幽灵缓存命中)
RUN --mount=type=cache,target=/app/apps/web/.tscache,sharing=locked \
mkdir -p .tscache \
&& echo "SOURCE_HASH=${SOURCE_HASH}" > .cache_bust \
&& ./node_modules/.bin/tsc --incremental --tsBuildInfoFile .tscache/tsconfig.tsbuildinfo \
&& ./node_modules/.bin/vite build
@@ -30,7 +30,6 @@ class SQLAlchemyGeneratedVideoRepository:
video_fingerprint=json.dumps(video.video_fingerprint) if video.video_fingerprint else None,
is_duplicate=video.is_duplicate,
duplicate_of=video.duplicate_of,
duplicate_rate=video.duplicate_rate,
generated_at=video.generated_at,
created_at=video.created_at,
)
@@ -61,7 +60,6 @@ class SQLAlchemyGeneratedVideoRepository:
video_fingerprint=json.loads(getattr(model, "video_fingerprint", "null") or "null"),
is_duplicate=getattr(model, "is_duplicate", False),
duplicate_of=getattr(model, "duplicate_of", None),
duplicate_rate=getattr(model, "duplicate_rate", None),
generated_at=model.generated_at,
created_at=model.created_at,
)
@@ -76,7 +74,6 @@ class SQLAlchemyGeneratedVideoRepository:
model.video_fingerprint = json.dumps(video.video_fingerprint) if video.video_fingerprint else None
model.is_duplicate = video.is_duplicate
model.duplicate_of = video.duplicate_of
model.duplicate_rate = video.duplicate_rate
self.session.add(model)
self.session.commit()
return video
@@ -207,7 +204,6 @@ class SQLAlchemyGeneratedVideoRepository:
video_fingerprint=json.loads(getattr(model, "video_fingerprint", "null") or "null"),
is_duplicate=getattr(model, "is_duplicate", False),
duplicate_of=getattr(model, "duplicate_of", None),
duplicate_rate=getattr(model, "duplicate_rate", None),
generated_at=model.generated_at,
created_at=model.created_at,
)
@@ -338,7 +338,6 @@ class GeneratedVideoModel(Base):
video_fingerprint = Column(Text, nullable=True)
is_duplicate = Column(Boolean, nullable=False, default=False)
duplicate_of = Column(String(36), nullable=True)
duplicate_rate = Column(Float, nullable=True)
class TitleLibraryModel(Base):
@@ -1,9 +1,4 @@
"""SQLAlchemy implementation of TemplateRepository.
模板 segments 数据源已统一为 template_clip_configs 表。
读取时优先 template_clip_configs,回退 template_segments(兼容历史数据)。
写入全部走 template_clip_configs。
"""
"""SQLAlchemy implementation of TemplateRepository."""
from __future__ import annotations
@@ -15,7 +10,6 @@ from sqlalchemy.orm import Session
from packages.adapters.sqlalchemy_impl.models import (
EditPlanModel,
TemplateCategoryModel,
TemplateClipConfigModel,
TemplateModel,
TemplateSegmentModel,
)
@@ -53,38 +47,27 @@ class SQLAlchemyTemplateRepository:
like_pattern = f"%{keyword}%"
query = query.filter(TemplateModel.name.like(like_pattern))
if tag:
query = query.filter(TemplateModel.tags.like(f'"%{tag}"%'))
# JSON 数组包含指定标签(MySQL JSON_CONTAINS / SQLite json_each 兼容写法用 LIKE
query = query.filter(TemplateModel.tags.like(f'%"{tag}"%'))
models = query.order_by(TemplateModel.created_at.desc()).offset(skip).limit(limit).all()
templates = [self._model_to_entity(m) for m in models]
# 批量加载 segments —— 优先 template_clip_configs
# 批量加载所有 segments,避免 N+1 查询
if templates:
template_ids = [t.id for t in templates]
clip_models = (
self.session.query(TemplateClipConfigModel)
.filter(TemplateClipConfigModel.template_id.in_(template_ids))
.order_by(TemplateClipConfigModel.order)
seg_models = (
self.session.query(TemplateSegmentModel)
.filter(TemplateSegmentModel.template_id.in_(template_ids))
.order_by(TemplateSegmentModel.segment_order)
.all()
)
clip_map: dict[str, list] = {}
for cm in clip_models:
clip_map.setdefault(cm.template_id, []).append(
self._clip_config_to_segment(cm),
# 按 template_id 分组
seg_map: dict[str, list] = {}
for sm in seg_models:
seg_map.setdefault(sm.template_id, []).append(
self._segment_model_to_entity(sm),
)
# 对没有 clip_configs 的模板,回退读 template_segments
missing_ids = [t.id for t in templates if t.id not in clip_map]
if missing_ids:
old_models = (
self.session.query(TemplateSegmentModel)
.filter(TemplateSegmentModel.template_id.in_(missing_ids))
.order_by(TemplateSegmentModel.segment_order)
.all()
)
for om in old_models:
clip_map.setdefault(om.template_id, []).append(
self._segment_model_to_entity(om),
)
for t in templates:
t.segments = clip_map.get(t.id, [])
t.segments = seg_map.get(t.id, [])
return templates
def get(self, template_id: str, user_id: str) -> Optional[Template]:
@@ -117,6 +100,7 @@ class SQLAlchemyTemplateRepository:
is_active=template.is_active,
)
self.session.add(model)
# flush 而非 commit,让 create + create_segments 在同一事务中提交
self.session.flush()
self.session.refresh(model)
result = self._model_to_entity(model)
@@ -161,8 +145,11 @@ class SQLAlchemyTemplateRepository:
if model is None:
return False
model.is_active = False
# 复用 delete_segments_by_template 清理两张表的关联数据
self.delete_segments_by_template(template_id)
# 级联清理关联的 segments,避免孤儿数据
self.session.query(TemplateSegmentModel).filter(
TemplateSegmentModel.template_id == template_id,
).delete(synchronize_session=False)
self.session.commit()
return True
def count_by_user(
@@ -185,7 +172,7 @@ class SQLAlchemyTemplateRepository:
if keyword:
query = query.filter(TemplateModel.name.like(f"%{keyword}%"))
if tag:
query = query.filter(TemplateModel.tags.like(f'"%{tag}"%'))
query = query.filter(TemplateModel.tags.like(f'%"{tag}"%'))
return query.count()
def copy_template(self, template_id: str, user_id: str, new_name: str) -> Template:
@@ -194,8 +181,9 @@ class SQLAlchemyTemplateRepository:
if source is None:
raise ValueError(f"Template {template_id} not found")
new_id = str(uuid.uuid4())
new_template = Template(
id=str(uuid.uuid4()),
id=new_id,
user_id=user_id,
name=new_name,
mode=source.mode,
@@ -209,22 +197,28 @@ class SQLAlchemyTemplateRepository:
)
created = self.create(new_template)
# 复用 create_segments 写入 template_clip_configs
# 复制 segments
new_segments: List[TemplateSegment] = []
for seg in source.segments:
new_segments.append(
TemplateSegment(
id=str(uuid.uuid4()),
template_id=created.id,
segment_order=seg.segment_order,
duration_min=seg.duration_min,
duration_max=seg.duration_max,
material_type=seg.material_type,
)
new_seg = TemplateSegment(
id=str(uuid.uuid4()),
template_id=new_id,
segment_order=seg.segment_order,
duration_min=seg.duration_min,
duration_max=seg.duration_max,
material_type=seg.material_type,
)
new_segments.append(new_seg)
model = TemplateSegmentModel(
id=new_seg.id,
template_id=new_seg.template_id,
segment_order=new_seg.segment_order,
duration_min=new_seg.duration_min,
duration_max=new_seg.duration_max,
material_type=new_seg.material_type,
)
self.session.add(model)
if new_segments:
self.create_segments(new_segments)
else:
self.session.commit()
created.segments = new_segments
@@ -233,58 +227,34 @@ class SQLAlchemyTemplateRepository:
# ── Segments ──
def list_segments(self, template_id: str) -> List[TemplateSegment]:
"""优先从 template_clip_configs 读取,回退读 template_segments。"""
clips = (
self.session.query(TemplateClipConfigModel)
.filter(TemplateClipConfigModel.template_id == template_id)
.order_by(TemplateClipConfigModel.order)
.all()
)
if clips:
return [self._clip_config_to_segment(m) for m in clips]
# 回退:旧表
old = (
models = (
self.session.query(TemplateSegmentModel)
.filter(TemplateSegmentModel.template_id == template_id)
.order_by(TemplateSegmentModel.segment_order)
.all()
)
return [self._segment_model_to_entity(m) for m in old]
return [self._segment_model_to_entity(m) for m in models]
def create_segments(self, segments: List[TemplateSegment]) -> List[TemplateSegment]:
"""写入 template_clip_configs 表。material_type 存入 config JSON。"""
for seg in segments:
config = {"material_type": seg.material_type} if seg.material_type else {}
model = TemplateClipConfigModel(
model = TemplateSegmentModel(
id=seg.id,
template_id=seg.template_id,
clip_type="main",
order=seg.segment_order,
min_duration=seg.duration_min,
max_duration=seg.duration_max,
text_template="",
material_requirements={},
transition_effect="cut",
config=config,
segment_order=seg.segment_order,
duration_min=seg.duration_min,
duration_max=seg.duration_max,
material_type=seg.material_type,
)
self.session.add(model)
self.session.commit()
return segments
def delete_segments_by_template(self, template_id: str) -> int:
"""删除两张表中的 segments 数据,返回删除总数。"""
c1 = (
self.session.query(TemplateClipConfigModel)
.filter(TemplateClipConfigModel.template_id == template_id)
.delete(synchronize_session=False)
)
c2 = (
self.session.query(TemplateSegmentModel)
.filter(TemplateSegmentModel.template_id == template_id)
.delete(synchronize_session=False)
count = (
self.session.query(TemplateSegmentModel).filter(TemplateSegmentModel.template_id == template_id).delete()
)
self.session.commit()
return c1 + c2
return count
# ── Categories ──
@@ -396,23 +366,6 @@ class SQLAlchemyTemplateRepository:
updated_at=model.updated_at,
)
@staticmethod
def _clip_config_to_segment(model: TemplateClipConfigModel) -> TemplateSegment:
"""将 TemplateClipConfigModel 转换为 TemplateSegment 域实体。"""
material_type = None
if model.config and isinstance(model.config, dict):
material_type = model.config.get("material_type")
return TemplateSegment(
id=model.id,
template_id=model.template_id,
segment_order=model.order,
duration_min=model.min_duration,
duration_max=model.max_duration,
material_type=material_type,
created_at=model.created_at,
updated_at=model.updated_at,
)
@staticmethod
def _category_model_to_entity(model: TemplateCategoryModel) -> TemplateCategory:
return TemplateCategory(
+1 -50
View File
@@ -256,42 +256,6 @@ def _wrap_title_text(
return "\\N".join(wrapped_segments)
def _parse_title_position(
title_config: dict[str, Any],
video_width: int,
video_height: int,
) -> tuple[int, int] | None:
"""解析标题自由拖拽坐标 pos_x/pos_yPlayRes 像素坐标系)。
要求两个字段同时存在、可转 int,且落在 [0, video_width] × [0, video_height]
闭区间内。任一条件不满足返回 None,调用方回退 position 三档逻辑。
Args:
title_config: 标题配置 dict
video_width: PlayResX(视频宽度像素)
video_height: PlayResY(视频高度像素)
Returns:
(x, y) 整数坐标,或 None 表示不使用自由位置
"""
if "pos_x" not in title_config or "pos_y" not in title_config:
return None
raw_x = title_config["pos_x"]
raw_y = title_config["pos_y"]
# 坐标必须是 PlayRes 像素整数:bool 是 int 子类(isinstance(True,int)=True
# 但 True/False 作坐标无意义;float 静默截断会造成拖拽位置偏差,一律按非法回退
if isinstance(raw_x, bool) or isinstance(raw_y, bool):
return None
if not isinstance(raw_x, int) or not isinstance(raw_y, int):
return None
x, y = raw_x, raw_y
if video_width <= 0 or video_height <= 0:
return None
if not (0 <= x <= video_width and 0 <= y <= video_height):
return None
return (x, y)
def build_ass_content(
*,
video_width: int,
@@ -379,16 +343,7 @@ def build_ass_content(
title_shadow.get("offset_y", 2) if title_shadow.get("enabled", False) else 0,
)
# ── 自由位置拖拽(工单 #1405 方案 B)────────────────────────────
# pos_x/pos_y 为 PlayRes 坐标系像素整数(PlayResX/Y = video_width/height)。
# 合法时:TitleStyle Alignment 固定 5\an5 中对齐,使 \pos 锚点为文本块中心),
# Dialogue 文本前注入 {\pos(x,y)}。字段缺失/非法/越界时一律回退
# position → alignment 三档逻辑,现有输出保持一字节不变。
title_pos = _parse_title_position(title_config, video_width, video_height)
title_alignment = 5 if title_pos is not None else position_to_ass_alignment(
title_config.get("position", "top")
)
title_alignment = position_to_ass_alignment(title_config.get("position", "top"))
styles.append(
build_ass_style(
@@ -415,10 +370,6 @@ def build_ass_content(
safe_title_text_raw = escape_ass_text(title_text)
safe_title_text = _wrap_title_text(safe_title_text_raw, video_width, title_font_size)
# 自由位置:在文本前注入 \pos override tag(锚点为文本块中心,配合 \an5)
if title_pos is not None:
safe_title_text = f"{{\\pos({title_pos[0]},{title_pos[1]})}}{safe_title_text}"
events.append(
"Dialogue: 0,0:00:00.00," f"{format_ass_time(video_duration)}," "TitleStyle,,0,0,0,," f"{safe_title_text}"
)
-1
View File
@@ -26,7 +26,6 @@ class GeneratedVideo:
video_fingerprint: dict[str, Any] | None = None
is_duplicate: bool = False
duplicate_of: str | None = None
duplicate_rate: float | None = None
generated_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
created_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
+9 -6
View File
@@ -28,12 +28,15 @@ if [ "$SKIP_BACKEND" = "true" ]; then
CONTEXTS=("CI/CD Pipeline / Frontend Lint (pull_request)")
else
CONTEXTS=(
# CI 优化后 job 名称(2026-08):Code Quality 拆分为 Style+SecurityType Check+Migration 合并为 Python
# 与 pr_auto_scan.py 的 REQUIRED_CONTEXTS_APPROVE 保持一致
"CI/CD Pipeline / Validate - Style (pull_request)"
"CI/CD Pipeline / Validate - Security (pull_request)"
"CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request)"
"CI/CD Pipeline / Validate - Code Quality (pull_request)"
"CI/CD Pipeline / Validate - Type Check (mypy) (pull_request)"
"CI/CD Pipeline / Validate - Migration (alembic) (pull_request)"
"CI/CD Pipeline / Frontend Lint (pull_request)"
"CI/CD Pipeline / Unit Tests (pull_request)"
"CI/CD Pipeline / Frontend Unit Tests (pull_request)"
"CI/CD Pipeline / PR Build API Image (pull_request)"
"CI/CD Pipeline / PR Build Web Image (pull_request)"
"CI/CD Pipeline / PR Build Worker Image (pull_request)"
)
fi
@@ -150,4 +153,4 @@ done
echo
echo "⏰ 快速检查超时(2分钟),CI尚未完成,退出等待下次触发(workflow_run事件或5分钟定时扫描)"
exit 0
exit 0
+1 -70
View File
@@ -13,7 +13,6 @@ before="${GITHUB_EVENT_BEFORE:-}"
after="${GITHUB_SHA:-}"
repo="${GITHUB_REPOSITORY:-}"
base="${GITHUB_API_URL:-}"
ZERO="0000000000000000000000000000000000000000"
# Gitea Actions 中 push 事件的前一个 SHA 在 event payload 的 before 字段
if [ -z "$before" ] && [ -n "${GITHUB_EVENT_PATH:-}" ] && [ -f "$GITHUB_EVENT_PATH" ]; then
@@ -29,76 +28,8 @@ fi
echo "改动范围检测: before=${before:-<empty>} after=${after}"
# ── 安全回溯:确保 diff 基准覆盖所有未构建的改动 ──
# 问题:concurrency 取消机制会导致前端改动被跳过。被取消的 push 的改动不会被
# 后续 push 的 diff 覆盖到,因为 GITHUB_EVENT_BEFORE 只指向上一次 push 的 SHA。
# 修复:查询最近一次**实际构建了 web 镜像**的成功 push run,用其 head_sha 作为
# diff 基准。这样被取消/跳过的 run 的改动都会被包含在当前 diff 中。
if [ -n "$before" ] && [ "$before" != "$ZERO" ] && [ -n "${GITHUB_TOKEN:-}" ]; then
BRANCH="${GITHUB_REF_NAME:-}"
if [ -n "$BRANCH" ]; then
SAFE_BASE=$(python3 -c "
import json, subprocess, sys
base = '${base}'
repo = '${repo}'
token = '${GITHUB_TOKEN}'
branch = '${BRANCH}'
cur_sha = '${after}'
def check_run(run_id):
\"\"\"Check if this run actually built the web image.\"\"\"
try:
r = subprocess.run(
['curl', '-sf', '--max-time', '10',
'-H', f'Authorization: token {token}',
f'{base}/repos/{repo}/actions/runs/{run_id}/jobs'],
capture_output=True, text=True, timeout=15)
if r.returncode != 0:
return False
jobs = json.loads(r.stdout).get('jobs', [])
return any(
'Build Staging Web' in j.get('name', '')
and j.get('conclusion') == 'success'
for j in jobs
)
except Exception:
return False
try:
r = subprocess.run(
['curl', '-sf', '--max-time', '15',
'-H', f'Authorization: token {token}',
f'{base}/repos/{repo}/actions/runs?status=success&event=push&branch={branch}&per_page=30'],
capture_output=True, text=True, timeout=20)
if r.returncode != 0:
sys.exit(0)
d = json.loads(r.stdout)
runs = d.get('workflow_runs', []) if isinstance(d, dict) else d
for run in runs:
sha = run.get('head_sha', '')
if sha and sha != cur_sha:
if check_run(run['id']):
print(sha)
break
except Exception:
pass
" 2>/dev/null || true)
if [ -n "$SAFE_BASE" ] && [ "$SAFE_BASE" != "$before" ]; then
echo "🔒 安全回溯: 使用最近实际构建 web 的 commit ${SAFE_BASE:0:8} 替代 before=${before:0:8}"
before="$SAFE_BASE"
elif [ -z "$SAFE_BASE" ]; then
echo "⚠️ 未找到历史成功构建 web 的 push run,保守走全量构建"
echo "skip_backend=false" >> "$OUTPUT"
echo "skip_frontend=false" >> "$OUTPUT"
exit 0
fi
fi
fi
FILES=""
if [ -n "$before" ] && [ "$before" != "$ZERO" ]; then
if [ -n "$before" ] && [ "$before" != "0000000000000000000000000000000000000000" ]; then
# Gitea 1.26.x compare API 的顶层 files 字段不填充(始终为空),
# 但响应里每个 commit 条目自带的 files 完整可用;聚合区间内所有提交的 files 即可。
API_URL="${base}/repos/${repo}/compare/${before}...${after}?per_page=300"
-12
View File
@@ -19,18 +19,6 @@ for arg in "$@"; do
BUILD_ARGS="$BUILD_ARGS --build-arg $arg"
done
# Web 镜像 cache bust:计算 apps/web/ 的 git tree hash
# 当源码变化时 hash 变化,buildx 的 ARG 缓存键失效 → vite build 必定重新执行
if [ "${DOCKERFILE##*/}" = "web.Dockerfile" ]; then
SOURCE_HASH=$(git rev-parse HEAD:apps/web 2>/dev/null || echo "")
if [ -n "$SOURCE_HASH" ]; then
echo "Web cache bust: SOURCE_HASH=${SOURCE_HASH}"
BUILD_ARGS="$BUILD_ARGS --build-arg SOURCE_HASH=${SOURCE_HASH}"
else
echo "⚠️ 无法计算 apps/web tree hash,跳过 cache bust"
fi
fi
BUILDER_NAME="ci-builder-persist"
if ! docker buildx inspect "$BUILDER_NAME" > /dev/null 2>&1; then
echo "持久 builder 不存在,创建中..."
-12
View File
@@ -31,18 +31,6 @@ for arg in "$@"; do
BUILD_ARGS="$BUILD_ARGS --build-arg $arg"
done
# Web 镜像 cache bust:计算 apps/web/ 的 git tree hash
# 当源码变化时 hash 变化,buildx 的 ARG 缓存键失效 → vite build 必定重新执行
if [ "${DOCKERFILE##*/}" = "web.Dockerfile" ]; then
SOURCE_HASH=$(git rev-parse HEAD:apps/web 2>/dev/null || echo "")
if [ -n "$SOURCE_HASH" ]; then
echo "Web cache bust: SOURCE_HASH=${SOURCE_HASH}"
BUILD_ARGS="$BUILD_ARGS --build-arg SOURCE_HASH=${SOURCE_HASH}"
else
echo "⚠️ 无法计算 apps/web tree hash,跳过 cache bust"
fi
fi
# 确保持久 builder 存在并使用(幂等)
if ! docker buildx inspect "$BUILDER_NAME" > /dev/null 2>&1; then
echo "持久 builder 不存在,创建中..."
+3 -4
View File
@@ -303,10 +303,9 @@ def main():
"CI/CD Pipeline / CI Gate (pull_request)",
]
REQUIRED_CONTEXTS_APPROVE = [
# CI 优化后 job 名称(2026-08):Code Quality 拆分为 Style+SecurityType Check+Migration 合并为 Python
"CI/CD Pipeline / Validate - Style (pull_request)",
"CI/CD Pipeline / Validate - Security (pull_request)",
"CI/CD Pipeline / Validate - Python (mypy + alembic) (pull_request)",
"CI/CD Pipeline / Validate - Code Quality (pull_request)",
"CI/CD Pipeline / Validate - Type Check (mypy) (pull_request)",
"CI/CD Pipeline / Validate - Migration (alembic) (pull_request)",
"CI/CD Pipeline / Frontend Lint (pull_request)",
]
FRONTEND_ONLY_CONTEXT = [
+5 -5
View File
@@ -197,8 +197,8 @@ if [ "$USE_SHARED_PG" = "true" ]; then
# 创建主数据库(xdist 模式下各 worker 会创建自己的数据库,主库作为 fallback)
echo "创建主测试数据库: $CI_DB_NAME"
PGPASSWORD="$SHARED_PG_PASSWORD" python3 -c "
import psycopg
conn = psycopg.connect(host='$SHARED_PG_HOST', port=$SHARED_PG_PORT, user='$SHARED_PG_USER', password='$SHARED_PG_PASSWORD', dbname='postgres')
import psycopg2
conn = psycopg2.connect(host='$SHARED_PG_HOST', port=$SHARED_PG_PORT, user='$SHARED_PG_USER', password='$SHARED_PG_PASSWORD', dbname='postgres')
conn.autocommit = True
cur = conn.cursor()
cur.execute(f'DROP DATABASE IF EXISTS \"$CI_DB_NAME\" WITH (FORCE)')
@@ -304,8 +304,8 @@ if [ "$USE_SHARED_PG" = "true" ]; then
# 清理所有以 CI_DB_NAME 开头的数据库(主库 + worker 库)
PGPASSWORD="${SHARED_PG_PASSWORD}" python3 -c "
import psycopg
conn = psycopg.connect(host='${SHARED_PG_HOST}', port=${SHARED_PG_PORT}, user='${SHARED_PG_USER}', password='${SHARED_PG_PASSWORD}', dbname='postgres')
import psycopg2
conn = psycopg2.connect(host='${SHARED_PG_HOST}', port=${SHARED_PG_PORT}, user='${SHARED_PG_USER}', password='${SHARED_PG_PASSWORD}', dbname='postgres')
conn.autocommit = True
cur = conn.cursor()
@@ -344,4 +344,4 @@ python3 scripts/ci_coverage_summary.py
set -e
echo ""
echo "=== CI Integration Tests 全部通过 ✅ ==="
echo "=== CI Integration Tests 全部通过 ✅ ==="
+4 -4
View File
@@ -409,8 +409,8 @@ except:
echo "创建测试数据库: $CI_DB_NAME"
PGPASSWORD="$SHARED_PG_PASSWORD" python3 -c "
import psycopg
conn = psycopg.connect(host='$SHARED_PG_HOST', port=$SHARED_PG_PORT, user='$SHARED_PG_USER', password='$SHARED_PG_PASSWORD', dbname='postgres')
import psycopg2
conn = psycopg2.connect(host='$SHARED_PG_HOST', port=$SHARED_PG_PORT, user='$SHARED_PG_USER', password='$SHARED_PG_PASSWORD', dbname='postgres')
conn.autocommit = True
cur = conn.cursor()
cur.execute(f'DROP DATABASE IF EXISTS \"$CI_DB_NAME\" WITH (FORCE)')
@@ -431,8 +431,8 @@ conn.close()
# 清理数据库
echo "清理测试数据库: $CI_DB_NAME"
PGPASSWORD="$SHARED_PG_PASSWORD" python3 -c "
import psycopg
conn = psycopg.connect(host='$SHARED_PG_HOST', port=$SHARED_PG_PORT, user='$SHARED_PG_USER', password='$SHARED_PG_PASSWORD', dbname='postgres')
import psycopg2
conn = psycopg2.connect(host='$SHARED_PG_HOST', port=$SHARED_PG_PORT, user='$SHARED_PG_USER', password='$SHARED_PG_PASSWORD', dbname='postgres')
conn.autocommit = True
cur = conn.cursor()
cur.execute(f'DROP DATABASE IF EXISTS \"$CI_DB_NAME\" WITH (FORCE)')
+14 -26
View File
@@ -1,39 +1,18 @@
#!/bin/sh
# CI 公共步骤:Checkout 代码(流式下载+解压,带重试)
# CI 公共步骤:Checkout 代码(带重试)
# 用法:直接 source 或调用,需要 GITHUB_TOKEN 环境变量
set -eu
python3 - <<'PY'
import io, os, sys, tarfile, time, urllib.request, urllib.error
import io, os, tarfile, time, urllib.request, urllib.error
url = f"{os.environ['GITHUB_API_URL']}/repos/{os.environ['GITHUB_REPOSITORY']}/archive/{os.environ['GITHUB_SHA']}.tar.gz"
request = urllib.request.Request(url, headers={"Authorization": f"token {os.environ['GITHUB_TOKEN']}"})
last_err = None
for attempt in range(5):
try:
with urllib.request.urlopen(request, timeout=120) as response:
# 流式读取:先读少量数据确认连接成功,再大块读取
first_chunk = response.read(8192)
buf = io.BytesIO()
buf.write(first_chunk)
while True:
chunk = response.read(65536)
if not chunk:
break
buf.write(chunk)
buf.seek(0)
with tarfile.open(fileobj=buf, mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
break
archive = response.read()
break
except urllib.error.HTTPError as e:
last_err = e
if e.code >= 500 and attempt < 4:
@@ -52,5 +31,14 @@ for attempt in range(5):
raise
else:
raise last_err
with tarfile.open(fileobj=io.BytesIO(archive), mode='r:gz') as tar:
root_prefix = tar.getmembers()[0].name.split('/', 1)[0] + '/'
for member in tar.getmembers():
name = member.name
if name == root_prefix[:-1]:
continue
if name.startswith(root_prefix):
member.name = name[len(root_prefix):]
if member.name:
tar.extract(member, '.')
PY
# CI pipeline speedup batch 1
+54 -38
View File
@@ -1,20 +1,14 @@
#!/bin/bash
# CI Validate: 安全扫描(validate-security
# 包含:密钥扫描、bandit 安全扫描(仅告警)、pip-audit 依赖漏洞(仅告警)、CI 脚本语法校验
# CI Validate: 代码质量与安全扫描(并行Job 1/3
# 包含:密钥扫描、格式检查、安全扫描、依赖漏洞、死代码检测、脚本语法校验
set -eu
echo "=== CI Validate: 安全扫描 ==="
echo "=== CI Validate: 代码质量与安全扫描 ==="
# --- 密钥检测 ---
echo ""
echo "=== [1/4] Secret detection (detect-secrets) ==="
python3 -m pip install -q --no-cache-dir detect-secrets || {
echo "⚠️ detect-secrets install failed, retrying without cache..."
python3 -m pip install -q --no-cache-dir --no-binary :all: detect-secrets || {
echo "❌ detect-secrets install failed after retry"
exit 1
}
}
echo "=== [1/6] Secret detection (detect-secrets) ==="
python3 -m pip install -q detect-secrets
detect-secrets --version
detect-secrets scan \
@@ -62,9 +56,23 @@ for fpath, items in data.get('results', {}).items():
fi
echo "✅ Secret scan passed"
# --- Bandit 安全扫描(仅告警---
# --- 代码质量检查(全量,PR 和 push 统一标准---
# 历史:PR 侧用增量检查以加速,但会导致 push 侧全量检查失败时 PR 侧感知不到
# 现在统一全量检查,确保 CI 真正保护主分支(black/isort/ruff 全量仅多几十秒)
echo ""
echo "=== [2/4] Security scan (bandit, advisory only) ==="
echo "=== [2/6] Code quality checks (full scan) ==="
SCAN_MODE="full"
echo "Full scan mode"
python3 -m compileall -q alembic apps packages tests scripts
python3 -m black --check --fast alembic apps packages tests scripts
python3 -m isort --check-only alembic apps packages tests scripts
python3 -m ruff check apps packages tests --statistics
echo "✅ Code quality checks passed"
# --- Bandit 安全扫描(仅告警) ---
echo ""
echo "=== [3/6] Security scan (bandit, advisory only) ==="
set +e
bandit -r apps packages -q -ll
BANDIT_EXIT=$?
@@ -75,34 +83,42 @@ else
echo "✅ Bandit security scan passed"
fi
# --- Pip-audit 依赖漏洞扫描(仅告警)---
# --- Pip-audit 依赖漏洞扫描(仅告警) ---
echo ""
echo "=== [3/4] Python dependency vulnerability scan (pip-audit, advisory only) ==="
python3 -m pip install -q --no-cache-dir pip-audit || {
echo "⚠️ pip-audit install failed (cache issue?), retrying..."
python3 -m pip install -q --no-cache-dir pip-audit || {
echo "⚠️ pip-audit unavailable, skipping dependency vulnerability scan (advisory)"
pip-audit --version 2>/dev/null || true
}
}
if command -v pip-audit >/dev/null 2>&1 || python3 -m pip show pip-audit >/dev/null 2>&1; then
pip-audit --version
EXIT_CODE=0
for req_file in requirements.txt requirements-base.txt requirements-dev.txt; do
if [ -f "$req_file" ]; then
echo "--- Scanning $req_file ---"
pip-audit -r "$req_file" --desc on 2>&1 | head -40 || EXIT_CODE=$?
echo ""
fi
done
echo "pip-audit scan completed (advisory mode - warnings only, not blocking CI)"
else
echo "⚠️ pip-audit not available, skipping dependency vulnerability scan (advisory)"
fi
echo "=== [4/6] Python dependency vulnerability scan (pip-audit, advisory only) ==="
python3 -m pip install -q pip-audit
pip-audit --version
EXIT_CODE=0
for req_file in requirements.txt requirements-base.txt requirements-dev.txt; do
if [ -f "$req_file" ]; then
echo "--- Scanning $req_file ---"
pip-audit -r "$req_file" --desc on 2>&1 | head -40 || EXIT_CODE=$?
echo ""
fi
done
echo "pip-audit scan completed (advisory mode - warnings only, not blocking CI)"
# --- Vulture 死代码检测(仅告警) ---
echo ""
echo "=== [5/6] Dead code detection (vulture, advisory only) ==="
set +e
python3 -m pip install -q vulture
vulture --version
echo "告警模式,不阻断CI。置信度>=90%建议尽快确认。"
echo ""
vulture apps packages scripts \
--exclude "tests,test,migrations,.gitea,docs,node_modules,site-packages,*/test_*.py,*/conftest.py" \
--min-confidence 70 \
2>&1 | sort -t'(' -k2 -rn | head -80
echo ""
echo "=== vulture scan summary ==="
echo "发现潜在死代码(可能包含框架装饰器注册的函数,为误报)"
echo "建议:定期人工审查高置信度(>=90%)条目"
set -e
# --- CI脚本语法校验 ---
echo ""
echo "=== [4/4] CI & shell scripts syntax validation ==="
echo "=== [6/6] CI & shell scripts syntax validation ==="
SYNTAX_ERROR=0
# 检查所有 CI shell 脚本
for script in scripts/ci/*.sh; do
@@ -138,4 +154,4 @@ fi
echo "✅ All CI scripts syntax OK"
echo ""
echo "=== CI Validate: 安全扫描 全部通过 ✅ ==="
echo "=== CI Validate: 代码质量与安全扫描 全部通过 ✅ ==="
+4 -4
View File
@@ -169,8 +169,8 @@ if [ "$USE_SHARED_PG" = "true" ]; then
echo "创建测试数据库: $CI_DB_NAME"
PGPASSWORD="$SHARED_PG_PASSWORD" python3 -c "
import psycopg
conn = psycopg.connect(host='$SHARED_PG_HOST', port=$SHARED_PG_PORT, user='$SHARED_PG_USER', password='$SHARED_PG_PASSWORD', dbname='postgres')
import psycopg2
conn = psycopg2.connect(host='$SHARED_PG_HOST', port=$SHARED_PG_PORT, user='$SHARED_PG_USER', password='$SHARED_PG_PASSWORD', dbname='postgres')
conn.autocommit = True
cur = conn.cursor()
cur.execute(f'DROP DATABASE IF EXISTS \"$CI_DB_NAME\" WITH (FORCE)')
@@ -185,8 +185,8 @@ conn.close()
echo ""
echo "清理测试数据库: $CI_DB_NAME"
PGPASSWORD="$SHARED_PG_PASSWORD" python3 -c "
import psycopg
conn = psycopg.connect(host='$SHARED_PG_HOST', port=$SHARED_PG_PORT, user='$SHARED_PG_USER', password='$SHARED_PG_PASSWORD', dbname='postgres')
import psycopg2
conn = psycopg2.connect(host='$SHARED_PG_HOST', port=$SHARED_PG_PORT, user='$SHARED_PG_USER', password='$SHARED_PG_PASSWORD', dbname='postgres')
conn.autocommit = True
cur = conn.cursor()
cur.execute(f'DROP DATABASE IF EXISTS \"$CI_DB_NAME\" WITH (FORCE)')
-42
View File
@@ -1,42 +0,0 @@
#!/bin/bash
# CI Validate: 代码风格检查(validate-style
# 包含:Python 字节码编译、black 格式、isort 排序、ruff lint、vulture 死代码(仅告警)
set -eu
echo "=== CI Validate: 代码风格检查 ==="
# --- Python 字节码编译 ---
echo ""
echo "=== [1/3] Python bytecode compilation ==="
python3 -m compileall -q alembic apps packages tests scripts
echo "✅ Bytecode compilation passed"
# --- 代码格式检查(全量)---
echo ""
echo "=== [2/3] Code formatting (black + isort + ruff) ==="
echo "Full scan mode"
python3 -m black --check --fast alembic apps packages tests scripts
python3 -m isort --check-only alembic apps packages tests scripts
python3 -m ruff check apps packages tests --statistics
echo "✅ Code formatting checks passed"
# --- Vulture 死代码检测(仅告警)---
echo ""
echo "=== [3/3] Dead code detection (vulture, advisory only) ==="
set +e
python3 -m pip install -q --no-cache-dir vulture || echo "⚠️ vulture install failed, skipping dead code detection"
vulture --version
echo "告警模式,不阻断CI。置信度>=90%建议尽快确认。"
echo ""
vulture apps packages scripts \
--exclude "tests,test,migrations,.gitea,docs,node_modules,site-packages,*/test_*.py,*/conftest.py" \
--min-confidence 70 \
2>&1 | sort -t'(' -k2 -rn | head -80
echo ""
echo "=== vulture scan summary ==="
echo "发现潜在死代码(可能包含框架装饰器注册的函数,为误报)"
echo "建议:定期人工审查高置信度(>=90%)条目"
set -e
echo ""
echo "=== CI Validate: 代码风格检查 全部通过 ✅ ==="
-145
View File
@@ -602,148 +602,3 @@ class TestFontsizeCompensation:
result = build_ass_style("S", font_size=0)
parts = result.split(",")
assert int(parts[2]) >= 1
# ── 标题自由位置拖拽(工单 #1405 方案 B)──────────────────────────────────────
def _title_style_line(content: str) -> str:
return [line for line in content.splitlines() if line.startswith("Style: TitleStyle")][0]
def _title_dialogue_line(content: str) -> str:
return [line for line in content.splitlines() if line.startswith("Dialogue:") and "TitleStyle" in line][0]
class TestTitleFreePosition:
"""pos_x/pos_y 合法时注入 \\pos 且 Alignment=5;非法/缺失时回退原逻辑。"""
def _base_kwargs(self):
return dict(
video_width=1080,
video_height=1920,
video_duration=8.0,
title_text="测试标题",
)
def test_valid_position_injects_pos_tag_and_alignment_5(self):
content = build_ass_content(
**self._base_kwargs(),
title_config={"position": "top", "size": 36, "pos_x": 540, "pos_y": 300},
)
# Dialogue 文本前注入 {\pos(540,300)}
dialogue = _title_dialogue_line(content)
assert "{\\pos(540,300)}" in dialogue
# TitleStyle Alignment 固定 5\an5 中对齐,\pos 锚点为文本块中心)
fields = [f.strip() for f in _title_style_line(content).split(",")]
assert fields[18] == "5"
def test_boundary_coordinates_zero_and_max_accepted(self):
"""边界值 0 和 video_width/video_height 合法(闭区间)。"""
content = build_ass_content(
**self._base_kwargs(),
title_config={"pos_x": 0, "pos_y": 1920},
)
assert "{\\pos(0,1920)}" in _title_dialogue_line(content)
content2 = build_ass_content(
**self._base_kwargs(),
title_config={"pos_x": 1080, "pos_y": 0},
)
assert "{\\pos(1080,0)}" in _title_dialogue_line(content2)
def test_no_coords_output_identical_to_before(self):
"""不传坐标 → 输出与现有断言完全一致(回归保护)。"""
content = build_ass_content(
**self._base_kwargs(),
title_config={"position": "top", "size": 36},
)
# 无 \pos 注入
assert "\\pos(" not in content
# Alignment 走 position 映射(top → 8
fields = [f.strip() for f in _title_style_line(content).split(",")]
assert fields[18] == "8"
def test_out_of_bounds_falls_back(self):
"""越界坐标 → 回退 position 三档逻辑,输出与无坐标一致。"""
base = build_ass_content(
**self._base_kwargs(),
title_config={"position": "top", "size": 36},
)
for pos_x, pos_y in [(-1, 300), (540, -1), (1081, 300), (540, 1921), (99999, 99999)]:
content = build_ass_content(
**self._base_kwargs(),
title_config={"position": "top", "size": 36, "pos_x": pos_x, "pos_y": pos_y},
)
assert "\\pos(" not in content, f"({pos_x},{pos_y}) should be rejected"
assert content == base, f"({pos_x},{pos_y}) output differs from fallback"
def test_invalid_coords_falls_back(self):
"""非法类型坐标 → 回退原逻辑。"""
base = build_ass_content(
**self._base_kwargs(),
title_config={"position": "top", "size": 36},
)
for pos_x, pos_y in [("abc", 300), (540, None), (None, None), (True, 300), (540, False), (540.5, 300.9)]:
content = build_ass_content(
**self._base_kwargs(),
title_config={"position": "top", "size": 36, "pos_x": pos_x, "pos_y": pos_y},
)
assert "\\pos(" not in content, f"({pos_x!r},{pos_y!r}) should be rejected"
assert content == base, f"({pos_x!r},{pos_y!r}) output differs from fallback"
def test_only_one_coord_falls_back(self):
"""只传 pos_x 或 pos_y → 回退原逻辑。"""
base = build_ass_content(
**self._base_kwargs(),
title_config={"position": "center", "size": 36},
)
content_x = build_ass_content(
**self._base_kwargs(),
title_config={"position": "center", "size": 36, "pos_x": 540},
)
content_y = build_ass_content(
**self._base_kwargs(),
title_config={"position": "center", "size": 36, "pos_y": 300},
)
assert content_x == base
assert content_y == base
assert "\\pos(" not in content_x
assert "\\pos(" not in content_y
def test_position_three_levels_unchanged_without_coords(self):
"""无坐标时 top/center/bottom 三档 Alignment 输出不变。"""
for position, expected_align in [("top", "8"), ("center", "5"), ("bottom", "2")]:
content = build_ass_content(
**self._base_kwargs(),
title_config={"position": position, "size": 36},
)
fields = [f.strip() for f in _title_style_line(content).split(",")]
assert fields[18] == expected_align
def test_pos_overrides_position_alignment(self):
"""有合法坐标时,无论 position 是什么,Alignment 都固定为 5。"""
for position in ["top", "center", "bottom"]:
content = build_ass_content(
**self._base_kwargs(),
title_config={"position": position, "size": 36, "pos_x": 100, "pos_y": 200},
)
fields = [f.strip() for f in _title_style_line(content).split(",")]
assert fields[18] == "5"
assert "{\\pos(100,200)}" in _title_dialogue_line(content)
def test_subtitle_not_affected_by_pos(self):
"""pos_x/pos_y 只影响 TitleSubtitle 输出不变。"""
content = build_ass_content(
**self._base_kwargs(),
title_config={"pos_x": 540, "pos_y": 300},
subtitle_text="配音字幕",
subtitle_config={"position": "bottom", "size": 24},
)
sub_style = [line for line in content.splitlines() if line.startswith("Style: SubtitleStyle")][0]
sub_fields = [f.strip() for f in sub_style.split(",")]
assert sub_fields[18] == "2" # bottom
sub_dialogue = [
line for line in content.splitlines() if line.startswith("Dialogue:") and "SubtitleStyle" in line
][0]
assert "\\pos(" not in sub_dialogue
+3 -5
View File
@@ -395,13 +395,11 @@ class TestSmartMatchFiltersExhausted:
# 返回的素材全部 usable=True
assert all(item.usable for item in resp.items)
def test_all_exhausted_falls_back_to_all(self):
"""全部素材已用尽时回退保留全部(不返回空——空结果会让前端回退全选,
反而绕过评分排序;耗尽素材仍可走复用区间)。"""
def test_all_exhausted_returns_empty(self):
"""全部素材已用尽时返回空列表(不报错,前端显示空结果)。"""
assets = [_exhausted_asset("a-ex-1"), _exhausted_asset("a-ex-2")]
resp = self._call(assets)
returned_ids = {item.id for item in resp.items}
assert returned_ids == {"a-ex-1", "a-ex-2"}
assert resp.items == []
assert resp.total_candidates == 2
def test_fresh_assets_all_returned(self):
-221
View File
@@ -1,221 +0,0 @@
"""Tests for duplicate_rate computation and API response."""
import json
import sys
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
# Mock cv2 and numpy before any imports that need them
sys.modules.setdefault("cv2", MagicMock())
ROOT = Path(__file__).resolve().parents[2]
sys.path.insert(0, str(ROOT / "apps" / "api"))
sys.path.insert(0, str(ROOT / "packages"))
sys.path.insert(0, str(ROOT / "apps" / "worker"))
class TestComputeDuplicateRate:
"""Test VideoDeduplicator.compute_duplicate_rate."""
def _make_fingerprint(self, md5="abc123", phashes=None):
from video_processing.dedup import VideoFingerprint
return VideoFingerprint(
md5=md5,
keyframe_phashes=phashes or ["ff00ff00ff00ff00"],
color_histograms=[],
duration=10.0,
resolution=(1920, 1080),
)
def _make_existing_video(self, vid, fingerprint_dict):
from packages.domain import GeneratedVideo
return GeneratedVideo(
id=vid,
project_id="proj1",
generation_task_id="task1",
name=f"video-{vid}",
file_url=f"https://example.com/{vid}.mp4",
file_size=1000,
duration=10.0,
width=1920,
height=1080,
fps=25.0,
video_fingerprint=fingerprint_dict,
)
def test_no_existing_videos_returns_zero(self):
from video_processing.dedup import VideoDeduplicator
deduplicator = VideoDeduplicator()
fingerprint = self._make_fingerprint()
session = MagicMock()
with patch("video_processing.dedup.SQLAlchemyGeneratedVideoRepository") as MockRepo:
mock_repo = MockRepo.return_value
session.query.return_value.filter.return_value.order_by.return_value.limit.return_value.all.return_value = (
[]
)
rate = deduplicator.compute_duplicate_rate(fingerprint, "proj1", "vid1", session)
assert rate == 0.0
def test_md5_match_returns_100(self):
from video_processing.dedup import VideoDeduplicator
from packages.adapters.sqlalchemy_impl.models import GeneratedVideoModel
deduplicator = VideoDeduplicator()
fingerprint = self._make_fingerprint(md5="exact_match_md5")
session = MagicMock()
existing = self._make_existing_video("existing1", {"md5": "exact_match_md5", "keyframe_phashes": ["aa"]})
# Create a mock model with the domain attributes
mock_model = MagicMock(spec=GeneratedVideoModel)
mock_model.id = existing.id
mock_model.project_id = existing.project_id
mock_model.video_fingerprint = existing.video_fingerprint
mock_model.generated_at = "2026-01-01"
with patch("video_processing.dedup.SQLAlchemyGeneratedVideoRepository") as MockRepo:
mock_repo = MockRepo.return_value
mock_repo._to_domain.return_value = existing
# Mock the session.query chain
session.query.return_value.filter.return_value.order_by.return_value.limit.return_value.all.return_value = [
mock_model
]
rate = deduplicator.compute_duplicate_rate(fingerprint, "proj1", "vid1", session)
assert rate == 100.0
def test_phash_similarity_computed(self):
from video_processing.dedup import VideoDeduplicator
from packages.adapters.sqlalchemy_impl.models import GeneratedVideoModel
deduplicator = VideoDeduplicator()
fingerprint = self._make_fingerprint(md5="different_md5", phashes=["ff00ff00ff00ff00"])
session = MagicMock()
existing = self._make_existing_video(
"existing1",
{"md5": "other_md5", "keyframe_phashes": ["ff00ff00ff00ff03"]},
)
mock_model = MagicMock(spec=GeneratedVideoModel)
mock_model.id = existing.id
mock_model.project_id = existing.project_id
mock_model.video_fingerprint = existing.video_fingerprint
mock_model.generated_at = "2026-01-01"
with patch("video_processing.dedup.SQLAlchemyGeneratedVideoRepository") as MockRepo:
mock_repo = MockRepo.return_value
mock_repo._to_domain.return_value = existing
session.query.return_value.filter.return_value.order_by.return_value.limit.return_value.all.return_value = [
mock_model
]
rate = deduplicator.compute_duplicate_rate(fingerprint, "proj1", "vid1", session)
# hamming distance = 2, similarity = (1 - 2/64) * 100 = 96.875
assert rate == pytest.approx(96.88, abs=0.1)
def test_excludes_self_video(self):
from video_processing.dedup import VideoDeduplicator
from packages.adapters.sqlalchemy_impl.models import GeneratedVideoModel
deduplicator = VideoDeduplicator()
fingerprint = self._make_fingerprint(md5="same_md5")
session = MagicMock()
self_video = self._make_existing_video("vid1", {"md5": "same_md5", "keyframe_phashes": ["aa"]})
mock_model = MagicMock(spec=GeneratedVideoModel)
mock_model.id = self_video.id
mock_model.project_id = self_video.project_id
mock_model.video_fingerprint = self_video.video_fingerprint
mock_model.generated_at = "2026-01-01"
with patch("video_processing.dedup.SQLAlchemyGeneratedVideoRepository") as MockRepo:
mock_repo = MockRepo.return_value
mock_repo._to_domain.return_value = self_video
session.query.return_value.filter.return_value.order_by.return_value.limit.return_value.all.return_value = [
mock_model
]
rate = deduplicator.compute_duplicate_rate(fingerprint, "proj1", "vid1", session)
assert rate == 0.0
def test_takes_max_similarity(self):
from video_processing.dedup import VideoDeduplicator
from packages.adapters.sqlalchemy_impl.models import GeneratedVideoModel
deduplicator = VideoDeduplicator()
fingerprint = self._make_fingerprint(md5="new_md5", phashes=["ff00ff00ff00ff00"])
session = MagicMock()
existing1 = self._make_existing_video("e1", {"md5": "md5_1", "keyframe_phashes": ["ff00ff00ff00ff0f"]})
existing2 = self._make_existing_video("e2", {"md5": "md5_2", "keyframe_phashes": ["ff00ff00ff00ff01"]})
mock_model1 = MagicMock(spec=GeneratedVideoModel)
mock_model1.id = existing1.id
mock_model1.project_id = existing1.project_id
mock_model1.video_fingerprint = existing1.video_fingerprint
mock_model1.generated_at = "2026-01-02"
mock_model2 = MagicMock(spec=GeneratedVideoModel)
mock_model2.id = existing2.id
mock_model2.project_id = existing2.project_id
mock_model2.video_fingerprint = existing2.video_fingerprint
mock_model2.generated_at = "2026-01-01"
with patch("video_processing.dedup.SQLAlchemyGeneratedVideoRepository") as MockRepo:
mock_repo = MockRepo.return_value
mock_repo._to_domain.side_effect = [existing1, existing2]
session.query.return_value.filter.return_value.order_by.return_value.limit.return_value.all.return_value = [
mock_model1,
mock_model2,
]
rate = deduplicator.compute_duplicate_rate(fingerprint, "proj1", "vid1", session)
# max similarity: e2 distance=1, (1-1/64)*100 = 98.4375
assert rate == pytest.approx(98.44, abs=0.1)
class TestDuplicateRateAPI:
"""Test that duplicate_rate is returned in API responses."""
def test_video_item_response_has_duplicate_rate(self):
from app.schemas.video_center import VideoItemResponse
resp = VideoItemResponse(
id="v1",
project_id="p1",
generation_task_id="t1",
name="test.mp4",
file_url="https://example.com/test.mp4",
file_size=1000,
duration=10.0,
width=1920,
height=1080,
fps=25.0,
duplicate_rate=75.5,
)
assert resp.duplicate_rate == 75.5
def test_video_item_response_duplicate_rate_default_none(self):
from app.schemas.video_center import VideoItemResponse
resp = VideoItemResponse(
id="v1",
project_id="p1",
generation_task_id="t1",
name="test.mp4",
file_url="https://example.com/test.mp4",
file_size=1000,
duration=10.0,
width=1920,
height=1080,
fps=25.0,
)
assert resp.duplicate_rate is None
@@ -1,243 +0,0 @@
"""_get_template_segments 回退路径测试.
验证三级回退链:
1. 新模板系统(tpl_svc.list_clip_configs)正常 → 直接返回
2. 新模板系统主表不存在(ValueError)→ 直接查 template_clip_configs 表兜底
3. 直接查表也失败 → 回退旧模板系统(template_segments
4. 全部失败 → 返回空列表
覆盖 P0 修复:自建模板在 edit_templates 主表不存在但在 template_clip_configs 有记录时,
from-assets 流程不再 400。
"""
from __future__ import annotations
import os
import sys
from pathlib import Path
from unittest.mock import MagicMock, PropertyMock
os.environ.setdefault("JWT_SECRET_KEY", "unit-test-secret-key-for-testing")
os.environ.setdefault("DATABASE_URL", "sqlite:///test.db")
sys.path.insert(0, str(Path(__file__).resolve().parents[2] / "apps" / "api"))
from app.api.routes.templates_editor.clips import _get_template_segments
TEST_TEMPLATE_ID = "tmpl-orphan-001"
DEFAULT_DUR = 5.0 # _DEFAULT_EDITOR_CLIP_DURATION
def _make_clip_config(order: int, min_dur: float = 3.0, max_dur: float = 8.0):
"""构造 mock TemplateClipConfig 领域实体."""
cc = MagicMock()
cc.order = order
cc.min_duration = min_dur
cc.max_duration = max_dur
return cc
def _make_old_segment(segment_order: int, dur_min: float = 4.0, dur_max: float = 7.0):
"""构造 mock 旧 TemplateSegment."""
s = MagicMock()
s.segment_order = segment_order
s.duration_min = dur_min
s.duration_max = dur_max
return s
# ---------------------------------------------------------------------------
# 测试
# ---------------------------------------------------------------------------
class TestGetTemplateSegmentsFallback:
"""_get_template_segments 三级回退链."""
def test_new_system_works(self):
"""路径1:新模板系统正常返回 → 直接使用."""
configs = [_make_clip_config(0, 2.0, 6.0), _make_clip_config(1, 3.0, 9.0)]
tpl_svc = MagicMock()
tpl_svc.list_clip_configs.return_value = configs
db = MagicMock()
result = _get_template_segments(TEST_TEMPLATE_ID, tpl_svc, db)
assert len(result) == 2
assert result[0] == (0, 2.0, 6.0)
assert result[1] == (1, 3.0, 9.0)
tpl_svc.list_clip_configs.assert_called_once_with(TEST_TEMPLATE_ID)
def test_main_table_missing_direct_query_succeeds(self):
"""路径2(P0修复):主表不存在 ValueError → 直接查表成功.
模拟自建模板在 edit_templates 主表已删除/不存在,
但 template_clip_configs 表有记录。
"""
tpl_svc = MagicMock()
tpl_svc.list_clip_configs.side_effect = ValueError(f"模板不存在: {TEST_TEMPLATE_ID}")
db = MagicMock()
# Mock SQLAlchemyTemplateClipConfigRepository
direct_configs = [
_make_clip_config(0, 2.0, 5.0),
_make_clip_config(1, 3.0, 7.0),
_make_clip_config(2, 4.0, 8.0),
]
with (
__import__("unittest.mock", fromlist=["patch"]).patch(
"app.api.routes.templates_editor.clips.SQLAlchemyTemplateClipConfigRepository"
) as mock_repo_cls,
):
mock_repo = MagicMock()
mock_repo.list_by_template.return_value = direct_configs
mock_repo_cls.return_value = mock_repo
result = _get_template_segments(TEST_TEMPLATE_ID, tpl_svc, db)
assert len(result) == 3
assert result[0] == (0, 2.0, 5.0)
assert result[1] == (1, 3.0, 7.0)
assert result[2] == (2, 4.0, 8.0)
mock_repo.list_by_template.assert_called_once_with(TEST_TEMPLATE_ID)
def test_main_table_missing_direct_query_empty_falls_to_old(self):
"""路径2→3:主表不存在 + 直接查表为空 → 回退旧系统."""
tpl_svc = MagicMock()
tpl_svc.list_clip_configs.side_effect = ValueError("模板不存在")
db = MagicMock()
old_segments = [_make_old_segment(0, 3.0, 6.0)]
with __import__("unittest.mock", fromlist=["patch"]).patch(
"app.api.routes.templates_editor.clips.SQLAlchemyTemplateClipConfigRepository"
) as mock_repo_cls:
mock_repo = MagicMock()
mock_repo.list_by_template.return_value = [] # 新表也没记录
mock_repo_cls.return_value = mock_repo
with __import__("unittest.mock", fromlist=["patch"]).patch(
"app.api.routes.templates_editor.clips.SQLAlchemyTemplateRepository"
) as mock_old_cls:
mock_old = MagicMock()
mock_old.list_segments.return_value = old_segments
mock_old_cls.return_value = mock_old
result = _get_template_segments(TEST_TEMPLATE_ID, tpl_svc, db)
assert len(result) == 1
assert result[0] == (0, 3.0, 6.0)
def test_all_fail_returns_empty(self):
"""路径4:三级全部失败 → 返回空列表."""
tpl_svc = MagicMock()
tpl_svc.list_clip_configs.side_effect = ValueError("模板不存在")
db = MagicMock()
with __import__("unittest.mock", fromlist=["patch"]).patch(
"app.api.routes.templates_editor.clips.SQLAlchemyTemplateClipConfigRepository"
) as mock_repo_cls:
mock_repo = MagicMock()
mock_repo.list_by_template.side_effect = Exception("DB error")
mock_repo_cls.return_value = mock_repo
with __import__("unittest.mock", fromlist=["patch"]).patch(
"app.api.routes.templates_editor.clips.SQLAlchemyTemplateRepository"
) as mock_old_cls:
mock_old = MagicMock()
mock_old.list_segments.return_value = [] # 旧表也空
mock_old_cls.return_value = mock_old
result = _get_template_segments(TEST_TEMPLATE_ID, tpl_svc, db)
assert result == []
def test_direct_query_sorts_by_order(self):
"""直接查表返回的结果按 order 排序."""
tpl_svc = MagicMock()
tpl_svc.list_clip_configs.side_effect = ValueError("模板不存在")
db = MagicMock()
# 故意乱序
configs = [
_make_clip_config(2, 5.0, 10.0),
_make_clip_config(0, 2.0, 4.0),
_make_clip_config(1, 3.0, 6.0),
]
with __import__("unittest.mock", fromlist=["patch"]).patch(
"app.api.routes.templates_editor.clips.SQLAlchemyTemplateClipConfigRepository"
) as mock_repo_cls:
mock_repo = MagicMock()
mock_repo.list_by_template.return_value = configs
mock_repo_cls.return_value = mock_repo
result = _get_template_segments(TEST_TEMPLATE_ID, tpl_svc, db)
assert [r[0] for r in result] == [0, 1, 2]
assert result[0] == (0, 2.0, 4.0)
assert result[1] == (1, 3.0, 6.0)
assert result[2] == (2, 5.0, 10.0)
def test_direct_query_handles_none_durations(self):
"""直接查表时 min/max_duration 为 None → 使用默认值."""
tpl_svc = MagicMock()
tpl_svc.list_clip_configs.side_effect = ValueError("模板不存在")
db = MagicMock()
cc = MagicMock()
cc.order = 0
cc.min_duration = None
cc.max_duration = None
with __import__("unittest.mock", fromlist=["patch"]).patch(
"app.api.routes.templates_editor.clips.SQLAlchemyTemplateClipConfigRepository"
) as mock_repo_cls:
mock_repo = MagicMock()
mock_repo.list_by_template.return_value = [cc]
mock_repo_cls.return_value = mock_repo
result = _get_template_segments(TEST_TEMPLATE_ID, tpl_svc, db)
assert len(result) == 1
# None → default (5.0), max(None or None) → default (5.0)
assert result[0] == (0, DEFAULT_DUR, DEFAULT_DUR)
def test_new_system_returns_empty_tries_direct(self):
"""新模板系统返回空列表(非异常)→ 继续尝试直接查表."""
tpl_svc = MagicMock()
tpl_svc.list_clip_configs.return_value = [] # 空列表,非异常
db = MagicMock()
direct_configs = [_make_clip_config(0, 3.0, 6.0)]
with __import__("unittest.mock", fromlist=["patch"]).patch(
"app.api.routes.templates_editor.clips.SQLAlchemyTemplateClipConfigRepository"
) as mock_repo_cls:
mock_repo = MagicMock()
mock_repo.list_by_template.return_value = direct_configs
mock_repo_cls.return_value = mock_repo
result = _get_template_segments(TEST_TEMPLATE_ID, tpl_svc, db)
# 新系统返回空 → 不走 except → 但也没 return → 继续往下走
# 直接查表有数据 → 返回
assert len(result) == 1
assert result[0] == (0, 3.0, 6.0)
def test_existing_template_unaffected(self):
"""正常模板(主表存在)行为不变."""
configs = [_make_clip_config(0, 2.0, 5.0)]
tpl_svc = MagicMock()
tpl_svc.list_clip_configs.return_value = configs
db = MagicMock()
with __import__("unittest.mock", fromlist=["patch"]).patch(
"app.api.routes.templates_editor.clips.SQLAlchemyTemplateClipConfigRepository"
) as mock_repo_cls:
result = _get_template_segments(TEST_TEMPLATE_ID, tpl_svc, db)
# 直接查表不应被调用(新系统已返回)
mock_repo_cls.assert_not_called()
assert len(result) == 1
assert result[0] == (0, 2.0, 5.0)
-93
View File
@@ -1,93 +0,0 @@
"""Tests for get_asset_recent_use_counts and smart-match high-use exclusion."""
import json
import sys
from pathlib import Path
from unittest.mock import MagicMock, patch
import pytest
ROOT = Path(__file__).resolve().parents[2]
sys.path.insert(0, str(ROOT / "apps" / "api"))
sys.path.insert(0, str(ROOT / "packages"))
class TestGetAssetRecentUseCounts:
"""Test asset_segment_tracker.get_asset_recent_use_counts."""
def _make_asset_model(self, asset_id, used_time_ranges=None):
from packages.adapters.sqlalchemy_impl.models import AssetModel
model = AssetModel(
id=asset_id,
name=f"asset-{asset_id}",
file_type="video",
status="ready",
asset_library_id="lib1",
project_id="proj1",
file_size=1000,
file_url=f"https://example.com/{asset_id}.mp4",
uploaded_by_user_id="user1",
)
model.classification_result = json.dumps(
{
"used_time_ranges": used_time_ranges or [],
}
)
return model
def test_empty_asset_ids_returns_empty(self):
from app.services.asset_segment_tracker import get_asset_recent_use_counts
db = MagicMock()
result = get_asset_recent_use_counts(db, [])
assert result == {}
def test_no_usage_returns_zero(self):
from app.services.asset_segment_tracker import get_asset_recent_use_counts
db = MagicMock()
model = self._make_asset_model("a1", [])
db.query.return_value.filter.return_value.all.return_value = [model]
result = get_asset_recent_use_counts(db, ["a1"])
assert result == {"a1": 0}
def test_counts_distinct_plan_ids(self):
from app.services.asset_segment_tracker import get_asset_recent_use_counts
db = MagicMock()
ranges = [
{"start": 0, "end": 5, "plan_id": "plan1", "created_at": "2026-08-01T00:00:00"},
{"start": 5, "end": 10, "plan_id": "plan1", "created_at": "2026-08-01T00:01:00"},
{"start": 0, "end": 5, "plan_id": "plan2", "created_at": "2026-08-02T00:00:00"},
{"start": 0, "end": 5, "plan_id": "plan3", "created_at": "2026-08-03T00:00:00"},
]
model = self._make_asset_model("a1", ranges)
db.query.return_value.filter.return_value.all.return_value = [model]
result = get_asset_recent_use_counts(db, ["a1"])
assert result == {"a1": 3}
def test_limits_to_recent_n(self):
from app.services.asset_segment_tracker import get_asset_recent_use_counts
db = MagicMock()
ranges = [
{"start": 0, "end": 5, "plan_id": f"plan{i}", "created_at": f"2026-08-{i+1:02d}T00:00:00"}
for i in range(10)
]
model = self._make_asset_model("a1", ranges)
db.query.return_value.filter.return_value.all.return_value = [model]
result = get_asset_recent_use_counts(db, ["a1"], recent_video_count=5)
assert result["a1"] == 5
def test_missing_asset_defaults_to_zero(self):
from app.services.asset_segment_tracker import get_asset_recent_use_counts
db = MagicMock()
db.query.return_value.filter.return_value.all.return_value = []
result = get_asset_recent_use_counts(db, ["missing_asset"])
assert result == {"missing_asset": 0}
-258
View File
@@ -1,258 +0,0 @@
"""smart-match 过滤回退策略测试:余量过滤/高频排除导致结果集为空时必须回退。
线上事故:路由旧实现先 smart_select_assets(limit=N) 截取,再对这 N 条做
usable / 高频过滤,过滤后不回补——排名靠前素材全部被排除时返回空 items,
前端回退全选。修复后过滤全部前置,且每级过滤后为空/不足时回退保留。
"""
import json
import os
import sys
from pathlib import Path
from unittest.mock import MagicMock
os.environ.setdefault("JWT_SECRET_KEY", "unit-test-secret-key-for-testing")
os.environ.setdefault("DATABASE_URL", "sqlite:///test.db")
import pytest
from fastapi import FastAPI
from fastapi.testclient import TestClient
ROOT = Path(__file__).resolve().parents[2]
sys.path.insert(0, str(ROOT / "apps" / "api"))
sys.path.insert(0, str(ROOT / "packages"))
from app.api.routes.assets import router
from app.auth import AuthenticatedUser, get_current_user
from app.core.storage import get_storage_service
from app.dependencies import (
get_asset_library_repository,
get_asset_repository,
get_project_repository,
)
from packages.domain import (
Asset,
AssetLibrary,
AssetLibraryKind,
AssetStatus,
Project,
User,
)
class _StubProjectRepo:
def __init__(self, projects):
self._projects = projects
def find_by_id(self, pid):
return self._projects.get(pid)
class _StubAssetLibraryRepo:
def __init__(self, libraries):
self._libraries = libraries
def get(self, lid):
return self._libraries.get(lid)
class _StubAssetRepo:
"""模拟仓储;session 属性供 get_asset_recent_use_counts 使用(测试中会被 patch)。"""
def __init__(self, assets):
self._assets = assets
self.session = MagicMock(name="stub-session")
def find_by_library(self, lid, skip=0, limit=100, status=None):
result = [a for a in self._assets if a.library_id == lid]
if status:
result = [a for a in result if (a.status.value if hasattr(a.status, "value") else a.status) in status]
return result[skip : skip + limit]
def find_by_library_and_file_type(self, lid, file_type, skip=0, limit=100, status=None):
result = [a for a in self._assets if a.library_id == lid and a.file_type == file_type]
if status:
result = [a for a in result if (a.status.value if hasattr(a.status, "value") else a.status) in status]
return result[skip : skip + limit]
def _make_app(asset_repo, lib_repo, proj_repo):
app = FastAPI()
app.include_router(router, prefix="/assets")
fake_user = MagicMock()
fake_user.user = User(id="user-1", email="test@test.com", display_name="Test")
app.dependency_overrides[get_current_user] = lambda: AuthenticatedUser(user=fake_user.user)
app.dependency_overrides[get_asset_repository] = lambda: asset_repo
app.dependency_overrides[get_asset_library_repository] = lambda: lib_repo
app.dependency_overrides[get_project_repository] = lambda: proj_repo
app.dependency_overrides[get_storage_service] = lambda: MagicMock()
return app
def _library():
return AssetLibrary(id="lib-1", project_id="proj-1", name="Videos", kind=AssetLibraryKind.VIDEO)
def _exhausted_ranges(duration=15.0):
"""构造 used_time_ranges:整片覆盖 + 每区间 use_count 达上限 3 → usable=False。"""
return [
{"start": 0.0, "end": duration, "use_count": 3, "plan_id": "p1"},
]
def _video_asset(name, duration=15.0, quality=90, used_ranges=None):
meta = {"used_time_ranges": used_ranges} if used_ranges is not None else {}
return Asset.create(
project_id="proj-1",
library_id="lib-1",
name=name,
storage_key=f"key-{name}",
mime_type="video/mp4",
metadata=meta,
quality_score=quality,
duration=duration,
status=AssetStatus.READY,
)
class TestSmartMatchAvailabilityFallback:
"""余量过滤回退:全部素材 usable=False 时不返回空。"""
def test_all_exhausted_returns_assets_instead_of_empty(self):
project = Project(id="proj-1", name="Test", owner_user_id="user-1")
assets = [
_video_asset("v1.mp4", used_ranges=_exhausted_ranges(15)),
_video_asset("v2.mp4", used_ranges=_exhausted_ranges(25)),
]
repo = _StubAssetRepo(assets)
app = _make_app(repo, _StubAssetLibraryRepo({"lib-1": _library()}), _StubProjectRepo({"proj-1": project}))
with TestClient(app) as client:
resp = client.post("/assets/smart-match", json={"library_id": "lib-1"})
assert resp.status_code == 200, resp.text
data = resp.json()
# 回退:保留全部候选,不返回空
assert len(data["items"]) == 2
assert data["total_candidates"] == 2
def test_mixed_exhausted_and_fresh_excludes_exhausted(self):
project = Project(id="proj-1", name="Test", owner_user_id="user-1")
assets = [
_video_asset("exhausted.mp4", quality=99, used_ranges=_exhausted_ranges(15)),
_video_asset("fresh.mp4", quality=50, used_ranges=None),
]
repo = _StubAssetRepo(assets)
app = _make_app(repo, _StubAssetLibraryRepo({"lib-1": _library()}), _StubProjectRepo({"proj-1": project}))
with TestClient(app) as client:
resp = client.post("/assets/smart-match", json={"library_id": "lib-1"})
assert resp.status_code == 200, resp.text
names = [item["name"] for item in resp.json()["items"]]
assert "fresh.mp4" in names
assert "exhausted.mp4" not in names
def test_limit_backfills_from_lower_ranked_when_top_exhausted(self):
"""limit=1 且排名第一的素材耗尽时,必须回补排名靠后的可用素材,不返回空。"""
project = Project(id="proj-1", name="Test", owner_user_id="user-1")
assets = [
_video_asset("top-exhausted.mp4", quality=100, used_ranges=_exhausted_ranges(15)),
_video_asset("second-fresh.mp4", quality=40, used_ranges=None),
_video_asset("third-fresh.mp4", quality=30, used_ranges=None),
]
repo = _StubAssetRepo(assets)
app = _make_app(repo, _StubAssetLibraryRepo({"lib-1": _library()}), _StubProjectRepo({"proj-1": project}))
with TestClient(app) as client:
resp = client.post("/assets/smart-match", json={"library_id": "lib-1", "limit": 1})
assert resp.status_code == 200, resp.text
data = resp.json()
assert len(data["items"]) == 1
# 不能是空,也不能是耗尽的高分素材
assert data["items"][0]["name"] == "second-fresh.mp4"
class TestSmartMatchHighFreqFallback:
"""高频排除回退:排除后为空/不足 limit 时保留全部可用素材。"""
def test_all_high_freq_keeps_all(self, monkeypatch):
import app.api.routes.assets as routes_mod
project = Project(id="proj-1", name="Test", owner_user_id="user-1")
assets = [_video_asset("v1.mp4", quality=90), _video_asset("v2.mp4", quality=80)]
repo = _StubAssetRepo(assets)
# 全部素材在最近 5 个视频中使用 5 次(> 3)
fake_counts = {assets[0].id: 5, assets[1].id: 5}
monkeypatch.setattr(
routes_mod, "get_asset_recent_use_counts", lambda db, asset_ids, recent_video_count=5: fake_counts
)
app = _make_app(repo, _StubAssetLibraryRepo({"lib-1": _library()}), _StubProjectRepo({"proj-1": project}))
with TestClient(app) as client:
resp = client.post("/assets/smart-match", json={"library_id": "lib-1"})
assert resp.status_code == 200, resp.text
data = resp.json()
# 全部高频 → 回退保留全部
assert len(data["items"]) == 2
def test_high_freq_partial_exclusion_with_enough_remaining(self, monkeypatch):
import app.api.routes.assets as routes_mod
project = Project(id="proj-1", name="Test", owner_user_id="user-1")
assets = [
_video_asset("hot.mp4", quality=99),
_video_asset("cool1.mp4", quality=80),
_video_asset("cool2.mp4", quality=70),
]
repo = _StubAssetRepo(assets)
fake_counts = {assets[0].id: 9, assets[1].id: 1, assets[2].id: 0}
monkeypatch.setattr(
routes_mod, "get_asset_recent_use_counts", lambda db, asset_ids, recent_video_count=5: fake_counts
)
app = _make_app(repo, _StubAssetLibraryRepo({"lib-1": _library()}), _StubProjectRepo({"proj-1": project}))
with TestClient(app) as client:
resp = client.post("/assets/smart-match", json={"library_id": "lib-1"})
assert resp.status_code == 200, resp.text
names = [item["name"] for item in resp.json()["items"]]
assert "hot.mp4" not in names
assert set(names) == {"cool1.mp4", "cool2.mp4"}
def test_high_freq_insufficient_for_limit_keeps_all(self, monkeypatch):
"""3 个素材、limit=5、2 个高频 → 剩余 1 < limit → 保留全部。"""
import app.api.routes.assets as routes_mod
project = Project(id="proj-1", name="Test", owner_user_id="user-1")
assets = [
_video_asset("hot1.mp4", quality=99),
_video_asset("hot2.mp4", quality=98),
_video_asset("cool.mp4", quality=50),
]
repo = _StubAssetRepo(assets)
fake_counts = {assets[0].id: 8, assets[1].id: 7, assets[2].id: 0}
monkeypatch.setattr(
routes_mod, "get_asset_recent_use_counts", lambda db, asset_ids, recent_video_count=5: fake_counts
)
app = _make_app(repo, _StubAssetLibraryRepo({"lib-1": _library()}), _StubProjectRepo({"proj-1": project}))
with TestClient(app) as client:
resp = client.post("/assets/smart-match", json={"library_id": "lib-1", "limit": 5})
assert resp.status_code == 200, resp.text
names = [item["name"] for item in resp.json()["items"]]
# 剩余 1 < limit=5 → 回退保留全部 3 条
assert set(names) == {"hot1.mp4", "hot2.mp4", "cool.mp4"}
def test_high_freq_query_failure_skips_exclusion(self, monkeypatch):
import app.api.routes.assets as routes_mod
def _boom(db, asset_ids, recent_video_count=5):
raise RuntimeError("DB down")
project = Project(id="proj-1", name="Test", owner_user_id="user-1")
assets = [_video_asset("v1.mp4", quality=90), _video_asset("v2.mp4", quality=80)]
repo = _StubAssetRepo(assets)
monkeypatch.setattr(routes_mod, "get_asset_recent_use_counts", _boom)
app = _make_app(repo, _StubAssetLibraryRepo({"lib-1": _library()}), _StubProjectRepo({"proj-1": project}))
with TestClient(app) as client:
resp = client.post("/assets/smart-match", json={"library_id": "lib-1"})
assert resp.status_code == 200, resp.text
assert len(resp.json()["items"]) == 2
-209
View File
@@ -1,209 +0,0 @@
"""统一模板 segments 数据源单元测试。
验证 template_repository 从 template_clip_configs 读取 segments
写入走 template_clip_configs,回退兼容 template_segments。
"""
from __future__ import annotations
import sys
import uuid
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parents[2] / "apps" / "api"))
import pytest
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from packages.adapters.sqlalchemy_impl.models import (
Base,
TemplateClipConfigModel,
TemplateModel,
TemplateSegmentModel,
)
from packages.adapters.sqlalchemy_impl.template_repository import (
SQLAlchemyTemplateRepository,
)
from packages.domain.template import Template, TemplateSegment
@pytest.fixture()
def session():
engine = create_engine("sqlite:///:memory:")
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
s = Session()
try:
yield s
finally:
s.close()
@pytest.fixture()
def repo(session):
return SQLAlchemyTemplateRepository(session)
def _make_template(template_id=None, user_id="u1", name="测试模板", mode="one_take"):
tid = template_id or str(uuid.uuid4())
return Template(
id=tid,
user_id=user_id,
name=name,
mode=mode,
category="",
tags=[],
estimated_duration=30.0,
is_active=True,
segments=[],
)
def _make_segment(template_id, order=1, material_type=None):
return TemplateSegment(
id=str(uuid.uuid4()),
template_id=template_id,
segment_order=order,
duration_min=5.0,
duration_max=10.0,
material_type=material_type,
)
class TestCreateSegments:
def test_writes_to_clip_configs(self, repo, session):
tpl = _make_template()
repo.create(tpl)
seg = _make_segment(tpl.id, order=1)
repo.create_segments([seg])
clips = session.query(TemplateClipConfigModel).filter(TemplateClipConfigModel.template_id == tpl.id).all()
assert len(clips) == 1
assert clips[0].clip_type == "main"
assert clips[0].order == 1
assert clips[0].min_duration == 5.0
def test_material_type_stored_in_config(self, repo, session):
tpl = _make_template()
repo.create(tpl)
seg = _make_segment(tpl.id, order=1, material_type="voiceover")
repo.create_segments([seg])
clip = session.query(TemplateClipConfigModel).filter(TemplateClipConfigModel.template_id == tpl.id).first()
assert clip.config["material_type"] == "voiceover"
class TestListSegments:
def test_reads_from_clip_configs(self, repo, session):
tpl = _make_template()
repo.create(tpl)
seg = _make_segment(tpl.id, order=1, material_type="voiceover")
repo.create_segments([seg])
result = repo.list_segments(tpl.id)
assert len(result) == 1
assert result[0].material_type == "voiceover"
def test_fallback_to_old_table(self, repo, session):
tpl = _make_template()
repo.create(tpl)
old = TemplateSegmentModel(
id=str(uuid.uuid4()),
template_id=tpl.id,
segment_order=1,
duration_min=3.0,
duration_max=8.0,
material_type="场景",
)
session.add(old)
session.commit()
result = repo.list_segments(tpl.id)
assert len(result) == 1
assert result[0].material_type == "场景"
def test_clip_configs_takes_priority(self, repo, session):
tpl = _make_template()
repo.create(tpl)
seg = _make_segment(tpl.id, order=1)
repo.create_segments([seg])
old = TemplateSegmentModel(
id=str(uuid.uuid4()), template_id=tpl.id, segment_order=1, duration_min=1.0, duration_max=2.0
)
session.add(old)
session.commit()
result = repo.list_segments(tpl.id)
assert len(result) == 1
assert result[0].duration_min == 5.0
class TestListByUser:
def test_batch_loads_from_clip_configs(self, repo, session):
tpl = _make_template()
repo.create(tpl)
seg = _make_segment(tpl.id, order=1, material_type="人物")
repo.create_segments([seg])
result = repo.list_by_user("u1")
assert len(result) == 1
assert len(result[0].segments) == 1
assert result[0].segments[0].material_type == "人物"
def test_fallback_for_old_data(self, repo, session):
tpl = _make_template()
repo.create(tpl)
old = TemplateSegmentModel(
id=str(uuid.uuid4()), template_id=tpl.id, segment_order=1, duration_min=2.0, duration_max=6.0
)
session.add(old)
session.commit()
result = repo.list_by_user("u1")
assert len(result) == 1
assert len(result[0].segments) == 1
assert result[0].segments[0].duration_min == 2.0
class TestCopyTemplate:
def test_copy_writes_to_clip_configs(self, repo, session):
tpl = _make_template()
repo.create(tpl)
seg = _make_segment(tpl.id, order=1, material_type="voiceover")
repo.create_segments([seg])
copied = repo.copy_template(tpl.id, "u1", "副本模板")
assert copied.id != tpl.id
clips = session.query(TemplateClipConfigModel).filter(TemplateClipConfigModel.template_id == copied.id).all()
assert len(clips) == 1
assert clips[0].config["material_type"] == "voiceover"
def test_copy_empty_segments(self, repo, session):
tpl = _make_template()
repo.create(tpl)
copied = repo.copy_template(tpl.id, "u1", "空副本")
assert len(copied.segments) == 0
class TestDelete:
def test_delete_cleans_both_tables(self, repo, session):
tpl = _make_template()
repo.create(tpl)
seg = _make_segment(tpl.id, order=1)
repo.create_segments([seg])
old = TemplateSegmentModel(
id=str(uuid.uuid4()), template_id=tpl.id, segment_order=1, duration_min=1.0, duration_max=2.0
)
session.add(old)
session.commit()
repo.delete(tpl.id, "u1")
c1 = session.query(TemplateClipConfigModel).filter(TemplateClipConfigModel.template_id == tpl.id).count()
c2 = session.query(TemplateSegmentModel).filter(TemplateSegmentModel.template_id == tpl.id).count()
assert c1 == 0
assert c2 == 0
def test_delete_segments_by_template(self, repo, session):
tpl = _make_template()
repo.create(tpl)
seg = _make_segment(tpl.id, order=1)
repo.create_segments([seg])
old = TemplateSegmentModel(
id=str(uuid.uuid4()), template_id=tpl.id, segment_order=2, duration_min=1.0, duration_max=2.0
)
session.add(old)
session.commit()
count = repo.delete_segments_by_template(tpl.id)
assert count == 2