fix(vlm): 图片VLM分析牛头不对马嘴 — 改用视觉模型 + prompt结构化强化 #2114

Merged

xiaoxia merged 6 commits from fix/2114-vlm-vision-model-fix into develop 2026-10-01 10:32:48 +08:00

Conversation 2 Commits 6 Files Changed 10

+264 -46

6 Commits

Author	SHA1	Message	Date
CI Bot	a9cbe7d4c9	style: auto-format with black + isort + ruff + prettier [skip ci-format-check]	2026-10-01 02:11:07 +00:00
xiaoxia	3a8ef857ac	fix(vlm): call_vision 走视觉模型而非文本模型 + 图片分析 prompt 结构化强化 ... 根因（VLM 牛头不对马嘴）：packages/shared/ai_service.call_vision() 之前误用 client.chat_completion（文本模型 doubao-seed-1.6-250615）发送多模态 content list。 文本模型不认图片 content type → 返回 None → _step_image_analysis 的 except 静默吞掉 → fallback 到占位结果 {name:'未识别'} → 后续 intent/copy/storyboard 完全没图信息，自然胡编。 修复： 1) call_vision 改用 client.vision_completion（模型 doubao-1-5-vision-pro-250915）， 走标准 OpenAI 多模态 chat/completions + image_url 格式 2) 超时提到 60s，温度降到 0.2，解析 ```json 代码块包裹 3) 增加详细 INFO 日志：打印 vision_model/image_url/prompt_len/原始返回前 400 字， 方便下次直接在 worker 日志排查 4) 重写 _IMAGE_ANALYSIS_PROMPT：强制结构化 JSON schema（category/name/brand/colors/ material_or_texture/key_features/visual_style/scene/target_audience_hint/ text_on_image），明确『无法判断就填无法判断，不许编造』，key_features 只能写外观可见特征、不许编功效 5) _step_image_analysis 健壮化：images 为空/None/文本返回/异常分别落 _source 标记； 每张图独立 try/except，单张失败不影响其他图 6) 下游 intent_parsing/copy_fusion/storyboard 兼容新字段 key_features/brand/ category/colors/visual_style（同时向后兼容旧 features 字段）	2026-10-01 10:06:59 +08:00
xiaoxia	3d8882c479	fix(tests+robustness): mock-safe status_code int-cast in ai_client; seedance test responses default 200 ... - test_ai_client_video: fake task/poll responses explicitly set status_code=200/text='' to avoid MagicMock()>=int TypeError leaking into retry logic - ai_client.video_generation: wrap resp.status_code comparisons in try/except (int() cast) so any non-int status (MagicMock in tests) degrades to 200 instead of bombing out with TypeError during create/poll	2026-10-01 04:59:29 +08:00
CI Bot	a74be7c717	style: auto-format with black + isort + ruff + prettier [skip ci-format-check]	2026-09-30 20:43:44 +00:00
xiaoxia	09b8b2990f	test: fix unit tests to match TTS import path and storage_service.get_url mock ... - test_viral_video_p0: patch new apps.worker.services.tts_service_factory path, assert format='mp3' is passed - test_prepare_dedup: storage_service.get_url.return_value = '' so DirectUploadPrepareResponse.url: str doesn't receive a MagicMock	2026-10-01 04:38:58 +08:00
xiaoxia	cdce1b2e10	fix(viral-video): fix 6 E2E bugs — fusion_level alias, concat silent segments, ingest lookup, duplicated URL, TTS voice/format, Seedance first-frame ratio ... Bug1 (P0): schema accepts 'full_ai' as alias for 'ai_full' (Pydantic field_validator normalizes) Bug2 (P0): concat_video_files probes each segment audio stream; Seedance gen_audio=False segments now marked has_audio=False so concat filter uses aevalsrc silence instead of failing with ffmpeg exit 234 Bug3 (P1): find_by_storage_key now queries (storage_key OR file_url) to cover historical data where the legacy file_url column held assets/<project>/<date>/... paths Bug4 (P1): duplicated-hit response no longer accesses non-existent domain Asset.file_url; new helper _get_existing_asset_url uses storage_key (fallback file_url) through storage_service.get_url() Bug5 (P1): _step_tts passes job.persona_id as voice_id (default longxiaochun_v3) and forces format='mp3' so downstream ffmpeg -map 1:a:0 works regardless of provider Bug6 (P1): call_video_generation omits ratio param in first-frame (image_url) mode; ai_client.video_generation ratio becomes Optional[str] and is omitted from payload when None, fixing 400 InvalidParameter from Seedance	2026-10-01 04:10:06 +08:00