🎯 Implement builtin analyze_image tool for vision

[up-guillaume]

Problem

Agents need to analyze images and PDFs they generate locally. Current vision tools (MMX, Z.AI) cannot read local files because they're HTTP MCPs running on LiteLLM (Hetzner), not on the backend server.

Example:

Geo generates preview.pdf → wants to analyze it → ❌ MMX can't read workspace/preview.pdf

Discovery: Vision System Already Exists!

File: backend/app/services/vision_inject.py

Clawith already has a vision system for AgentBay screenshots:

# Pattern actuel :
1. Agent appelle un tool screenshot (AgentBay)
2. store_temp_screenshot(image_data) → stocke en cache mémoire
3. Retourne [ImageID: <uuid>]
4. try_inject_screenshot_vision(messages) → injecte l'image dans le contexte LLM
5. LLM (Qwen, GPT-4o) analyse l'image

Current Limitation

Whitelist hardcodée dans SCREENSHOT_TOOL_NAMES :

SCREENSHOT_TOOL_NAMES = {
    "agentbay_browser_screenshot",
    "agentbay_computer_screenshot",
    "agentbay_computer_precision_screenshot",
}

Seuls les screenshots AgentBay bénéficient du vision.

Recommended Solution

Create a builtin tool analyze_image that uses the same pattern:

# backend/app/tools/analyze_image.py
async def analyze_image(
    file_path: str,
    prompt: str = "Describe this image in detail",
    agent_id: str = None,
    db: AsyncSession = None
) -> dict:
    # 1. Read the file from workspace
    full_path = Path(f"/data/agents/{agent_id}/workspace/{file_path}")
    image_data = full_path.read_bytes()
    
    # 2. Store in vision cache (existing function!)
    image_id = store_temp_screenshot(
        image_data=image_data,
        metadata={"source": "analyze_image", "file_path": file_path},
        agent_id=agent_id
    )
    
    # 3. Return ID for LLM injection
    return {"success": True, "image_id": image_id}

Advantages

✅ Universal - Works for all agents
✅ Reads local files - Direct access to workspace
✅ Uses existing vision system - No reinvention
✅ 1 step - No need for upload_static_file + MMX
✅ Secure - Files not exposed publicly
✅ Supports all formats - PDF, PNG, JPG, etc.

Implementation Checklist

• Create backend/app/tools/analyze_image.py
• Add to BUILTIN_TOOLS in tool_seeder.py
• Auto-assign to all agents
• Test with Geo Presence
• Document in TOOLS.md
• Update MCP_STATUS.md

Documentation

Full analysis: docs/VISION_TOOLS_ANALYSIS.md

Priority

🟡 P2 - Important but not urgent