Skip to content

feat: Support local LLM inference engines (Ollama, LM Studio / OpenAI-compatible API) #31

Description

@CodeMaverick-143

Feature Request

Add support for local AI models (e.g., Ollama running llama3, deepseek-coder, qwen, or LM Studio) so users can analyze private repositories locally without sending proprietary code to cloud endpoints.

Proposed CLI Options

# Using Ollama
insight . --provider ollama --model llama3

# Using custom OpenAI-compatible endpoint (LM Studio, vLLM)
insight . --provider local --api-base http://localhost:1234/v1 --model local-model

Proposed Architecture

Abstract the LLM provider in insight/detector.py:

  • GeminiProvider: Uses Google Generative AI
  • OllamaProvider: Connects to http://localhost:11434/api/generate
  • OpenAICompatibleProvider: Connects to any OpenAI-compatible /v1/chat/completions endpoint

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions