Skip to content

Allow /model to switch between multiple models, including BYOK/local providers, in one session #3709

@juancarlosjr97

Description

@juancarlosjr97

Describe the feature or problem you'd like to solve

BYOK mode pins a session to a single model through COPILOT_MODEL, and the /model picker only lists GitHub-hosted models. It does not list the models served by a configured local BYOK provider. As a result, I cannot select a local model from /model, and I cannot switch models without quitting the CLI and re-exporting COPILOT_MODEL. I would like the CLI to know about models from both sources, GitHub-hosted and local, and let me switch between all of them in-session through /model.

Proposed solution

Make the CLI aware of multiple models across sources and selectable at runtime through /model. Today BYOK and offline mode (#1219) pin a session to one model, so there is no way to switch in-session or to mix a fast local model with a remote one in the same conversation. A suggested approach:

  1. Enumerate BYOK models from the provider's /v1/models endpoint (OpenAI-compatible) and merge them into the /model picker alongside GitHub-hosted models, with a source label:
    /model
      gpt-5.1                     (github)
      claude-opus-4.8             (github)
      google/gemma-4-12b          (local)
      qwen3-coder-30b             (local)
    
  2. Allow an explicit list or opt-out for users who do not want auto-enumeration, for example a models array in ~/.github/copilot-cli.json or COPILOT_MODELS="github/gpt-5.1,local/google/gemma-4-12b".
  3. In-session switching through /model <name> re-points subsequent requests to the chosen model and provider, with no restart and no loss of conversation context.
  4. Coexistence of remote and local: when not in COPILOT_OFFLINE=true, list both GitHub-hosted and local BYOK models together; in strict offline mode, list only the local provider's models.

Benefit: users get the same model-switching experience they already have with GitHub-hosted models, extended to local endpoints, so they can pick the right model for cost, speed, or privacy per task without restarting or re-exporting environment variables.

Example prompts or workflows

export COPILOT_PROVIDER_BASE_URL="http://127.0.0.1:1234/v1"
export COPILOT_PROVIDER_API_KEY="local"
# not setting COPILOT_OFFLINE so remote and local are both available
export COPILOT_MODELS="github/gpt-5.1,local/google/gemma-4-12b"
copilot
/model                       # lists github models and local models
/model google/gemma-4-12b    # switch to a local model, no restart
/model gpt-5.1               # switch back to a remote model mid-session
  • Quick refactor on a local model, then switch to a remote model for a complex design discussion in the same session with the same context.
  • Keep sensitive code on a local model while still being able to reach for a remote model when needed.

Additional context

Related issues: #1219 (offline mode, which gave us environment-variable BYOK but a single model), #3624 (BYOM registration for local endpoints, which was closed but the picker gap remains), and #3682 (BYOK credential refresh, which is complementary and a different problem).

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:modelsModel selection, availability, switching, rate limits, and model-specific behavior
    No fields configured for Feature.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions