I'm getting this issue log : The maximum tokens you requested exceeds the model limit of 4096. Try again with a maximum tokens value that is lower than 4096.

[dmazevedo711]

Before submitting your bug report

• I believe this is a bug. I'll try to join the Continue Discord for questions
• I'm not able to find an open issue that reports the same bug
• I've seen the troubleshooting guide on the Continue Docs

Relevant environment info

- OS:
- Continue version:
- IDE version:
- Model:
- config.json:

Description

No response

To reproduce

No response

Log output

No response

[dbillon]

Same behavior here : MacOS 15.2 / Version 0.8.65 / Visual Studio Code 1.96.2 / Model anthropic.claude-3-5-sonnet-20240620-v1:0 via AWS Bedrock

[adam-r-hope]

I also have this error using any AWS Bedrock model using MacOS 15.2 / Continue: 0.8.65 and 0.9.246 / VS Code 1.96.2

[dbillon]

Same behavior here : MacOS 15.2 / Version 0.8.65 / Visual Studio Code 1.96.2 / Model anthropic.claude-3-5-sonnet-20240620-v1:0 via AWS Bedrock

Same behavior today with V0.9.246 and V0.8.66

[wei-m-teh]

I am getting the same error. with v0.8.66 - 2024-12-20.

[cosinusalpha]

I got the same error with v0.8.66 and 0.9.248. The last working version is 0.8.61, it starts failing with 0.8.63 .

[tomasz-stefaniak]

@sestinj is it possible there is a bug in how we calculate tokens?

[sestinj]

This isn't related to token counting, it's related to completionOptions.maxTokens. Can anyone experiencing share a config.json so we can reproduce?

[dbillon]

Hi @sestinj / @tomasz-stefaniak ,

You willl find below a minimalist configuration that reproduces this issue with Visual Studio Code 1.96.2 / Continue 0.8.66 / MacOS 15.2

I'm getting the error message with a simple "hello" in the chat box.

Then I switch back to a previous Continue version that was working :
code --install-extension continue.continue@0.8.59

Everything's back to normal and I get an answer from Claude with the, same "hello" message in the chat box.

BTW, I've tried to set "contextLength": 180000 in the Bedrock provider but no luck ... I get the same error.

Damien.

{
  "models": [
    {
      "title": "Bedrock: Claude 3.5 Sonnet",
      "provider": "bedrock",
      "model": "anthropic.claude-3-5-sonnet-20240620-v1:0",
      "region": "eu-central-1",
      "profile": "bedrock"
    }
  ],
  "tabAutocompleteModel": {
    "title": "Codestral",
    "provider": "mistral",
    "model": "codestral-latest",
    "apiKey": ""
  },
  "customCommands": [
    {
      "name": "test",
      "prompt": "{{{ input }}}\n\nWrite a comprehensive set of unit tests for the selected code. It should setup, run tests that check for correctness including important edge cases, and teardown. Ensure that the tests are complete and sophisticated. Give the tests just as chat output, don't edit any file.",
      "description": "Write unit tests for highlighted code"
    }
  ],
  "contextProviders": [
    {
      "name": "code",
      "params": {}
    },
    {
      "name": "docs",
      "params": {}
    },
    {
      "name": "diff",
      "params": {}
    },
    {
      "name": "terminal",
      "params": {}
    },
    {
      "name": "problems",
      "params": {}
    },
    {
      "name": "folder",
      "params": {}
    },
    {
      "name": "codebase",
      "params": {}
    }
  ],
  "slashCommands": [
    {
      "name": "share",
      "description": "Export the current chat session to markdown"
    },
    {
      "name": "cmd",
      "description": "Generate a shell command"
    },
    {
      "name": "commit",
      "description": "Generate a git commit message"
    }
  ]
}

[cosinusalpha]

This is my config.json. With this config and version >= 0.8.63 I get the error.

{
  "models": [
    {
      "title": "Claude 3.5 Sonnet",
      "provider": "bedrock",
      "model": "anthropic.claude-3-5-sonnet-20240620-v1:0",
      "region": "eu-central-1",
      "profile": "bedrock"
    }
  ],
  "docs": [],
  "embeddingsProvider": {
    "title": "Embeddings Model",
    "provider": "bedrock",
    "model": "amazon.titan-embed-text-v2:0",
    "region": "eu-central-1"
  }
}