Skip to content

MiniCPM5 tool parser #4373

Merged
dtrawins merged 32 commits into
mainfrom
przepeck/minicpm5
Jul 17, 2026
Merged

MiniCPM5 tool parser #4373
dtrawins merged 32 commits into
mainfrom
przepeck/minicpm5

Conversation

@przepeck

@przepeck przepeck commented Jul 13, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

🛠 Summary

https://jira.devtools.intel.com/browse/CVS-187940
Adding tool parser for MiniCPM5, based on PR#4339

MiniCPM5 - int4

unary results:
image

streaming results:
image

MiniCPM5 - int8

unary results(without reasoning):
image

unary results(with reasoning):
image
image

Adding autodetection for this parser and reasoning parser.
Minja is currently unsupported for this model.

🧪 Checklist

  • Unit tests added.
  • The documentation updated.
  • Change follows security best practices.
    ``

exzile and others added 17 commits June 28, 2026 22:42
@exzile @claude
Add minicpm5 tool_parser for MiniCPM5 XML tool calls (prototype)
288ade9 
MiniCPM5 emits XML-style tool calls (<function name="..."><param
name="...">value</param></function>) which OVMS had no parser for, so
/v3/chat/completions left the XML in message.content with empty
tool_calls. Add a minicpm5 tool parser modeled on the qwen3coder XML
parser:
- new src/llm/io_processing/minicpm5/ tool parser (unary + streaming
  state machine; attribute-style tags; <think>...</think> stripping;
  schema-driven argument typing)
- register "minicpm5" in output_parser.cpp and supported parser names
- bazel target + unit tests

Relates to #4268. Prototype for maintainer coordination; not yet a PR.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@exzile @claude
minicpm5 parser: add streaming split-chunk and edge-case tests
0d45d35 
Harden the prototype's test coverage:
- streaming: char-by-char and every-fragment-size feeds must yield the
  same tool calls as a single-chunk parse (single call, two calls,
  leading content + <think> block)
- edge cases that must not crash or emit spurious calls: unterminated
  <function>, empty <param> value, stray '<' in plain prose

All 15 Minicpm5 impl/stream tests pass locally.

Relates to #4268.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@exzile
Merge branch 'main' into feature/minicpm5-tool-parser
5c6e4a7
@exzile
Merge branch 'main' into feature/minicpm5-tool-parser
e9a9d86
@exzile @claude
Address review: share parser helpers, split state machine, separate r…
869484c 
…easoning, add tests (#4268)

Responds to review feedback on the MiniCPM5 tool parser:
- Extract helpers shared with qwen3coder: trimNewline, jsonTypeOf,
  enforceStringValue and Python-bool normalization move to io_processing/utils;
  parseToolSchema/createToolsParametersTypesMap move to base_output_parser
  (which already owns the ParameterType/ToolsSchemas types). Both parsers now
  use the shared versions; qwen3coder behavior is unchanged.
- Split the long parseUntilStateChange switch into per-state handler methods.
- Stop handling <think> in the tool parser; reasoning is now the reasoning
  parser's job (reuse Qwen3ReasoningParser via reasoning_parser: qwen3, same
  <think> tags), so reasoning is preserved as reasoning_content.
- Tests: drive the public parser streaming path (name delta then arguments
  delta) and add complex-content cases (embedded quotes/backslashes, code with
  braces/newlines, JSON-looking string values, special/unicode chars).

The special-token-tag handling (requiresStreamingWithSpecialTokens) is deferred
to a follow-up; it needs the model to confirm which tags are special tokens and
is coupled to the reasoning parser's matching flag.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@exzile @claude
minicpm5: stream special tokens for tool tags + matching reasoning pa…
b77edee 
…rser (#4268)

MiniCPM5-1B emits its tool-call tags as special tokens (<function id=18,
</function>=19, <param>=20, </param>=21 in the openbmb/MiniCPM5-1B tokenizer),
which the text streamer skips by default — so the tag-scanning parser never
saw them during streaming. Override requiresStreamingWithSpecialTokens() to
return true so they are preserved.

The framework requires a paired reasoning parser to agree on that flag. Since
MiniCPM5's <think>/</think> are ordinary (non-special) tokens, the Qwen3
reasoning logic still applies verbatim; add a thin Minicpm5ReasoningParser
subclass that only flips the flag, and register "minicpm5" as a reasoning
parser. Tests updated to pair minicpm5+minicpm5 and assert the flag.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@exzile
test(minicpm5): clarify why the Qwen3-8B tokenizer is reused in tests
0ceda28 
Addresses review feedback: no MiniCPM5 tokenizer exists in the test
assets. The parser operates on decoded text (tokenizer.decode) via
plain string/XML matching, never token ids or vocab-specific special
tokens, so any tokenizer that round-trips the test strings via
encode()+decode() is safe to use here.

Signed-off-by: exzile <joeypongallo@gmail.com>
@exzile
minicpm5: implement native token-based reasoning parser
e1bea89 
Addresses review feedback: Minicpm5ReasoningParser previously inherited
Qwen3ReasoningParser's text-based parse() verbatim and only overrode
requiresStreamingWithSpecialTokens(). MiniCPM5's <think>/</think> tags
are real dedicated special tokens in its tokenizer (ids 8 and 9), not
incidental text, so parse() now locates the reasoning segment by
scanning generatedTokens for these token ids directly -- mirroring the
existing Gemma4ReasoningParser pattern -- instead of searching decoded
text for the tag substrings.

requiresStreamingWithSpecialTokens() remains true: MiniCPM5's tool-call
tags (<function>, <param>, ...) are also special tokens, and the
framework requires the paired tool/reasoning parser to agree on this
setting (same as Gemma4).

Test fix: the existing reasoning-parser test encoded '<think>...</think>'
text with the Qwen3-8B stand-in tokenizer used elsewhere in this test
file (no MiniCPM5 tokenizer test asset exists). Qwen3-8B has its own,
different ids for those strings (151667/151668), so the new token-id
based parse() would not have found them. The test now assembles
generatedTokens directly with the real ids (8/9) as delimiters and uses
the stand-in tokenizer only to encode/decode the surrounding free-form
text, which round-trips correctly regardless of whose vocab it is.

Signed-off-by: exzile <joeypongallo@gmail.com>
@exzile
Merge branch 'main' into feature/minicpm5-tool-parser
2243ea4
@exzile
minicpm5: preserve special tokens when decoding content, not just rea…
7a3e1da 
…soning

Self-review fix: parse() previously decoded the content spans (outside
the <think>...</think> boundary) with skip_special_tokens(true), copied
from Gemma4ReasoningParser's pattern. That's safe for Gemma4 because its
tool parser reads raw token ids, but MiniCPM5's tool parser is text-based
-- Minicpm5ToolParserImpl matches the literal "<function"/"<param"
substrings in decoded content. Those tags are themselves special tokens
in MiniCPM5's tokenizer (<function = id 18, <param = id 20, both
"special": true), so skip_special_tokens(true) silently erased them
from content before the tool parser ever ran, breaking tool-call
extraction whenever a reasoning block was present.

Content spans now decode with skip_special_tokens(false), matching the
framework's own default full-decode behavior. Only the extracted
reasoning text keeps skip_special_tokens(true), since it's a
human-facing string not matched against tags downstream. Also documented
the existing find-first-occurrence limitation shared with
Gemma4ReasoningParser (a stray </think> before any <think> would hide a
later well-formed pair).

Signed-off-by: exzile <joeypongallo@gmail.com>
@exzile
test(minicpm5): clarify streaming test scope, add multi-call delta co…
ff1694a 
…verage

Addresses review feedback: the Minicpm5ToolParserImplStreamTest section
(streamInFragments-based) only asserted that the aggregated final result
is fragmentation-invariant, not that the actual user-facing streaming
deltas are surfaced correctly. Retitled/documented that section to make
its scope explicit and point to Minicpm5PublicStreamTest (already added
in an earlier review round, commit 869484c) for the delta-timing
concern, which drives the real Minicpm5ToolParser::parseChunk API and
asserts on the actual emitted name/arguments deltas.

That existing coverage was single-call only, so added
EmitsDeltasForTwoSequentialCalls: verifies each of two sequential tool
calls gets exactly one name delta and one arguments delta, in order,
each carrying the correct (distinct, increasing) tool_call index -- not
just that the final parse result is correct.

Signed-off-by: exzile <joeypongallo@gmail.com>
@przepeck
Merge remote-tracking branch 'minifork/feature/minicpm5-tool-parser' …
b77dd79 
…into przepeck/minicpm5
@przepeck
save
d7a00ec
@przepeck
fix content with reasoning and without tool calls
00c099a
@przepeck
streaming tests
dc115a9
@przepeck
fixing array and object parsing
4376744
@przepeck
styles
db2abda
Copilot AI review requested due to automatic review settings July 13, 2026 13:05
Copilot AI reviewed Jul 13, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds MiniCPM5 support to OVMS LLM output parsing by introducing a MiniCPM5 XML-style tool-call parser and a paired reasoning parser, wiring both into OutputParser selection and supported-parser validation, and adding unit/streaming tests plus model-prep script support.

Changes:

  • Implemented minicpm5 tool parser + reasoning parser and registered them in the parser factory/validation lists.
  • Refactored/shared schema + JSON/string helper utilities to support attribute/tag-style tool parsers.
  • Added a comprehensive MiniCPM5 output-parser test suite and updated LLM model preparation script for the MiniCPM5 tokenizer.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
src/test/llm/output_parsers/minicpm5_output_parser_test.cpp Adds unary + streaming tests for MiniCPM5 tool calls and reasoning handling.
src/llm/io_processing/utils.hpp Exposes shared helper APIs for tool parser implementations.
src/llm/io_processing/utils.cpp Implements shared helper utilities (newline trim, JSON type, boolean normalization, quote normalization).
src/llm/io_processing/qwen3coder/qwen3coder_tool_parser.cpp Switches qwen3coder to shared helpers (reduces duplication).
src/llm/io_processing/parser_config_validation.cpp Registers minicpm5 as supported tool + reasoning parser.
src/llm/io_processing/output_parser.cpp Wires minicpm5 parser implementations into the factory.
src/llm/io_processing/minicpm5/minicpm5_tool_parser.hpp Declares MiniCPM5 tool parser + state-machine impl.
src/llm/io_processing/minicpm5/minicpm5_tool_parser.cpp Implements MiniCPM5 XML tool-call parsing and streaming deltas.
src/llm/io_processing/minicpm5/minicpm5_reasoning_parser.hpp Declares MiniCPM5 reasoning parser.
src/llm/io_processing/minicpm5/minicpm5_reasoning_parser.cpp Implements token-based reasoning extraction and streaming reasoning deltas.
src/llm/io_processing/base_output_parser.hpp Exposes shared schema→type-map helpers for tool parsers.
src/llm/io_processing/base_output_parser.cpp Implements shared schema→type-map helpers.
src/llm/BUILD Adds Bazel target for MiniCPM5 parsers and updates deps.
prepare_llm_models.sh Adds MiniCPM5 tokenizer conversion/download step.

Comment thread src/llm/io_processing/minicpm5/minicpm5_reasoning_parser.cpp Outdated
Comment thread src/llm/io_processing/minicpm5/minicpm5_reasoning_parser.cpp
Comment thread src/llm/io_processing/minicpm5/minicpm5_tool_parser.cpp
Comment thread src/test/llm/output_parsers/minicpm5_output_parser_test.cpp Outdated
Comment thread src/test/llm/output_parsers/minicpm5_output_parser_test.cpp Outdated
Comment thread src/test/llm/output_parsers/minicpm5_output_parser_test.cpp Outdated
@przepeck

przepeck commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator Author
image

Model's BFCL accuracy, without reasoning.
With reasoning it's ~ 40%, it has tendency to loop and returning arguments as object with the function name of "arguments".

przepeck added 2 commits July 14, 2026 17:34
@przepeck
copilot review
a60410e 
handling case indicated by copilot, when the whole tool call is being delivered in one chunk
@przepeck
style
88bfa43
Copilot AI reviewed Jul 15, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 12 comments.

Comment thread src/llm/io_processing/minicpm5/minicpm5_tool_parser.cpp
Comment thread src/llm/io_processing/minicpm5/minicpm5_reasoning_parser.cpp
Comment thread src/llm/io_processing/minicpm5/minicpm5_tool_parser.cpp Outdated
Comment thread src/llm/io_processing/base_output_parser.cpp Outdated
Comment thread src/llm/io_processing/minicpm5/minicpm5_tool_parser.cpp Outdated
Comment thread src/test/llm/output_parsers/minicpm5_output_parser_test.cpp Outdated
Comment thread src/test/llm/output_parsers/minicpm5_output_parser_test.cpp Outdated
Comment thread src/test/llm/output_parsers/minicpm5_output_parser_test.cpp Outdated
Comment thread prepare_llm_models.sh
Comment thread src/test/llm/output_parsers/minicpm5_output_parser_test.cpp
@dtrawins
dtrawins requested a review from Copilot July 16, 2026 12:24
Copilot AI reviewed Jul 16, 2026
View reviewed changes

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 4 comments.

Comment thread src/llm/io_processing/minicpm5/minicpm5_tool_parser.cpp
Comment thread src/llm/io_processing/utils.cpp
Comment thread src/test/llm/output_parsers/minicpm5_output_parser_test.cpp Outdated
Comment thread src/test/llm/output_parsers/minicpm5_output_parser_test.cpp Outdated
@dtrawins dtrawins added this to the 2026.3 milestone Jul 16, 2026
atobiszei
atobiszei reviewed Jul 16, 2026
View reviewed changes
Comment thread src/llm/io_processing/minicpm5/minicpm5_tool_parser.cpp
atobiszei
atobiszei reviewed Jul 17, 2026
View reviewed changes
Comment thread src/llm/io_processing/minicpm5/minicpm5_reasoning_parser.cpp Outdated
Comment thread src/llm/io_processing/minicpm5/minicpm5_reasoning_parser.cpp Outdated
Comment thread src/llm/io_processing/minicpm5/minicpm5_reasoning_parser.hpp Outdated
Comment thread src/test/llm/chat_template_end_to_end_minja_test.cpp Outdated
atobiszei
atobiszei reviewed Jul 17, 2026
View reviewed changes
Comment thread src/test/llm/output_parsers/minicpm5_output_parser_test.cpp Outdated
atobiszei
atobiszei reviewed Jul 17, 2026
View reviewed changes
Comment thread src/test/llm/output_parsers/minicpm5_output_parser_test.cpp Outdated
atobiszei
atobiszei reviewed Jul 17, 2026
View reviewed changes
Comment thread src/test/llm/output_parsers/minicpm5_output_parser_test.cpp
atobiszei
atobiszei reviewed Jul 17, 2026
View reviewed changes
Comment thread src/test/llm/output_parsers/minicpm5_output_parser_test.cpp Outdated
atobiszei
atobiszei reviewed Jul 17, 2026
View reviewed changes
Comment thread src/llm/io_processing/minicpm5/minicpm5_tool_parser.cpp Outdated
atobiszei
atobiszei reviewed Jul 17, 2026
View reviewed changes
Comment thread src/llm/io_processing/minicpm5/minicpm5_tool_parser.cpp Outdated
@dtrawins
dtrawins merged commit 9d67863 into main Jul 17, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments
@dtrawins dtrawins dtrawins approved these changes
@atobiszei atobiszei atobiszei approved these changes
@dkalinowski dkalinowski Awaiting requested review from dkalinowski

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

2026.3

Development

Successfully merging this pull request may close these issues.

5 participants

@przepeck @dtrawins @atobiszei @exzile