diff --git a/docs/parameters.md b/docs/parameters.md index 7651cf0fa2..b20c93fc9c 100644 --- a/docs/parameters.md +++ b/docs/parameters.md @@ -57,6 +57,7 @@ Configuration options for the server are defined only via command-line options a | `api_key_file` | `string` | Path to the text file with the API key for generative endpoints `/v3/`. The value of first line is used. If not specified, server is using environment variable API_KEY. If not set, requests will not require authorization.| | `allowed_local_media_path` | `string` | Path to the directory containing images to include in requests. If unset, local filesystem images in requests are not supported.| | `allowed_media_domains` | `string` | Comma separated list of media domains from which URLs can be used as input for LLMs. Set to \"all\" to disable this restrictions. If unset, URLs in requests are not supported." +| `verbose_response` | `NA` | When enabled, responses include an extra `__verbose` object with additional debug information. Applies for text generation models | ## Config management mode options @@ -68,10 +69,29 @@ Configuration options for the config management mode, which is used to manage co | `list_models` | `NA` | List all models paths in the model repository. | | `model_name` | `string` | Name of the model as visible in serving. If `--model_path` is not provided, path is deduced from name. | | `model_path` | `string` | Optional. Path to the model repository. If path is relative then it is prefixed with `--model_repository_path`. | -| `add_to_config` | `NA` | Directive to add new model to the config file. | +| `add_to_config` | `NA` | Directive to add new model to the config file. Accepts optional model parameters: `--batch_size`, `--shape`, `--layout`, `--mean`, `--scale`, `--color_format`, `--precision`, `--model_version_policy`, `--nireq`, `--target_device`, `--plugin_config`. | | `remove_from_config` | `NA` | Directive to remove model from the config file. | | `config_path` | `string` | Path to the configuration file. | +## Configure mode options + +Configure mode creates or updates `graph.pbtxt` for a local model without starting the server. It requires `--model_path` and `--task` parameters along with task-specific options. + +| Option | Value format | Description | +|-------------------------|--------------|-----------------------------------------------------------------------------------------------------------------------------------------------------| +| `--configure` | `NA` | Runs in configure mode to create or update `graph.pbtxt` for a local model. Does not start the server. | +| `--model_path` | `string` | Path to the local model directory where `graph.pbtxt` will be created. | +| `--model_name` | `string` | Optional. Name of the model as exposed by the server. | +| `--task` | `string` | Task type for the model (`text_generation`, `embeddings`, `rerank`, `image_generation`, `text2speech`, `speech2text`). | +| `--target_device` | `string` | Device name to be used to execute inference operations. Accepted values are: `"CPU"/"GPU"/"NPU"/"MULTI"/"HETERO"`. | + +Task-specific options (e.g., `--max_num_seqs`, `--cache_size`, `--num_streams`) are the same as documented in the [pull mode task options](#text-generation) below. + +**Example:** +```text +ovms --configure --model_path /models/my_llm --task text_generation --target_device GPU --max_num_seqs 128 --cache_size 8 +``` + ## Pull mode configuration options Shared configuration options for the pull, and pull & start mode. In the presence of ```--pull``` parameter OVMS will only pull model without serving. @@ -85,7 +105,7 @@ Shared configuration options for the pull, and pull & start mode. In the presenc | `--model_repository_path` | `string` | Directory where all required model files will be saved. | | `--model_name` | `string` | Name of the model as exposed externally by the server. | | `--target_device` | `string` | Device name to be used to execute inference operations. Accepted values are: `"CPU"/"GPU"/"MULTI"/"HETERO"` | -| `--task` | `string` | Task type the model will support (`text_generation`, `embeddings`, `rerank`, `image_generation`). | +| `--task` | `string` | Task type the model will support (`text_generation`, `embeddings`, `rerank`, `image_generation`, `text2speech`, `speech2text`). | | `--overwrite_models` | `NA` | If set, an existing model with the same name will be overwritten. If not set, the server will use existing model files if available. | | `--gguf_filename` | `string` | Filename of the wanted quantization type from Hugging Face GGUF repository. | @@ -116,10 +136,9 @@ There are also additional environment variables that may change the behavior of |-------------------------------------|---------|------------------------------------------------------------------------------------------------------------| | `GIT_OPT_SET_SERVER_CONNECT_TIMEOUT`| `int` | Timeout to attempt connections to a remote server. Default value 4000 ms. | | `GIT_OPT_SET_SERVER_TIMEOUT` | `int` | Timeout for reading from and writing to a remote server. Default value 4000 ms. | -| `GIT_OPT_SET_SSL_CERT_LOCATIONS` | `string`| Path to check for ssl certificates. | -| `GIT_OPT_SET_ENABLE_SEARCH_PATHS`| `int` | When set to 1, the pull functionality reads host-level git configuration locations like ~/.gitconfig. Default value 0. | +| `GIT_OPT_SET_SSL_CERT_LOCATIONS` | `string`| Path to check for ssl certificates. | -Task specific parameters for different tasks (text generation/image generation/embeddings/rerank) are listed below: +Task specific parameters for different tasks (text generation/image generation/embeddings/rerank/text2speech/speech2text) are listed below: ### Text generation | option | Value format | Description | @@ -137,6 +156,7 @@ Task specific parameters for different tasks (text generation/image generation/e | `--reasoning_parser` | `string` | Type of parser to use for reasoning content extraction from model output. Auto-detected from chat template if not specified. Use `none` to explicitly disable. Supported: [qwen3, gptoss, lfm2, gemma4] | | `--tool_parser` | `string` | Type of parser to use for tool calls extraction from model output. Auto-detected from chat template if not specified. Use `none` to explicitly disable. Supported: [llama3, phi4, hermes3, mistral, qwen3coder, gptoss, devstral, lfm2, gemma4] | | `--enable_tool_guided_generation` | `bool` | Enables enforcing tool schema during generation. Requires setting response parser. Default: false. | +| `--cache_interval_multiplier` | `integer` | Multiplier for the KV cache block interval. Controls the granularity of cache allocation. Default: adaptive for the model. | ### Image generation | option | Value format | Description | @@ -165,3 +185,18 @@ Task specific parameters for different tasks (text generation/image generation/e |---------------------------|--------------|--------------------------------------------------------------------------------| | `--num_streams` | `integer` | The number of parallel execution streams to use for the model. Use at least 2 on 2 socket CPU systems. Default: 1. | | `--max_allowed_chunks` | `integer` | Maximum allowed chunks. Default: 10000. | + +### Text to speech +| option | Value format | Description | +|---------------------------|--------------|--------------------------------------------------------------------------------| +| `--num_streams` | `integer` | The number of parallel execution streams to use for the model. Use at least 2 on 2 socket CPU systems. Default: 1. | +| `--model_type` | `string` | Type of the source TTS model: `speecht5` (default) or `kokoro`. | +| `--vocoder` | `string` | The vocoder model to use for text2speech. For example `microsoft/speecht5_hifigan`. | + +### Speech to text +| option | Value format | Description | +|---------------------------|--------------|--------------------------------------------------------------------------------| +| `--num_streams` | `integer` | The number of parallel execution streams to use for the model. Use at least 2 on 2 socket CPU systems. Default: 1. | + + + diff --git a/src/capi_frontend/server_settings.hpp b/src/capi_frontend/server_settings.hpp index a6968994dc..a15db60680 100644 --- a/src/capi_frontend/server_settings.hpp +++ b/src/capi_frontend/server_settings.hpp @@ -91,6 +91,7 @@ enum OvmsServerMode : int { LIST_MODELS_MODE, MODIFY_CONFIG_MODE, IN_MEMORY_GRAPH_MODE, + CONFIGURE_MODE, UNKNOWN_MODE }; diff --git a/src/cli_parser.cpp b/src/cli_parser.cpp index 9c4d77911a..629281a697 100644 --- a/src/cli_parser.cpp +++ b/src/cli_parser.cpp @@ -72,7 +72,7 @@ std::variant> CLIParser::parse(int argc, char* std::stringstream ss; try { options = std::make_unique(argv[0], "OpenVINO Model Server"); - auto configOptions = std::make_unique("ovms --model_name --add_to_config --config_path --model_repository_path \n ovms --model_path --model_name --add_to_config --config_path \n ovms --remove_from_config --config_path --model_name ", "config management commands:"); + auto configOptions = std::make_unique("ovms --add_to_config --config_path --model_name --model_repository_path \n ovms --add_to_config --config_path --model_path --model_name \n ovms --remove_from_config --config_path --model_name ", "config management commands:"); // Adding this option to parse unrecognised options in another parser options->allow_unrecognised_options(); @@ -252,11 +252,7 @@ std::variant> CLIParser::parse(int argc, char* ("extra_quantization_params", "Model quantization parameters used in optimum-cli export with conversion for text generation models", cxxopts::value(), - "EXTRA_QUANTIZATION_PARAMS") - ("vocoder", - "The vocoder model to use for text2speech. For example microsoft/speecht5_hifigan", - cxxopts::value(), - "VOCODER"); + "EXTRA_QUANTIZATION_PARAMS"); options->add_options("single model") ("model_name", @@ -312,11 +308,15 @@ std::variant> CLIParser::parse(int argc, char* cxxopts::value(), "PLUGIN_CONFIG"); - options->add_options("generative task (applies to: pull hf model, single model)") + options->add_options("generative task (applies to: pull hf model, configure, single model)") ("task", "Specifies the generative task for the local model. It should be followed by task specific parameters. Supported tasks: text_generation, embeddings, rerank, image_generation, text2speech, speech2text. It creates the pipeline graph in memory based on the provided task-specific options. If not provided, default task value is inferred from model config.", cxxopts::value(), - "TASK"); + "TASK") + ("configure", + "Create or update graph.pbtxt for the model specified by --model_path and --task. Does not start the server.", + cxxopts::value()->default_value("false"), + "CONFIGURE"); configOptions->custom_help(""); configOptions->add_options(CONFIG_MANAGEMENT_HELP_GROUP) ("list_models", @@ -350,8 +350,8 @@ std::variant> CLIParser::parse(int argc, char* result = std::make_unique(options->parse(argc, argv)); - // HF pull mode, in-memory graph from local model path, or pull-and-start - if (isHFFlow(this->result) || isInMemoryGraphMode(this->result)) { + // HF pull mode, in-memory graph from local model path, pull-and-start, or configure mode + if (isHFFlow(this->result) || isInMemoryGraphMode(this->result) || isConfigureMode(this->result)) { std::vector unmatchedOptions; GraphExportType task; std::string taskValue; @@ -366,6 +366,9 @@ std::variant> CLIParser::parse(int argc, char* if (sourceModel.has_value() && !sourceModel->empty()) { // Always infer task when pulling from HuggingFace shouldInferTask = true; + } else if (isConfigureMode(this->result) && modelPath.has_value() && !modelPath->empty()) { + // Configure mode always infers task from model when --task not provided + shouldInferTask = true; } else if (modelPath.has_value() && !modelPath->empty()) { // For local model_path, infer task if: // 1. Unmatched options (task-specific parameters) are present, OR @@ -377,60 +380,66 @@ std::variant> CLIParser::parse(int argc, char* if (shouldInferTask) { inferredTaskParameter = determineDefaultTaskParameter(modelPath, sourceModel, modelRepositoryPath); + if (!result->count("task") && !inferredTaskParameter.has_value()) { + ss << "error parsing options - Could not infer model task - specify --task value explicitly" << std::endl; + return std::make_pair(OVMS_EX_USAGE, ss.str()); + } } } - taskValue = getEffectiveTaskParameter(); - task = stringToEnum(taskValue); - switch (task) { - case TEXT_GENERATION_GRAPH: { - GraphCLIParser cliParser; - unmatchedOptions = cliParser.parse(result->unmatched()); - this->graphOptionsParser = std::move(cliParser); - break; - } - case EMBEDDINGS_GRAPH: { - EmbeddingsGraphCLIParser cliParser; - unmatchedOptions = cliParser.parse(result->unmatched()); - this->graphOptionsParser = std::move(cliParser); - break; - } - case RERANK_GRAPH: { - RerankGraphCLIParser cliParser; - unmatchedOptions = cliParser.parse(result->unmatched()); - this->graphOptionsParser = std::move(cliParser); - break; - } - case IMAGE_GENERATION_GRAPH: { - ImageGenerationGraphCLIParser cliParser; - unmatchedOptions = cliParser.parse(result->unmatched()); - this->graphOptionsParser = std::move(cliParser); - break; - } - case TEXT_TO_SPEECH_GRAPH: { - TextToSpeechGraphCLIParser cliParser; - unmatchedOptions = cliParser.parse(result->unmatched()); - this->graphOptionsParser = std::move(cliParser); - break; - } - case SPEECH_TO_TEXT_GRAPH: { - SpeechToTextGraphCLIParser cliParser; - unmatchedOptions = cliParser.parse(result->unmatched()); - this->graphOptionsParser = std::move(cliParser); - break; - } - default: { - ss << "error parsing options - --task parameter unsupported value: " + taskValue; - return std::make_pair(OVMS_EX_USAGE, ss.str()); + if (result->count("task") || inferredTaskParameter.has_value()) { + taskValue = getEffectiveTaskParameter(); + task = stringToEnum(taskValue); + switch (task) { + case TEXT_GENERATION_GRAPH: { + GraphCLIParser cliParser; + unmatchedOptions = cliParser.parse(result->unmatched()); + this->graphOptionsParser = std::move(cliParser); + break; + } + case EMBEDDINGS_GRAPH: { + EmbeddingsGraphCLIParser cliParser; + unmatchedOptions = cliParser.parse(result->unmatched()); + this->graphOptionsParser = std::move(cliParser); + break; + } + case RERANK_GRAPH: { + RerankGraphCLIParser cliParser; + unmatchedOptions = cliParser.parse(result->unmatched()); + this->graphOptionsParser = std::move(cliParser); + break; + } + case IMAGE_GENERATION_GRAPH: { + ImageGenerationGraphCLIParser cliParser; + unmatchedOptions = cliParser.parse(result->unmatched()); + this->graphOptionsParser = std::move(cliParser); + break; + } + case TEXT_TO_SPEECH_GRAPH: { + TextToSpeechGraphCLIParser cliParser; + unmatchedOptions = cliParser.parse(result->unmatched()); + this->graphOptionsParser = std::move(cliParser); + break; + } + case SPEECH_TO_TEXT_GRAPH: { + SpeechToTextGraphCLIParser cliParser; + unmatchedOptions = cliParser.parse(result->unmatched()); + this->graphOptionsParser = std::move(cliParser); + break; + } + default: { + ss << "error parsing options - --task parameter unsupported value: " + taskValue; + return std::make_pair(OVMS_EX_USAGE, ss.str()); + } } - } - if (unmatchedOptions.size()) { - ss << "task: " << enumToString(task) << " - error parsing options - unmatched arguments : "; - for (auto& argument : unmatchedOptions) { - ss << argument << ", "; + if (unmatchedOptions.size()) { + ss << "task: " << enumToString(task) << " - error parsing options - unmatched arguments : "; + for (auto& argument : unmatchedOptions) { + ss << argument << ", "; + } + ss << std::endl; + return std::make_pair(OVMS_EX_USAGE, ss.str()); } - ss << std::endl; - return std::make_pair(OVMS_EX_USAGE, ss.str()); } } else if (result->unmatched().size()){ ss << "error parsing options - unmatched arguments: "; @@ -452,6 +461,18 @@ std::variant> CLIParser::parse(int argc, char* ss << "error parsing options - --add_to_config cannot be used with --pull or --task" << std::endl; return std::make_pair(OVMS_EX_USAGE, ss.str()); } + if (result->count("configure") && !result->count("model_path")) { + ss << "error parsing options - --configure requires --model_path" << std::endl; + return std::make_pair(OVMS_EX_USAGE, ss.str()); + } + if (result->count("configure") && !result->count("task") && !inferredTaskParameter.has_value()) { + ss << "error parsing options - --configure requires --task" << std::endl; + return std::make_pair(OVMS_EX_USAGE, ss.str()); + } + if (result->count("configure") && result->count("pull")) { + ss << "error parsing options - --configure cannot be used with --pull" << std::endl; + return std::make_pair(OVMS_EX_USAGE, ss.str()); + } if (result->count("add_to_config") && result->count("list_models")) { ss << "error parsing options - --list_models cannot be used with --add_to_config" << std::endl; return std::make_pair(OVMS_EX_USAGE, ss.str()); @@ -481,8 +502,13 @@ std::variant> CLIParser::parse(int argc, char* } if (result->count("help") || result->arguments().size() == 0) { - ss << options->help({"", "multi model", "single model", "pull hf model"}) << std::endl; + ss << options->help({"", "multi model", "single model", "pull hf model", "generative task (applies to: pull hf model, configure, single model)"}) << std::endl; + ss << "configure mode (create or update graph.pbtxt for a local model):" << std::endl; + ss << " ovms --configure --model_path --task [TASK OPTIONS ...]" << std::endl; + ss << std::endl; ss << configOptions->help({CONFIG_MANAGEMENT_HELP_GROUP}) << std::endl; + // Print main options first, then task-specific graph options + std::cout << ss.str(); GraphCLIParser parser1; RerankGraphCLIParser parser2; EmbeddingsGraphCLIParser parser3; @@ -493,7 +519,9 @@ std::variant> CLIParser::parse(int argc, char* parser2.printHelp(); parser3.printHelp(); imageGenParser.printHelp(); - return std::make_pair(OVMS_EX_OK, ss.str()); + ttsParser.printHelp(); + sttParser.printHelp(); + return std::make_pair(OVMS_EX_OK, ""); } return true; @@ -641,7 +669,7 @@ void CLIParser::prepareModel(ModelsSettingsImpl& modelsSettings, HFSettingsImpl& } if (result->count("mean")) { - if (modelsSettings.layout.empty()) { + if (modelsSettings.layout.empty() && !result->count("add_to_config")) { throw std::logic_error("error parsing options - --mean parameter requires --layout to be set"); } modelsSettings.mean = result->operator[]("mean").as(); @@ -649,7 +677,7 @@ void CLIParser::prepareModel(ModelsSettingsImpl& modelsSettings, HFSettingsImpl& } if (result->count("scale")) { - if (modelsSettings.layout.empty()) { + if (modelsSettings.layout.empty() && !result->count("add_to_config")) { throw std::logic_error("error parsing options - --scale parameter requires --layout to be set"); } modelsSettings.scale = result->operator[]("scale").as(); @@ -657,7 +685,7 @@ void CLIParser::prepareModel(ModelsSettingsImpl& modelsSettings, HFSettingsImpl& } if (result->count("color_format")) { - if (modelsSettings.layout.empty()) { + if (modelsSettings.layout.empty() && !result->count("add_to_config")) { throw std::logic_error("error parsing options - --color_format parameter requires --layout to be set"); } modelsSettings.colorFormat = result->operator[]("color_format").as(); @@ -665,7 +693,7 @@ void CLIParser::prepareModel(ModelsSettingsImpl& modelsSettings, HFSettingsImpl& } if (result->count("precision")) { - if (modelsSettings.layout.empty()) { + if (modelsSettings.layout.empty() && !result->count("add_to_config")) { throw std::logic_error("error parsing options - --precision parameter requires --layout to be set"); } modelsSettings.precision = result->operator[]("precision").as(); @@ -734,14 +762,20 @@ bool CLIParser::isInMemoryGraphMode(const std::unique_ptr& return !::ovms::graphPbtxtExists(modelPath) || ::ovms::hasTaskSpecificParameters(result->unmatched()); } +bool CLIParser::isConfigureMode(const std::unique_ptr& result) { + return result->count("configure") && !result->count("pull") && !result->count("source_model"); +} + void CLIParser::prepareGraph(ServerSettingsImpl& serverSettings, HFSettingsImpl& hfSettings, const std::string& modelName) { // Always propagate source_model so validation can detect misuse if (result->count("source_model")) { hfSettings.sourceModel = result->operator[]("source_model").as(); } - // Ovms Pull models mode || pull and start models mode - if (isHFFlow(this->result) || isInMemoryGraphMode(this->result)) { - if (isInMemoryGraphMode(this->result)) { + // Ovms Pull models mode || pull and start models mode || configure mode + if (isHFFlow(this->result) || isInMemoryGraphMode(this->result) || isConfigureMode(this->result)) { + if (isConfigureMode(this->result)) { + serverSettings.serverMode = CONFIGURE_MODE; + } else if (isInMemoryGraphMode(this->result)) { serverSettings.serverMode = IN_MEMORY_GRAPH_MODE; } else if (result->count("pull")) { serverSettings.serverMode = HF_PULL_MODE; @@ -780,8 +814,6 @@ void CLIParser::prepareGraph(ServerSettingsImpl& serverSettings, HFSettingsImpl& hfSettings.exportSettings.precision = result->operator[]("weight-format").as(); if (result->count("extra_quantization_params")) hfSettings.exportSettings.extraQuantizationParams = result->operator[]("extra_quantization_params").as(); - if (result->count("vocoder")) - hfSettings.exportSettings.vocoder = result->operator[]("vocoder").as(); hfSettings.exportSettings.restWorkers = serverSettings.restWorkers; hfSettings.downloadPath = result->operator[]("model_repository_path").as(); // When --task is used with --model_path but without --pull/--source_model, @@ -793,9 +825,9 @@ void CLIParser::prepareGraph(ServerSettingsImpl& serverSettings, HFSettingsImpl& } const std::string taskValue = getEffectiveTaskParameter(); if (result->count("task")) { - SPDLOG_INFO("Task '{}' provided by user", taskValue); + SPDLOG_DEBUG("Task '{}' provided by user", taskValue); } else { - SPDLOG_INFO("Task '{}' inferred from model config", taskValue); + SPDLOG_DEBUG("Task '{}' inferred from model config", taskValue); } if (!taskValue.empty()) { hfSettings.task = stringToEnum(taskValue); diff --git a/src/cli_parser.hpp b/src/cli_parser.hpp index 3ef08ace15..ad8ca3eec7 100644 --- a/src/cli_parser.hpp +++ b/src/cli_parser.hpp @@ -55,6 +55,7 @@ class CLIParser { void prepareConfigExport(ModelsSettingsImpl& modelsSettings); bool isHFFlow(const std::unique_ptr& result); bool isInMemoryGraphMode(const std::unique_ptr& result); + bool isConfigureMode(const std::unique_ptr& result); }; } // namespace ovms diff --git a/src/config.cpp b/src/config.cpp index 9d964557a9..cd5ac0cf7a 100644 --- a/src/config.cpp +++ b/src/config.cpp @@ -137,8 +137,13 @@ bool Config::check_hostname_or_ip(const std::string& input) { } } -bool Config::validateUserSettingsInConfigAddRemoveModel(const ModelsSettingsImpl& modelsSettings) { - static const std::vector allowedUserSettings = {"model_name", "model_path", "config_path"}; +bool Config::validateUserSettingsInConfigAddRemoveModel(const ModelsSettingsImpl& modelsSettings, ConfigExportType exportType) { + static const std::vector allowedForRemove = {"model_name", "config_path"}; + static const std::vector allowedForAdd = {"model_name", "model_path", "config_path", + "batch_size", "shape", "layout", "mean", "scale", "color_format", "precision", + "model_version_policy", "nireq", "target_device", "plugin_config"}; + + const auto& allowedUserSettings = (exportType == ENABLE_MODEL) ? allowedForAdd : allowedForRemove; std::vector usedButDisallowedUserSettings; for (const std::string& userSetting : modelsSettings.userSetSingleModelArguments) { bool isAllowed = false; @@ -156,7 +161,11 @@ bool Config::validateUserSettingsInConfigAddRemoveModel(const ModelsSettingsImpl for (const std::string& userSetting : usedButDisallowedUserSettings) { arguments += userSetting + ", "; } - std::cerr << "Adding or removing models from the configuration file, allows passing only model_name and model_path parameters. Invalid parameters passed: " << arguments << std::endl; + if (exportType == ENABLE_MODEL) { + std::cerr << "Adding models to the configuration file does not support parameters: " << arguments << std::endl; + } else { + std::cerr << "Removing models from the configuration file allows passing only model_name parameter. Invalid parameters passed: " << arguments << std::endl; + } return false; } @@ -169,9 +178,9 @@ bool Config::validate() { std::cerr << "--source_model should be used combined with --task" << std::endl; return false; } - if (this->serverSettings.serverMode == HF_PULL_MODE || this->serverSettings.serverMode == HF_PULL_AND_START_MODE || this->serverSettings.serverMode == IN_MEMORY_GRAPH_MODE) { + if (this->serverSettings.serverMode == HF_PULL_MODE || this->serverSettings.serverMode == HF_PULL_AND_START_MODE || this->serverSettings.serverMode == IN_MEMORY_GRAPH_MODE || this->serverSettings.serverMode == CONFIGURE_MODE) { // When --task is used with --model_path (no HF pulling), sourceModel and downloadPath are not required - bool taskWithModelPath = this->serverSettings.serverMode == IN_MEMORY_GRAPH_MODE && !this->modelsSettings.modelPath.empty(); + bool taskWithModelPath = (this->serverSettings.serverMode == IN_MEMORY_GRAPH_MODE || this->serverSettings.serverMode == CONFIGURE_MODE) && !this->modelsSettings.modelPath.empty(); if (!taskWithModelPath) { if (!serverSettings.hfSettings.sourceModel.size()) { std::cerr << "source_model parameter is required for pull mode"; @@ -248,7 +257,7 @@ bool Config::validate() { } } // No more validation needed - if (this->serverSettings.serverMode == HF_PULL_MODE) { + if (this->serverSettings.serverMode == HF_PULL_MODE || this->serverSettings.serverMode == CONFIGURE_MODE) { return true; } } @@ -316,7 +325,7 @@ bool Config::validate() { return false; } - if (!Config::validateUserSettingsInConfigAddRemoveModel(this->modelsSettings)) + if (!Config::validateUserSettingsInConfigAddRemoveModel(this->modelsSettings, this->serverSettings.exportConfigType)) return false; } diff --git a/src/config.hpp b/src/config.hpp index 881cccf7f3..d710bc4e9a 100644 --- a/src/config.hpp +++ b/src/config.hpp @@ -78,7 +78,7 @@ class Config { * * @return bool */ - static bool validateUserSettingsInConfigAddRemoveModel(const ModelsSettingsImpl& modelsSettings); + static bool validateUserSettingsInConfigAddRemoveModel(const ModelsSettingsImpl& modelsSettings, ConfigExportType exportType); /** * @brief Validate passed arguments * diff --git a/src/config_export_module/BUILD b/src/config_export_module/BUILD index cbad7badab..77f78cd7da 100644 --- a/src/config_export_module/BUILD +++ b/src/config_export_module/BUILD @@ -27,6 +27,7 @@ ovms_cc_library( "//src/filesystem:libovmslocalfilesystem", "//src:libovmslogging", "//src:libovmsschema", + "//src/utils:rapidjson_utils", ], visibility = ["//visibility:public",], ) diff --git a/src/config_export_module/config_export.cpp b/src/config_export_module/config_export.cpp index eed6eba16a..f2f5ecc39c 100644 --- a/src/config_export_module/config_export.cpp +++ b/src/config_export_module/config_export.cpp @@ -29,9 +29,35 @@ #include "src/logging.hpp" #include "src/schema.hpp" #include "src/status.hpp" +#include "src/utils/rapidjson_utils.hpp" namespace ovms { +static void addOptionalModelFields(rapidjson::Value& configObj, const ModelsSettingsImpl& modelSettings, rapidjson::Document::AllocatorType& alloc) { + if (!modelSettings.batchSize.empty()) + configObj.AddMember("batch_size", rapidjson::Value(modelSettings.batchSize.c_str(), alloc), alloc); + if (!modelSettings.shape.empty()) + addJsonOrStringMember(configObj, "shape", modelSettings.shape, alloc); + if (!modelSettings.layout.empty()) + addJsonOrStringMember(configObj, "layout", modelSettings.layout, alloc); + if (modelSettings.mean.has_value()) + configObj.AddMember("mean", rapidjson::Value(modelSettings.mean.value().c_str(), alloc), alloc); + if (modelSettings.scale.has_value()) + configObj.AddMember("scale", rapidjson::Value(modelSettings.scale.value().c_str(), alloc), alloc); + if (modelSettings.colorFormat.has_value()) + configObj.AddMember("color_format", rapidjson::Value(modelSettings.colorFormat.value().c_str(), alloc), alloc); + if (modelSettings.precision.has_value()) + configObj.AddMember("precision", rapidjson::Value(modelSettings.precision.value().c_str(), alloc), alloc); + if (!modelSettings.modelVersionPolicy.empty()) + addJsonOrStringMember(configObj, "model_version_policy", modelSettings.modelVersionPolicy, alloc); + if (modelSettings.nireq != 0) + configObj.AddMember("nireq", modelSettings.nireq, alloc); + if (!modelSettings.targetDevice.empty()) + configObj.AddMember("target_device", rapidjson::Value(modelSettings.targetDevice.c_str(), alloc), alloc); + if (!modelSettings.pluginConfig.empty()) + addJsonOrStringMember(configObj, "plugin_config", modelSettings.pluginConfig, alloc); +} + Status loadJsonConfig(const std::string& jsonFilename, rapidjson::Document& configJson) { std::string md5; Status status = parseConfig(jsonFilename, configJson, md5); @@ -49,34 +75,27 @@ Status loadJsonConfig(const std::string& jsonFilename, rapidjson::Document& conf } Status createModelConfig(const std::string& fullPath, const ModelsSettingsImpl& modelSettings) { - std::ostringstream oss; - - auto escapeBackslashes = [](const std::string& path) -> std::string { - std::string result; - result.reserve(path.size()); - for (char c : path) { - if (c == '\\') { - result += "\\\\"; - } else { - result += c; - } - } - return result; - }; - - // clang-format off - oss << R"({ - "model_config_list": [ - { - "config": { - "name": ")" << modelSettings.modelName << R"(", - "base_path": ")" << escapeBackslashes(modelSettings.modelPath) << R"(" - } - } - ] -})"; - // clang-format on - return FileSystem::createFileOverwrite(fullPath, oss.str()); + rapidjson::Document configJson; + configJson.SetObject(); + auto& alloc = configJson.GetAllocator(); + + rapidjson::Value modelConfigList(rapidjson::kArrayType); + rapidjson::Value configItem(rapidjson::kObjectType); + rapidjson::Value configObj(rapidjson::kObjectType); + + configObj.AddMember("name", rapidjson::Value(modelSettings.modelName.c_str(), alloc), alloc); + configObj.AddMember("base_path", rapidjson::Value(modelSettings.modelPath.c_str(), alloc), alloc); + addOptionalModelFields(configObj, modelSettings, alloc); + + configItem.AddMember("config", configObj, alloc); + modelConfigList.PushBack(configItem, alloc); + configJson.AddMember("model_config_list", modelConfigList, alloc); + + rapidjson::StringBuffer buffer; + rapidjson::PrettyWriter writer(buffer); + configJson.Accept(writer); + + return FileSystem::createFileOverwrite(fullPath, buffer.GetString()); } Status removeModelFromConfig(const std::string& fullPath, const ModelsSettingsImpl& modelSettings) { @@ -157,6 +176,7 @@ Status updateConfigAddModel(const std::string& fullPath, const ModelsSettingsImp rapidjson::Value path; path.SetString(modelSettings.modelPath.c_str(), alloc); newConfig.AddMember("base_path", path, alloc); + addOptionalModelFields(newConfig, modelSettings, alloc); rapidjson::Value newConfigItem; newConfigItem.SetObject(); diff --git a/src/default_task.cpp b/src/default_task.cpp index 79cef6509a..b00b0d974f 100644 --- a/src/default_task.cpp +++ b/src/default_task.cpp @@ -17,7 +17,6 @@ #include #include -#include #include #include #include @@ -69,14 +68,12 @@ std::optional determineDefaultTaskParameter(const std::optionalempty()) { - std::cout << "Could not infer model task: neither model_path nor source_model provided" << std::endl; return std::nullopt; } @@ -87,7 +84,6 @@ std::optional determineDefaultTaskParameter(const std::optional determineDefaultTaskParameter(const std::optionaloptions = std::make_unique("ovms --pull [PULL OPTIONS ... ]", "-pull --task embeddings graph options"); + this->options = std::make_unique("ovms --pull --task embeddings [OPTIONS...]\n ovms --configure --model_path --task embeddings [OPTIONS...]", "--task embeddings options"); options->allow_unrecognised_options(); // clang-format off @@ -89,7 +89,7 @@ void EmbeddingsGraphCLIParser::prepare(OvmsServerMode serverMode, HFSettingsImpl } if (nullptr == result) { // Pull with default arguments - no arguments from user - if (serverMode != HF_PULL_MODE && serverMode != HF_PULL_AND_START_MODE) { + if (serverMode != HF_PULL_MODE && serverMode != HF_PULL_AND_START_MODE && serverMode != CONFIGURE_MODE) { throw std::logic_error("Tried to prepare server and model settings without graph parse result"); } } else { diff --git a/src/graph_export/graph_cli_parser.cpp b/src/graph_export/graph_cli_parser.cpp index 2a8e281c94..dd838fe265 100644 --- a/src/graph_export/graph_cli_parser.cpp +++ b/src/graph_export/graph_cli_parser.cpp @@ -36,7 +36,7 @@ TextGenGraphSettingsImpl& GraphCLIParser::defaultGraphSettings() { } void GraphCLIParser::createOptions() { - this->options = std::make_unique("ovms --pull [PULL OPTIONS ... ]", "--pull --task text_generation graph options"); + this->options = std::make_unique("ovms --pull --task text_generation [OPTIONS...]\n ovms --configure --model_path --task text_generation [OPTIONS...]", "--task text_generation options"); options->allow_unrecognised_options(); // clang-format off @@ -134,7 +134,7 @@ void GraphCLIParser::prepare(OvmsServerMode serverMode, HFSettingsImpl& hfSettin if (nullptr == result) { // Pull with default arguments - no arguments from user - if (serverMode != HF_PULL_MODE && serverMode != HF_PULL_AND_START_MODE) { + if (serverMode != HF_PULL_MODE && serverMode != HF_PULL_AND_START_MODE && serverMode != CONFIGURE_MODE) { throw std::logic_error("Tried to prepare server and model settings without graph parse result"); } } else { diff --git a/src/graph_export/image_generation_graph_cli_parser.cpp b/src/graph_export/image_generation_graph_cli_parser.cpp index 4f32140632..5b5177f432 100644 --- a/src/graph_export/image_generation_graph_cli_parser.cpp +++ b/src/graph_export/image_generation_graph_cli_parser.cpp @@ -76,7 +76,7 @@ ImageGenerationGraphSettingsImpl& ImageGenerationGraphCLIParser::defaultGraphSet } void ImageGenerationGraphCLIParser::createOptions() { - this->options = std::make_unique("ovms --pull [PULL OPTIONS ... ]", "--pull --task image generation/edit/inpainting graph options"); + this->options = std::make_unique("ovms --pull --task image_generation [OPTIONS...]\n ovms --configure --model_path --task image_generation [OPTIONS...]", "--task image_generation options"); options->allow_unrecognised_options(); // clang-format off @@ -150,7 +150,7 @@ void ImageGenerationGraphCLIParser::prepare(ServerSettingsImpl& serverSettings, } if (nullptr == result) { // Pull with default arguments - no arguments from user - if (serverSettings.serverMode != HF_PULL_MODE && serverSettings.serverMode != HF_PULL_AND_START_MODE) { + if (serverSettings.serverMode != HF_PULL_MODE && serverSettings.serverMode != HF_PULL_AND_START_MODE && serverSettings.serverMode != CONFIGURE_MODE) { throw std::logic_error("Tried to prepare server and model settings without graph parse result"); } } else { diff --git a/src/graph_export/rerank_graph_cli_parser.cpp b/src/graph_export/rerank_graph_cli_parser.cpp index 80f1561a4a..ea79a31e72 100644 --- a/src/graph_export/rerank_graph_cli_parser.cpp +++ b/src/graph_export/rerank_graph_cli_parser.cpp @@ -35,7 +35,7 @@ RerankGraphSettingsImpl& RerankGraphCLIParser::defaultGraphSettings() { } void RerankGraphCLIParser::createOptions() { - this->options = std::make_unique("ovms --pull [PULL OPTIONS ... ]", "-pull --task rerank graph options"); + this->options = std::make_unique("ovms --pull --task rerank [OPTIONS...]\n ovms --configure --model_path --task rerank [OPTIONS...]", "--task rerank options"); options->allow_unrecognised_options(); // clang-format off @@ -82,7 +82,7 @@ void RerankGraphCLIParser::prepare(OvmsServerMode serverMode, HFSettingsImpl& hf } if (nullptr == result) { // Pull with default arguments - no arguments from user - if (serverMode != HF_PULL_MODE && serverMode != HF_PULL_AND_START_MODE) { + if (serverMode != HF_PULL_MODE && serverMode != HF_PULL_AND_START_MODE && serverMode != CONFIGURE_MODE) { throw std::logic_error("Tried to prepare server and model settings without graph parse result"); } } else { diff --git a/src/graph_export/s2t_graph_cli_parser.cpp b/src/graph_export/s2t_graph_cli_parser.cpp index c516581468..6ae7835f91 100644 --- a/src/graph_export/s2t_graph_cli_parser.cpp +++ b/src/graph_export/s2t_graph_cli_parser.cpp @@ -35,7 +35,7 @@ SpeechToTextGraphSettingsImpl& SpeechToTextGraphCLIParser::defaultGraphSettings( } void SpeechToTextGraphCLIParser::createOptions() { - this->options = std::make_unique("ovms --pull [PULL OPTIONS ... ]", "-pull --task speech2text graph options"); + this->options = std::make_unique("ovms --pull --task speech2text [OPTIONS...]\n ovms --configure --model_path --task speech2text [OPTIONS...]", "--task speech2text options"); options->allow_unrecognised_options(); // clang-format off @@ -77,7 +77,7 @@ void SpeechToTextGraphCLIParser::prepare(OvmsServerMode serverMode, HFSettingsIm } if (nullptr == result) { // Pull with default arguments - no arguments from user - if (serverMode != HF_PULL_MODE && serverMode != HF_PULL_AND_START_MODE) { + if (serverMode != HF_PULL_MODE && serverMode != HF_PULL_AND_START_MODE && serverMode != CONFIGURE_MODE) { throw std::logic_error("Tried to prepare server and model settings without graph parse result"); } } else { diff --git a/src/graph_export/t2s_graph_cli_parser.cpp b/src/graph_export/t2s_graph_cli_parser.cpp index 61ac98336d..3a42e2c9b8 100644 --- a/src/graph_export/t2s_graph_cli_parser.cpp +++ b/src/graph_export/t2s_graph_cli_parser.cpp @@ -35,7 +35,7 @@ TextToSpeechGraphSettingsImpl& TextToSpeechGraphCLIParser::defaultGraphSettings( } void TextToSpeechGraphCLIParser::createOptions() { - this->options = std::make_unique("ovms --pull [PULL OPTIONS ... ]", "-pull --task text2speech graph options"); + this->options = std::make_unique("ovms --pull --task text2speech [OPTIONS...]\n ovms --configure --model_path --task text2speech [OPTIONS...]", "--task text2speech options"); options->allow_unrecognised_options(); // clang-format off @@ -47,7 +47,11 @@ void TextToSpeechGraphCLIParser::createOptions() { ("model_type", "Type of the source TTS model: speecht5 (default) or kokoro.", cxxopts::value()->default_value("speecht5"), - "MODEL_TYPE"); + "MODEL_TYPE") + ("vocoder", + "The vocoder model to use for text2speech. For example microsoft/speecht5_hifigan, used only with export via optimum-cli.", + cxxopts::value(), + "VOCODER"); // clang-format on } @@ -82,7 +86,7 @@ void TextToSpeechGraphCLIParser::prepare(OvmsServerMode serverMode, HFSettingsIm } if (nullptr == result) { // Pull with default arguments - no arguments from user - if (serverMode != HF_PULL_MODE && serverMode != HF_PULL_AND_START_MODE) { + if (serverMode != HF_PULL_MODE && serverMode != HF_PULL_AND_START_MODE && serverMode != CONFIGURE_MODE) { throw std::logic_error("Tried to prepare server and model settings without graph parse result"); } } else { @@ -92,6 +96,8 @@ void TextToSpeechGraphCLIParser::prepare(OvmsServerMode serverMode, HFSettingsIm throw std::invalid_argument("--model_type must be one of: speecht5, kokoro"); } hfSettings.exportSettings.modelType = modelType; + if (result->count("vocoder")) + hfSettings.exportSettings.vocoder = result->operator[]("vocoder").as(); } hfSettings.graphSettings = std::move(textToSpeechGraphSettings); } diff --git a/src/server.cpp b/src/server.cpp index 49419f9239..f0a27a770b 100644 --- a/src/server.cpp +++ b/src/server.cpp @@ -125,6 +125,10 @@ static void logConfig(const Config& config) { SPDLOG_DEBUG("model_repository_path: {}", config.getServerSettings().hfSettings.downloadPath); return; } + if (config.getServerSettings().serverMode == CONFIGURE_MODE) { + SPDLOG_DEBUG("model_path: {}", config.modelPath()); + return; + } if (config.configPath().empty()) { SPDLOG_DEBUG("model_path: {}", config.modelPath()); SPDLOG_DEBUG("model_name: {}", config.modelName()); @@ -406,6 +410,19 @@ Status Server::startModules(ovms::Config& config) { status = hfModule->clone(); return status; } + if (config.getServerSettings().serverMode == CONFIGURE_MODE) { + GraphExport graphExporter; + HFSettingsImpl hfSettings = config.getServerSettings().hfSettings; + std::string modelPath = config.modelPath(); + hfSettings.exportSettings.modelPath = "."; + status = graphExporter.createServableConfig(modelPath, hfSettings, true); + if (!status.ok()) { + SPDLOG_ERROR("Failed to create graph config: {}", status.string()); + return status; + } + std::cout << "Graph: graph.pbtxt created in: " << modelPath << std::endl; + return status; + } #if (PYTHON_DISABLE == 0) if (config.getServerSettings().withPython) { @@ -445,7 +462,8 @@ Status Server::startModules(ovms::Config& config) { if (config.getServerSettings().serverMode == IN_MEMORY_GRAPH_MODE) { // --task with --model_path: create graph in memory without HF download GraphExport graphExporter; - const auto& hfSettings = config.getServerSettings().hfSettings; + HFSettingsImpl hfSettings = config.getServerSettings().hfSettings; + hfSettings.exportSettings.modelPath = "."; status = graphExporter.createServableConfig(config.modelPath(), hfSettings, false); if (!status.ok()) { SPDLOG_ERROR("Failed to create in-memory graph config: {}", status.string()); diff --git a/src/test/config_export_full_test.cpp b/src/test/config_export_full_test.cpp index 3fb363db51..8c857bbdca 100644 --- a/src/test/config_export_full_test.cpp +++ b/src/test/config_export_full_test.cpp @@ -34,7 +34,7 @@ namespace { const std::string expectedConfigContents = R"({ "model_config_list": [ - { + { "config": { "name": "model1", "base_path": "/model1/Path" @@ -133,3 +133,73 @@ TEST_F(ConfigCreationFullTest, positiveEndToEndEnableDisable) { configContents = GetFileContents(this->modelsSettings.configPath); ASSERT_EQ(expectedEmptyConfigContents, configContents) << configContents; } + +TEST_F(ConfigCreationFullTest, positiveConfigureGraphForLLMModel) { + ovms::Server& server = ovms::Server::instance(); + server.setShutdownRequest(0); + + std::string modelPath = this->directoryPath; + char* argv[] = { + (char*)"ovms", + (char*)"--configure", + (char*)"--model_path", + (char*)modelPath.c_str(), + (char*)"--task", + (char*)"text_generation", + (char*)"--target_device", + (char*)"CPU", + (char*)"--cache_size", + (char*)"1", + }; + int argc = 10; + + ASSERT_EQ(EXIT_SUCCESS, server.start(argc, argv)); + + std::string graphPath = ovms::FileSystem::appendSlash(modelPath) + "graph.pbtxt"; + std::string graphContents = GetFileContents(graphPath); + ASSERT_FALSE(graphContents.empty()) << "graph.pbtxt should be created"; + EXPECT_NE(std::string::npos, graphContents.find("models_path: \".\"")) << graphContents; + EXPECT_NE(std::string::npos, graphContents.find("device: \"CPU\"")) << graphContents; + EXPECT_NE(std::string::npos, graphContents.find("cache_size: 1,")) << graphContents; +} + +TEST_F(ConfigCreationFullTest, positiveAddToConfigWithBatchSize) { + ovms::Server& server = ovms::Server::instance(); + std::unique_ptr t; + server.setShutdownRequest(0); + + std::string dummyModelPath = getGenericFullPathForSrcTest("/ovms/src/test/dummy"); + std::string modelName = "dummy"; + std::string batchSize = "10"; + char* argv[] = { + (char*)"ovms", + (char*)"--add_to_config", + (char*)"--config_path", + (char*)this->modelsSettings.configPath.c_str(), + (char*)"--model_name", + (char*)modelName.c_str(), + (char*)"--model_path", + (char*)dummyModelPath.c_str(), + (char*)"--batch_size", + (char*)batchSize.c_str(), + }; + int argc = 10; + + t.reset(new std::thread([&argc, &argv, &server]() { + ASSERT_EQ(EXIT_SUCCESS, server.start(argc, argv)); + })); + + auto start = std::chrono::high_resolution_clock::now(); + while ((server.getModuleState(ovms::SERVABLES_CONFIG_MANAGER_MODULE_NAME) != ovms::ModuleState::NOT_INITIALIZED) && + (std::chrono::duration_cast(std::chrono::high_resolution_clock::now() - start).count() < 3)) { + } + + ASSERT_EQ(server.getModuleState(ovms::SERVABLES_CONFIG_MANAGER_MODULE_NAME), ovms::ModuleState::NOT_INITIALIZED); + server.setShutdownRequest(1); + t->join(); + + std::string configContents = GetFileContents(this->modelsSettings.configPath); + EXPECT_NE(std::string::npos, configContents.find("\"name\": \"dummy\"")) << configContents; + EXPECT_NE(std::string::npos, configContents.find("\"base_path\": \"" + dummyModelPath + "\"")) << configContents; + EXPECT_NE(std::string::npos, configContents.find("\"batch_size\": \"10\"")) << configContents; +} diff --git a/src/test/config_export_test.cpp b/src/test/config_export_test.cpp index 20d6af46da..fcc72371b5 100644 --- a/src/test/config_export_test.cpp +++ b/src/test/config_export_test.cpp @@ -31,7 +31,7 @@ const std::string expectedConfigContents = R"({ "model_config_list": [ - { + { "config": { "name": "model1", "base_path": "/model1/Path" @@ -43,7 +43,7 @@ const std::string expectedConfigContents = R"({ const std::string expectedConfigContentsWindows = R"({ "model_config_list": [ - { + { "config": { "name": "model1", "base_path": "model1\\Path" diff --git a/src/test/ovmsconfig_test.cpp b/src/test/ovmsconfig_test.cpp index 86b520aeec..656743ef28 100644 --- a/src/test/ovmsconfig_test.cpp +++ b/src/test/ovmsconfig_test.cpp @@ -802,10 +802,12 @@ TEST_F(OvmsConfigDeathTest, modifyModelConfigEnableWithBadAdditionalParameters) "/config/path", "--target_device", "GPU", + "--invalid_param", + "value", "--model_path", "/model/path"}; - int arg_count = 10; - EXPECT_EXIT(ovms::Config::instance().parse(arg_count, n_argv), ::testing::ExitedWithCode(OVMS_EX_USAGE), "Adding or removing models from the configuration file, allows passing only model_name and model_path parameters. Invalid parameters passed: target_device,"); + int arg_count = 12; + EXPECT_EXIT(ovms::Config::instance().parse(arg_count, n_argv), ::testing::ExitedWithCode(OVMS_EX_USAGE), "error parsing options - unmatched arguments: --invalid_param, value,"); } TEST_F(OvmsConfigDeathTest, modifyModelConfigDisableMissingModelName) { @@ -3150,4 +3152,38 @@ TEST(OvmsGraphCliParserTest, noneParserNamesAreAccepted) { EXPECT_EQ(graphSettings.reasoningParser.value(), "none"); } +TEST_F(OvmsInferredTaskTest, positiveConfigureModeInfersTaskFromModel) { + const std::string modelPath = resolveTestModelPath("llama"); + const std::filesystem::path configJson = std::filesystem::path(modelPath) / "config.json"; + if (!std::filesystem::exists(configJson)) { + FAIL() << "Test prerequisite missing: " << configJson.string(); + } + char* n_argv[] = { + (char*)"ovms", + (char*)"--configure", + (char*)"--model_path", + (char*)modelPath.c_str(), + (char*)"--cache_size", + (char*)"3", + }; + int arg_count = 6; + ConstructorEnabledConfig config; + config.parse(arg_count, n_argv); + ASSERT_EQ(config.getServerSettings().hfSettings.task, ovms::TEXT_GENERATION_GRAPH); + ASSERT_EQ(config.getServerSettings().serverMode, ovms::CONFIGURE_MODE); +} + +TEST_F(OvmsConfigDeathTest, negativeConfigureModeRequiresTaskWhenCannotInfer) { + char* n_argv[] = { + (char*)"ovms", + (char*)"--configure", + (char*)"--model_path", + (char*)"/non/existing/model/path", + }; + int arg_count = 4; + EXPECT_EXIT(ovms::Config::instance().parse(arg_count, n_argv), + ::testing::ExitedWithCode(OVMS_EX_USAGE), + "Could not infer model task"); +} + #pragma GCC diagnostic pop diff --git a/src/utils/rapidjson_utils.cpp b/src/utils/rapidjson_utils.cpp index e428cc6b47..bec15b650d 100644 --- a/src/utils/rapidjson_utils.cpp +++ b/src/utils/rapidjson_utils.cpp @@ -37,6 +37,16 @@ std::string documentToString(const rapidjson::Document& doc) { return buffer.GetString(); } +void addJsonOrStringMember(rapidjson::Value& obj, const char* key, const std::string& value, rapidjson::Document::AllocatorType& alloc) { + rapidjson::Document parsed(&alloc); + if (!parsed.Parse(value.c_str()).HasParseError() && parsed.IsObject()) { + rapidjson::Value jsonValue(parsed, alloc); + obj.AddMember(rapidjson::Value(key, alloc), jsonValue, alloc); + } else { + obj.AddMember(rapidjson::Value(key, alloc), rapidjson::Value(value.c_str(), alloc), alloc); + } +} + // Lightweight SAX handler that only tracks nesting depth. // No DOM allocation — all SAX events are accepted and discarded. struct DepthOnlyHandler : public rapidjson::BaseReaderHandler, DepthOnlyHandler> { diff --git a/src/utils/rapidjson_utils.hpp b/src/utils/rapidjson_utils.hpp index f7c70db217..18db649bef 100644 --- a/src/utils/rapidjson_utils.hpp +++ b/src/utils/rapidjson_utils.hpp @@ -22,6 +22,7 @@ namespace ovms { std::string documentToString(const rapidjson::Document& doc); +void addJsonOrStringMember(rapidjson::Value& obj, const char* key, const std::string& value, rapidjson::Document::AllocatorType& alloc); // Default maximum nesting depth allowed for incoming JSON request bodies. inline constexpr std::size_t DEFAULT_MAX_JSON_NESTING_DEPTH = 100;