test(e2e): add TC-INF-02, TC-INF-03, TC-INF-09 provider inference tests#2012
Draft
TruongNguyenG wants to merge 1 commit intoNVIDIA:mainfrom
Draft
test(e2e): add TC-INF-02, TC-INF-03, TC-INF-09 provider inference tests#2012TruongNguyenG wants to merge 1 commit intoNVIDIA:mainfrom
TruongNguyenG wants to merge 1 commit intoNVIDIA:mainfrom
Conversation
Contributor
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
Add end-to-end tests for three inference providers in test-inference-routing.sh: - TC-INF-02: OpenAI provider — onboard, send prompt through sandbox proxy, verify response. Requires OPENAI_API_KEY. - TC-INF-03: Anthropic provider — same flow via Anthropic Messages API. Requires ANTHROPIC_API_KEY. - TC-INF-09: Custom OpenAI-compatible endpoint — onboard with user-provided endpoint URL and model, verify inference through sandbox proxy. Requires NEMOCLAW_ENDPOINT_URL, NEMOCLAW_COMPAT_MODEL, COMPATIBLE_API_KEY. All three tests skip gracefully when their API keys are not set. Full end-to-end verification will be enabled once the corresponding secrets are added to the repository. TC-INF-09 validated locally with inference-api.nvidia.com (Llama 3.3 70B). Signed-off-by: Truong Nguyen <tgnguyen@nvidia.com> Made-with: Cursor
TruongNguyenG
force-pushed
the
feat/e2e-inf-09-compatible-endpoint
branch
from
c192a05 to
bc596c6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add end-to-end tests for three inference providers to
test-inference-routing.sh, covering the OpenAI, Anthropic, and custom OpenAI-compatible endpoint paths.Related Issue
Closes #1906
Changes
NEMOCLAW_PROVIDER=openai, send prompt through sandbox proxy, verify responseAll three tests skip gracefully when their API keys are not set. Full end-to-end verification will be enabled once the corresponding secrets (
OPENAI_API_KEY,ANTHROPIC_API_KEY,COMPATIBLE_API_KEY) are added to the repository.TC-INF-09 validated locally with
inference-api.nvidia.com(Llama 3.3 70B).Type of Change
Verification
npx prek run --all-filespassesAI Disclosure
Signed-off-by: Truong Nguyen tgnguyen@nvidia.com