FEAT AzureContentFilterScorer: Switch to async client and accept async auth providers#1467
Open
adrian-gavrila wants to merge 2 commits intoAzure:mainfrom
Open
FEAT AzureContentFilterScorer: Switch to async client and accept async auth providers#1467adrian-gavrila wants to merge 2 commits intoAzure:mainfrom
adrian-gavrila wants to merge 2 commits intoAzure:mainfrom
Conversation
|
@adrian-gavrila please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
Contributor License AgreementContribution License AgreementThis Contribution License Agreement (“Agreement”) is agreed to by the party signing below (“You”),
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Switches
AzureContentFilterScorerfrom the synchronousContentSafetyClientto the async version (azure.ai.contentsafety.aio), unblocking the event loop and accepting both sync and async token providers. This brings the scorer in line with PyRIT's async-first architecture and mirrors the pattern already used byOpenAIChatTarget.Problem: The scorer used the synchronous
ContentSafetyClient, blocking the event loop on everyanalyze_text()/analyze_image()call. It also explicitly rejected async token providers with aValueError, forcing sync-only auth even though the rest of PyRIT works with async auth.Changes (6 files, +149 / -61):
pyrit/auth/azure_auth.py— NewAsyncTokenProviderCredentialclass: wraps sync or async callables into anAsyncTokenCredentialfor the Azure SDK async client. Needed because unlike OpenAI's SDK (which accepts bare callables), the Azure SDK requires a credential object implementing the fullAsyncTokenCredentialprotocol (get_token,close,__aenter__,__aexit__). Async counterpart to the existingTokenProviderCredential.pyrit/auth/__init__.py— ExportAsyncTokenProviderCredential.pyrit/score/float_scale/azure_content_filter_scorer.py— Switched to asyncContentSafetyClient. Added_ensure_async_token_provider()(same pattern asopenai_target.py) to wrap sync callables at init time.api_keynow acceptsstr | Callable[[], str | Awaitable[str]]. Default (no key) falls back to Entra ID viaget_azure_async_token_provider(). Removed sync rejection guards.tests/unit/score/test_azure_content_filter.py— Updated mocks toAsyncMock, converted rejection tests to acceptance tests, addediscoroutinefunctionassertions. 17/17 passing.tests/integration/score/test_azure_content_filter_integration.py— RemovedAZURE_CONTENT_SAFETY_API_KEYgate; tests now use Entra ID auth by default, only requiringAZURE_CONTENT_SAFETY_API_ENDPOINT. 2/2 passing.tests/integration/mocks.py— Fixed pre-existing broken import (AttackIdentifier→ComponentIdentifier). This was blocking all 3 integration test files that import from this module.Auth paths after this change:
api_keyvalueNone(no env var)get_azure_async_token_provider()→ async callable →AsyncTokenProviderCredential→ async client"my-api-key"(string)AzureKeyCredential(key)→ async client_ensure_async_token_provider()wraps →AsyncTokenProviderCredential→ async clientAsyncTokenProviderCredential→ async clientTests and Documentation
Unit tests (
tests/unit/score/test_azure_content_filter.py): 17/17 passing. Updated 6 tests fromMagicMock()toAsyncMock()for the now-async client methods. Converted 2 rejection tests into acceptance tests (async callables are now accepted, not rejected). Addedinspect.iscoroutinefunctionassertions to verify sync→async wrapping. Added pre-condition assertions on sync callable inputs.Integration tests (
tests/integration/score/test_azure_content_filter_integration.py): 2/2 passing. Tests now use Entra ID auth by default (the scorer's default when no API key is provided), requiring onlyAZURE_CONTENT_SAFETY_API_ENDPOINTandaz login.Pre-commit hooks: All passing for our code — ruff format, ruff check, mypy (421 source files checked, 0 errors in our files).
JupyText: Not applicable — no documentation notebooks were modified in this change.