- Package Name:
azure-ai-contentunderstanding
- Package Version:
1.1.0
- Operating System: Using Docker-image -
python:3.11-slim with k8s
- Python Version:
3.11
Describe the bug
We are using Azure AI Content Understanding(ACU) to extract structured data from documents. Requests are submitted asynchronously via the Python SDK (begin_analyse), and results are retrieved through polling on the returned AsyncLROPoller. When multiple documents are submitted for processing at around the same time, a subset of the ACU requests intermittently time out.
Resubmitting a timed-out file in isolation succeeds, and the response is typically returned within 10–12 seconds. This indicates the issue is not related to the content of any specific file, but rather to how the service behaves under concurrent load.
Troubleshooting Performed -
We suspected the timeouts were caused by request concurrency (a "thundering herd" effect on the ACU service) and tried the following mitigations, none of which resolved the issue:
- Increased the SDK polling interval from 1s to 5s to reduce polling load.
- Increased the Tokens Per Minute (TPM) quota on the underlying
gpt-4.1-mini deployment from 250K to 35M TPM.
- Reduced the number of concurrent requests sent to ACU from 10 to 5 (this reduced but did not eliminate the errors, and increased overall processing latency), with exponential back-off retries.
- Added random jitter to concurrent requests so they would not fire at the exact same time.
- Added a rate-limiting mechanism to space out request submissions to ACU, with small randomized delays (0–200ms) between each request.
Key Finding -
We changed the chat completion model used by the ACU analyzer from gpt-4.1-mini to gpt-5.2 and gpt-4.1, and the timeout errors stopped occurring — under the same concurrency and request volume that previously produced timeouts.
This strongly suggests the root cause is specific to the gpt-4.1-mini deployment (or how ACU analyzers utilize that model), rather than general request concurrency on our end.
Request to Azure Support -
Could you help us understand:
- Whether there is a known issue or capacity constraint with ACU analyzers backed by
gpt-4.1-mini under concurrent load.
- Whether
gpt-4.1-mini deployments used by ACU have different throughput/latency characteristics or throttling behavior compared to gpt-5.2 / gpt-4.1 that would explain this.
- Any recommended configuration (deployment SKU, region, capacity type) to make
gpt-4.1-mini reliable for this workload?
azure-ai-contentunderstanding1.1.0python:3.11-slimwithk8s3.11Describe the bug
We are using Azure AI Content Understanding(ACU) to extract structured data from documents. Requests are submitted asynchronously via the Python SDK (
begin_analyse), and results are retrieved through polling on the returnedAsyncLROPoller. When multiple documents are submitted for processing at around the same time, a subset of the ACU requests intermittently time out.Resubmitting a timed-out file in isolation succeeds, and the response is typically returned within 10–12 seconds. This indicates the issue is not related to the content of any specific file, but rather to how the service behaves under concurrent load.
Troubleshooting Performed -
We suspected the timeouts were caused by request concurrency (a "thundering herd" effect on the ACU service) and tried the following mitigations, none of which resolved the issue:
gpt-4.1-minideployment from 250K to 35M TPM.Key Finding -
We changed the chat completion model used by the ACU analyzer from
gpt-4.1-minitogpt-5.2andgpt-4.1, and the timeout errors stopped occurring — under the same concurrency and request volume that previously produced timeouts.This strongly suggests the root cause is specific to the
gpt-4.1-minideployment (or how ACU analyzers utilize that model), rather than general request concurrency on our end.Request to Azure Support -
Could you help us understand:
gpt-4.1-miniunder concurrent load.gpt-4.1-minideployments used by ACU have different throughput/latency characteristics or throttling behavior compared togpt-5.2/gpt-4.1that would explain this.gpt-4.1-minireliable for this workload?