Python: fix(foundry_hosting): send the base64 image as the image_generation_call result

[Md. Shalah Uddin (shanto462)]

Motivation & Context

_OutputItemTracker in foundry_hosting passed str(content.outputs) to output_item_image_gen_call(result_base64). The OpenAI chat client stores the generated image as a data URI Content there, so a hosted agent that generated an image sent image_generation_call.result: "Content(type=data)" instead of the base64 image.

Description & Review Guide

• What are the major changes? A small helper, _image_generation_result, returns only non-empty, strictly valid base64 for the image_generation_call result: the payload of a base64 data URI Content (the shape the OpenAI chat client produces, also inside a list or tuple), or a plain string. Both pass the same base64.b64decode(..., validate=True) check. The data URI is read as RFC 2397 describes it (and as core's _parse_data_uri does): the base64 marker must be the last parameter before the first comma, so a ;base64, inside the payload is not a marker. A malformed data URI in a list is skipped, and a later valid image is still used. An https, file or blob URI is not base64, so it is not used. When the outputs hold no base64 image, the tracker emits the item with an empty result and logs a warning, instead of the old str(content.outputs) value. A parametrized regression test covers a data URI Content, one with an extra parameter, a list, a tuple, an image after a text item, a plain string or a malformed data URI, a base64 string, and the cases that now give an empty result: an https URI, a data URI without the marker, a malformed payload, missing padding, whitespace in the payload, an empty payload, ;base64, inside the payload, a hosted file, a URL string, a non-base64 string, an empty string, bytes and a mapping. It also checks that the warning is logged exactly when the result is empty.
• What is the impact of these changes? Hosted agents that use image generation now return the image in image_generation_call.result, and the field never carries a URI or other non-base64 text.
• What do you want reviewers to focus on? For outputs without a base64 image the item is still emitted, with an empty result: output_item_image_gen_call requires a string and always marks the item completed, and an empty string is the SDK's own placeholder before the result. Skipping the item instead would also work; the implementation does not fetch remote images.

Related Issue

Fixes #9350

No other open PR covers this issue.

AI Assistance

• No material AI assistance was used.
• This is an AI-assisted contribution. I reviewed, understood, and verified all submitted content and accept responsibility for it.

AI assistance details: issue analysis, implementation, tests, and this PR description.

Contribution Checklist

• The code builds clean without any errors or warnings
• All unit tests pass, and I have added new tests where possible
• The PR follows the Contribution Guidelines
• This PR links to an agreed issue with no competing open PR, or the Related Issue section documents a trivial-change or repository-automation exception.
• This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

[Copilot]

🟡 Changes recommended

String elements in sequences can incorrectly override a later generated-image URI.

1 open finding

• Avoid raw strings overriding URI Content in output sequences · New

What changed in this PR

Fixes Foundry-hosted image generation results by returning base64 image data instead of a Content string representation.

Changes:

• Extracts image data from URI-bearing outputs.
• Uses the extracted result when emitting image generation calls.
• Adds regression coverage for supported output shapes.

File	Description
agent_framework_foundry_hosting/​_responses.py	Extracts image generation results.
tests/​test_responses.py	Tests URI, sequence, and string outputs.

🧠 Review effort: Balanced

---

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

[Eduard van Valkenburg (eavanvalkenburg)]

/review

[github-actions]

MAF Automated Review — Iteration 1

Result: No findings
Scope: full PR (2 commit(s)): a292abf49237, 4b48493c52b5
Model: gpt-5.6-sol

Overview

The change extracts the base64 payload from URI-bearing image-generation outputs while retaining the previous string fallback for unsupported shapes. Its sequence filtering, parity with the sibling hosting-responses serializer, and regression coverage for direct, sequenced, URI, and plain-string outputs prevent a publishable correctness, compatibility, architectural, or security concern.

Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
No publishable findings remained after source verification for this scope.

[Eduard van Valkenburg (eavanvalkenburg)]

Md. Shalah Uddin (@shanto462) python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses.py:3547 returns a non-base64 URI unchanged when an image-generation result contains an HTTPS/file/blob URI. That value is passed to output_item_image_gen_call(result_base64=...) and emitted as image_generation_call.result, which clients interpret as base64. The new https-uri test therefore codifies an invalid provider payload.

Please only emit raw base64 strings or the payload extracted from a base64 data URI. Do not place external/file/blob URIs in the base64 result field unless the implementation actually fetches and converts them first; update the HTTPS test accordingly.

[Md. Shalah Uddin (shanto462)]

Eduard van Valkenburg (@eavanvalkenburg) Fixed in 57bb65d: the result is now only a base64 string or the payload of a base64 data URI; https, file and blob URIs and other shapes emit an empty result with a warning, and the HTTPS test now expects that.

[Eduard van Valkenburg (eavanvalkenburg)]

Md. Shalah Uddin (@shanto462) _image_generation_result() validates plain string outputs, but it returns the payload from any data:*;base64,... URI without validating that payload. A value such as data:image/png;base64,not-base64! is therefore emitted as image_generation_call.result, even though that field is result_base64. Please validate the extracted data-URI payload with the same strict base64 check and add a malformed-payload regression.

[Md. Shalah Uddin (shanto462)]

Eduard van Valkenburg (@eavanvalkenburg) Fixed in 68ff1b7: the data URI payload now passes the same strict base64 check (the marker must be the last parameter before the first comma, as in RFC 2397), a malformed payload gives an empty result with a warning, and new regressions cover a malformed, unpadded, empty or whitespace payload and a later valid image after a malformed one.