[joshistoast]

Summary

Hosted image providers (OpenAI, Gemini, Seedream and the other external generators) lost several legacy capabilities in webv2.

Provider options and named sizes. Generate offers the options each provider's node accepts (OpenAI quality, background and input fidelity for GPT Image models; Gemini temperature and thinking level; Seedream watermark and optimize-prompt) only for models whose provider and capabilities support them, plus the provider's named image sizes and preset resolutions. They compile into the external invocation with the node's own field names, are recorded in metadata and restored by Recall All and Remix. The named size follows the dimensions: a hand-typed size equal to a preset sends that preset's name, any other size sends none, and switching to a provider model snaps the frame to its nearest preset. Recall restores a named size only when the metadata pins one preset, and a same-provider recall resets options the image left at the provider default.

Canvas. External models generate from Canvas in text-to-image and image-to-image, with the composited frame as the init image. Canvas requests a size from the provider's own table (the closest ratio, smallest size covering the frame), so the backend sends it unchanged; Gemini sends its smallest covering preset. Text-to-image output is scaled to cover the bounding box and centre-cropped, image-to-image resizes the init image to the request size and its output back to the bounding box, and metadata keeps the provider's request identity and seed per image. Inpaint is refused with a named reason because the Canvas keep-mask matches no provider's edit-mask contract; outpaint, control layers and regional guidance are refused with named reasons both in readiness and at invoke. Readiness also checks the requested size against the model's maximum size, the Seedream image budget (references that will be sent, init image and output), and raster content in the frame for text-only models. The render-size slot and strength control are hidden for providers.

Model record editor. External models gain an editor for the provider (a select over the registered providers), provider model id, supported modes, seed and reference support, image and reference limits, and default width/height/image count, saved through the model-records update contract with untouched capability tables preserved. A record whose source matches a starter-catalog entry is shown read-only, routing ids included, because the backend overlays that entry's capabilities on every response, so a changed route would keep the original entry's capabilities. Edits wait for the catalog lookup, with a Retry on failure; a retained draft is shown with a note that it cannot be saved.

Related Issues / Discussions

Follows up the legacy parity audit behind #9803.

QA Instructions

From invokeai/frontend/webv2 (Node 22.14, pnpm 10.12): pnpm check:release passes; three browser files in untouched code (GalleryMaintenance, WorkbenchHotkeyRuntime.modal, ToastContrast) timed out once under a load average above 12 and pass when rerun alone. uv run pytest tests/app/invocations/test_frontend_external_canvas_graphs.py tests/app/services/external_generation validates the compiled Canvas graphs against the real invocation fields and edge types, executes them, and checks that every starter size reaches the provider unchanged.

Mock backend (representative profile; four fixture models are external, one of them catalog-managed):

1. Models → "Fixture GPT Image": the External Provider section shows provider (select), model id, modes, limits and defaults. Toggle "Accepts a seed", Save: toast, unsaved marker clears, and a reload shows the saved value. "Fixture Gemini Flash (Catalog)" shows the whole section read-only with a "Managed by the starter catalog" notice; while the catalog is loading or unavailable the section stays locked and offers Retry, which keeps keyboard focus. Saving an invalid limit marks that field and moves focus to it; a rejected save shows the backend's message rather than raw JSON.
2. Editor → model picker → External → "Fixture Gemini Image" (from a local model: no confirm dialog, a "Dimensions adjusted" notice): the Size section shows Provider size presets (pick "16:9 (2K)" → 2048×1152; type 1024×1024 → the select reads "1:1 (1K)"); Provider options show Temperature override and Thinking level. "Fixture GPT Image" shows Quality, Background and Input fidelity; "Fixture Seedream" shows the two toggles.
3. Canvas with an external model: the Size section shows the frame size only and the Render section has no strength. A control or regional guidance layer, raster content in the frame for a text-only model, or a frame over the model's maximum size disables Invoke with the reason; an inpaint mask is refused at invoke. Invoke and the queue cannot be exercised on the mock (no enqueue route); graph shape is covered by the compile tests and the Python graph test, which runs the graph with one and two fake provider images. A real provider key is needed to see output and recall end to end.

Review

Resolved from the first review: hosted-provider Canvas output was placed at the provider's bucketed size instead of the bounding box (now resized back, with metadata preserved); Canvas inpaint reached providers that ignore or misread the mask (now refused); the control-layer and regional-guidance rejections were unreachable (now named in readiness and at invoke); readiness validated only text-to-image, maximum size and the Seedream budget were checked only by the backend; recall could restore a named size without its dimensions and did not reset omitted provider defaults; a typed preset size read as custom and model switches never selected a preset; starter-managed records could change routing ids while keeping the old catalog overlay, and a failed catalog lookup exposed edits the backend reverted. The browser timing baselines carry main's values; bytes and the four new source owners (externalProvider.ts, dimensionLimits.ts, GenerateProviderOptionsSection.tsx, ExternalSizeSelect.tsx) are the branch's own.

Resolved from the second review: switching to a hosted model reported Dimensions as removed (size snaps are now an adjustment notice); Canvas text-to-image output was stretched when the frame ratio was not in the provider's table (now covered and centre-cropped); Gemini 3 on Canvas sent no preset; text-only models failed at invoke on frame content; save errors showed raw JSON, Retry dropped focus, and invalid fields were unmarked; DALL·E showed options its endpoint ignores; the Seedream budget counted references that are not sent; the timing baselines had been re-recorded on a busy machine.

Resolved from the approving review: hosted models were refused on a frame with no visible content (invoke now checks the composite's pixels, readiness each visible layer's bounds); a model switch or recall on Canvas resized the frame to a provider preset; recall of a Canvas output dropped its named preset on Generate; the Canvas Size hint did not name the requested size; a repeated catalog failure was not announced again.

Limitations: readiness cannot see transparent pixels inside a layer's bounds, so it may block a frame whose overlapping pixels are all transparent; Canvas derives the mode from pixel coverage at invoke time, so a Seedream budget or inpaint refusal that depends on the painted content appears as an invoke notice rather than a disabled button; the text-to-image crop drops the frame edges outside the provider's ratio; no real provider execution was performed; OpenAI input fidelity is offered only when the model takes image inputs (the backend uses it only on the edits endpoint).

Compatibility / Rollout

New generate settings are optional with defaults and normalize on load; no project-document migration. No backend or API changes; the Python addition is a test over the frontend's graph fixture.

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Meaningful regression coverage added / updated where needed; obsolete tests/code removed
• Persisted-state and API changes include required migrations / compatibility validation
• Relevant performance/efficiency opportunities considered; material claims have evidence
• Material review findings resolved and relevant checks rerun
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

[lstein]

Reviewed at 172193b. Format, oxlint, tsc, the architecture check, the unit suite (10,150 tests) and the browser tests in the touched files all pass. The performance gate was not run.

Blocker

Canvas output from hosted providers is not sized to the bounding box.

canvasProcessingSize.ts sends the raw bbox. compileCanvasGraph.ts adds no resize back to it. The backend then snaps the request to the provider's size table (_bucket_request / _bucket_to_size in external_generation_default.py), resizing the init image along with it. Only inpaint is resized back (_get_resize_target_for_inpaint). Running the backend's bucketing against the starter capabilities gives:

Model	bbox	sent / returned
gpt-image-1	640×512	1024×1024 (init squashed to 1024²)
gpt-image-1	512×512	1024×1024
seedream-4-0	1024×1024	2048×2048
seedream-4-0	1000×700	2496×1664

Gemini 3 with no matching preset gets no image_size, so a 2048 bbox comes back as a 1K image.

Staging places each result at its own size at the bbox origin (workbenchState.ts: placement = { width: image.width, height: image.height, x: bbox.x, y: bbox.y }). The staged layer therefore overflows or underfills the frame and changes aspect ratio. For img2img the content is also stretched. Core metadata records the bbox size (640×512) while the image is 1024×1024.

To reproduce: pick GPT Image 1, draw a 640×512 bbox over a full raster layer, and invoke. This is the same bbox the compile test uses, whose comment says "the provider returns one image at the bbox size".

The local path restores to the bbox, and the external path needs the same: either a resize after the provider node, or a backend resize back to the requested size for every canvas mode.

Non-blockers

1. Canvas inpaint does not work correctly with any shipped provider. graph.ts sends the canvas grayscale mask (white = keep, dark = edit), which the backend loads as L.
   • OpenAI (providers/openai.py) uploads that L PNG as the edits mask, but the endpoint expects alpha transparency to mark the edit region.
   • Gemini, Seedream, Alibaba and Atlas never read mask_image and repaint the whole frame. No blend tail is added, so unmasked pixels change.
   • Only custom records reach this today, because no starter advertises inpaint. Consider refusing inpaint until a provider handles masks, or compositing the result over the init image through the mask.
2. The regional guidance rejection is unreachable. In prepareCanvasInvocation.ts, createRegionalGuidanceCollector returns false for base external, because getRegionalGuidanceSupport is null. The regional layer is silently skipped and the run is queued without it, so the "does not support regional guidance" reason in compileCanvasGraph.ts never fires. Control layers throw ControlLayerValidationError('unsupported_adapter') in shouldComposite before compile runs, so users get the generic adapter message, not the named one. The compile tests pass regionalGuidance / controlLayers in directly, which hides this.
3. Canvas readiness validates only txt2img. workbench/invocation.ts calls getGenerationValidationReasons(values.model, values) with the default mode on the canvas branch too. A model advertising ['img2img'] is blocked from Canvas even when a full raster layer would compile it as img2img.
4. max_image_size is not checked before queueing. For example, Atlas FLUX.2 Pro / Z-Image Turbo (max 2048, no aspect buckets) with a 2500×1500 bbox passes the frontend and then fails in _validate_request. Separately, Seedream with 14 references plus the canvas init image plus one output is 16, over the 15 limit, so it raises at runtime (seedream.py). This happens only on Canvas.
5. Recall can restore a named size that is never sent. imageRecall.ts applies image_size when the model lists that name but does not apply the preset's width and height. resolveExternalImageSize then drops it.
   • Trigger: recall an image from a backend node run in the workflow editor without core metadata. It has image_size but no width/height.
   • Result: the size select shows "Custom" while the toast counts the field as applied.
   • Either apply the preset's dimensions with the name, or skip it. The test "restores … the named size … on a remix too" uses metadata without width/height, so it asserts this behaviour.
6. Recall does not reset nullable options. gemini_temperature, gemini_thinking_level and openai_input_fidelity are omitted from metadata when null, and recall only applies keys that are present. Remixing a default-temperature Gemini image while the current temperature is 2 keeps 2. A same-provider recall should reset these to null, as LoRAs are cleared when the image had none.
7. A typed size equal to a preset reads as "Custom" and sends no image_size. For example, Gemini 3 Pro with 2048×2048 typed in gets its default 1K. Model switches never set externalImageSize, whereas webv1 selects the model's first preset on switch (modelSelected.ts).
8. Performance timing baselines look loosened rather than re-measured (57273a4).
   • The launchpad's DOMContentLoaded, load and route-ready medians rose 21–34%, and editor-minimal rose 18–44% (projectSwitch included).
   • Canvas, workflow and gallery are flat.
   • The launchpad bundle grew about 9 KB, with the same request count.
   • With a 10% tolerance, this raises those routes' limits by several times the tolerance. Please re-record on a quiet machine, or explain the regression.
9. Starter-managed records: changing provider ids keeps the old catalog overlay. apply_external_starter_model_overrides matches on config.source, and a PATCH of provider_id / provider_model_id does not change source.
   • Trigger: change gemini-2.5-flash-image to gemini-3-pro-image-preview on an installed starter record.
   • Result: the record keeps 2.5-flash's modes, limits and defaults, and the read-only panel presents them as describing the new target.
   • Either lock the routing ids on starter-managed records, or explain in the notice that capabilities stay tied to the catalog entry.
10. Smaller editor issues:
    • If /starter_models fails for an admin, the full form shows. A capability edit then saves, the response re-applies the overlay, and every draft field is dropped as "submitted" behind a success toast, with no sign of the revert.
    • A capability draft made in that fallback form stays hidden, and "Unsaved changes" still shows, once a retry turns the panel read-only. Save then discards it.
    • After picking a different provider, a record's original unconfigured provider_id disappears from the select. Getting it back needs Reset, which discards the other edits.
    • The provider list comes from all registered providers, not only configured ones as the description says.

Test gaps

• In GenerateDimensionFields.tsx, replacing externalImageSize: settings.externalImageSize with null passes every GenerateDimensionFields and ExternalSizeSelect browser test. The wiring between the dimension fields and the select is untested; the select is only tested on its own.
• Removing the >= GEMINI_TEMPERATURE_MIN bound (in normalization or recall) fails no test.
• Removing the local-model early return in getMetadataExternalProvider fails no test.

[Pfannkuchensack]

Re-reviewed after lstein's review. Line numbers refer to 165bb7f; the later main merge resolved no conflicts in these files.

lstein's blocker is fixed, and so are almost all of the other points. Before merge I'd like two user-visible issues fixed (1 and 2 below) and the description corrected (3). The rest are minor points and nits.

lstein's review

Point	Status
Blocker: Canvas output not sized to the bbox	Resolved. compileCanvasGraph.ts:254-276 chains provider → iterate → img_resize to the bbox, with metadata_from_image. I ran the compiled graph through GraphExecutionState with fake provider outputs of 1 and 2 images. Every image comes out at the bbox size and keeps its own provider metadata.
1 inpaint	Resolved. It is refused on the real invoke path, and the docs are updated.
2 regional / control rejections	Resolved. The collectors throw named reasons, readiness names them, and real-path tests cover both.
3 readiness only txt2img	Partial, as the Limitations section says. See minor point 5.
4 max_image_size / Seedream budget	Resolved. The comparisons and the total of 15 match _validate_request and seedream.py, with no off-by-one.
5, 6, 7 recall and named sizes	Resolved.
8 perf baselines	Not resolved. See 3.
9 starter-managed routing ids	Resolved. Catalog-managed records lock provider, model id, capabilities and defaults.
10 editor issues	Resolved, all four.
Test gaps 1–3	Resolved. lstein's mutations now fail tests.

Before merge

1. Switching to a hosted model reports "Dimensions" as removed. At baseGenerationPolicies.ts:1511, snapping to a provider size adds 'Dimensions' to the cleared labels. GenerateModelCard.tsx:71 then treats the switch as lossy.

• Example: switch from SDXL at 1024×1024 to Seedream 4.5. You get a confirmation dialog saying "Switching to Seedream 4.5 turns off or removes: Dimensions". Nothing is removed; the size just becomes 2048×2048.
• Scope: I checked this against all 29 starter capability sets. It fires for most SD/SDXL sizes on GPT Image, DALL·E 3, Gemini, the Atlas preset models, Seedream and Qwen. Switching back to a local model from a grid-1 preset (e.g. 683×1024) hits the clamp label too.
• Suggested fix: report the snap as an adjustment, not as a cleared setting. The Video widget's switchModelAdjusts already does this. Please also add a clearedLabels assertion.

2. Canvas text-to-image output is stretched. img_resize doesn't keep the aspect ratio. Whenever the frame's ratio isn't in the provider's size table, the bucketed result is distorted on the way back to the bbox. I checked these numbers against the backend's _bucket_request:

• GPT Image 1, 640×512 frame: buckets to 1024×1024, so the output is stretched 1.25× horizontally.
• GPT Image 1, 1920×1080 frame: buckets to 1536×1024, stretched 1.19×.

Image-to-image mostly cancels out, because the backend squashes the init image the same way. For text-to-image, I'd resize to cover and center-crop, or constrain the frame ratio for providers with a size table. At the very least the docs should say so; right now they only say the output "scales back to the exact bounding box".

3. Performance baselines (lstein's point 8). c54124d restored main's timing values. The later merge 2493c04 then re-recorded them on your machine, and every route now sits 20–40% above main. For example:

• launchpad/empty: DOMContentLoaded 97.2 → 127.8 ms, routeReady 215.3 → 275.5 ms
• editor-minimal/empty: routeReady 360.9 → 448.9 ms

Since timingPolicy.enforce is false, no gate actually loosens, and this looks like machine variance. But the description says the baselines are back to main's values. Please copy main's timing fields back and keep only the byte and owner-set changes, which the new code justifies. Otherwise, correct the description.

Minor

4. Gemini 3 on Canvas only matches exact presets. resolveExternalImageSize needs an exact match, so a 2000×2000 frame sends image_size: null. It gets Gemini's 1K default and is upscaled about 2×. On Canvas, picking the smallest preset at least as large as the frame, at the bucketed ratio, would fix it.
5. Text-to-image-only models with raster content in the frame. Most starters are txt2img only (Gemini, Atlas, Alibaba, DALL·E 3). With raster content in the frame, Invoke is enabled and then fails with "… does not support image-to-image generation" or "… outpainting on the canvas". Neither message says the frame's content caused it. Canvas-specific wording would help, or a readiness check that raster-layer bounds intersect the bbox for these models. That check needs no pixel reads.
6. Save errors show raw JSON. The save-error toast (ExternalModelSection.tsx:223-226) uses saveError.message, which for an ApiError is the raw body, so a 422 toast reads {"detail":[{"type":"greater_than",…}]}. getApiErrorMessage(saveError, t('models.failedToSaveExternalSettings')) fixes it, as CpuOnlySetting and RelatedModels already do. The "surfaces a failed save" test throws a plain Error, which hides this.
7. Retry drops keyboard focus. Retry (ExternalModelSection.tsx:279-283) unmounts while the refetch runs, so focus falls to <body>.
8. Two test gaps, each checked by mutation:
   • Deleting replaceModelInStore(updated) (ExternalModelSection.tsx:219) passes every browser test, although the form then shows stale values after the success toast.
   • Dropping external from the syncWithLibrary surface loop (modelDraftsStore.ts:233) passes both the store tests and the section tests.

Nits

• QA step 3 says an inpaint mask disables Invoke. Readiness has no inpaint check (invocation.ts:267-279), so the refusal only comes at invoke, as the Limitations section says.
• DALL·E 3 shows Quality and Background (externalProvider.ts:90-95), but openai.py applies them only to _GPT_IMAGE_MODELS. The values still end up in metadata, and the docs line saying DALL·E doesn't expose them was removed.
• The Seedream budget (baseGenerationPolicies.ts:1894-1898) counts references even when supports_reference_images isn't true. graph.ts:1657 doesn't send them in that case, so a custom record can be refused falsely.
• The same refusal strings live in invocation.ts:270-277 and prepareCanvasInvocation.ts:142,262-266. A shared helper would keep them from drifting apart.
• In the editor, validation errors appear only as one alert at the top; the offending field isn't marked invalid. The mode checkboxes have no group label.
• The mock backend returns [] for provider configs and has no catalog-managed external fixture, so the read-only path from QA step 1 can't be tried on the mock.
• test_frontend_external_canvas_graphs.py only checks the graph's shape. A GraphExecutionState run with two fake provider images would pin the iterate expansion and the per-image metadata pairing.

Checked:

• lint:tsc, oxlint, format, the architecture policy tests, the touched unit tests (about 1,100), browser tests (about 60), the fixtures and the Python graph test all pass.
• I clicked through the record editor against the mock backend: save and reload, validation, the locked and catalog-error states, the 422 toast and focus on Retry.
• Not run: the performance gate, and any real provider.

[Pfannkuchensack]

Re-reviewed at 741cd92: the delta since my last review (4f81501, 99405a3) plus the main merge, which resolved no conflicts. Approving: every point from my previous review is fixed. I have one minor issue and a few nits below; none of them block the merge.

Previous review

Point	Status
1 Model switch reported "Dimensions" as removed	Fixed. I re-ran all 29 starters × the SD/SDXL sizes plus 683×1024 back to SDXL: 0 cleared labels, 103 adjustments. In the browser, SD1 1024² → Seedream switches without a dialog and shows the "Dimensions adjusted" toast. A genuinely lossy switch still asks for confirmation.
2 Canvas txt2img stretched	Fixed. It now requests a table size, then cover-resizes and center-crops: 640×512 → 1536×1024 → cover 768×512 → crop x=64. I compiled 292 starter cases (29 models × 8 frames × txt2img/img2img) and ran them through the backend's _bucket_request + _validate_request: 0 rejected, and the worst remaining txt2img aspect error is 0.065% (was 25%). Odd sizes are guarded, and the final node is always bbox-sized.
3 Perf baselines	Fixed. Every timing field equals main's; only the byte data and the four new source owners differ.
4 Gemini covering preset	Fixed. A 2000² frame sends 2K and is resized down. Frames above the largest preset use 4K.
5 Text-only models with frame content	Fixed: blocked in readiness and at compile with one shared message. See N1 below.
6 Raw-JSON save toast	Fixed. It now uses getApiErrorMessage, and the test now throws a real ApiError.
7 Retry focus	Fixed: Retry stays mounted while busy, and focus moves sensibly afterwards.
8 Test gaps	Fixed. Both mutations now fail tests.
Nits	All fixed: QA step 3 wording, the DALL·E 3 gate with the docs line restored, Seedream reference count, shared refusal helper (describeExternalCanvasRefusal), field-level invalid state and focus, modes role="group", the mock's provider configs and catalog fixture, and the GraphExecutionState run in the Python test.

I also ran the compiled graphs through GraphExecutionState with fake provider outputs of 1 and 2 images, in both modes. Each image ends up bbox-sized and keeps its own metadata and seed. The backend test changes in test_external_generation_service.py make the assertions stricter, not looser.

Minor

N1. Text-only providers can be blocked on an empty frame. Readiness (invocation.ts:141-153, via getBaseRasterContentBounds) and invoke (compositeForGeneration.ts:175-197) both decide "raster content in the frame" from the union of all raster-layer bounds.

• Example: with Gemini or DALL·E 3, put image A top-left and image B bottom-right, then place the frame in the empty top-right area. Invoke is disabled with "cannot use image content in the generation frame", although the frame is empty. A layer at opacity 0 inside the frame also blocks.
• For local models the same imprecision only causes a harmless outpaint; here it becomes a hard block with a misleading reason.
• Suggestion: in readiness, test each visible layer's bounds against the frame and skip opacity 0. At invoke, decide from the composite's alpha (any non-transparent pixel) rather than the union.

Nits

• img2img init is still squashed. It goes to the provider squashed when the frame ratio isn't in the table: 3000×500 on GPT Image becomes 1536×1024, a 4× vertical squash, and is undone afterwards. This matches the old backend behaviour and is documented. Padding the init to the request ratio and cropping back would keep the provider's input undistorted.
• Canvas Size hint. For an off-table frame the Size section reads "Custom — Other sizes are mapped to the closest one". It doesn't say that, e.g., 1536×1024 is generated and the edges are cropped. A short Canvas-specific hint would help.
• Switch snap vs. Canvas request policy. The model-switch snap (baseGenerationPolicies.ts:1512) and the Canvas request policy pick sizes differently. GPT Image at 640×512 snaps to 1024² on switch, but Canvas would request 1536×1024. Since Canvas now handles any frame, the snap may not be needed on a Canvas switch.
• image_size on Canvas outputs breaks recall. A Canvas output records image_size: '2K' at 2000×2000. Recall needs a preset whose dimensions match, so it drops the name, and Generate then sends 1K.
• Work on every project change. getInvocationRouteInput builds the full composite plan on every project change while a text-only model is selected, even when the invoke source isn't Canvas. Gating on sourceId === 'canvas' and computing bounds only would avoid that.
• Text layer extents differ. Readiness uses estimateTextExtent for text layers, while invoke uses measured rects, so the two can disagree near the frame edge.
• Retry failure isn't announced. A retry that fails again doesn't re-announce: the role="alert" text in ExternalModelSection.tsx:334-339 stays identical.
• Test gaps:
  • Deleting the notifyGenerateDimensionsAdjusted call in WorkbenchCommandPaletteDialog.tsx:225 passes all palette tests.
  • Deleting the mock's empty-key record removal and its 400 branch (scripts/mock-backend.mjs:1251-1261) passes test:fixtures.

Checked:

• lint:tsc, oxlint, format and the architecture policy tests pass.
• About 2,150 unit tests, 92 browser tests and 42 fixture tests in the touched areas pass, plus the two Python files (391 tests).
• Mutation probes on the new branches; only the two gaps above survived.
• In the browser against dev:mock:
  • the switch toast and the Canvas Size section;
  • the catalog-managed read-only record;
  • catalog failure → Retry with focus;
  • invalid-field marking;
  • the 422 toast.
• Not run: the performance gate, and any real provider.