feat(backends): EmbeddedBinding implements apply_activation (Granite Switch path); remove render_controls + set_request_adapter (Epic #929 Phase 2) - #1559
Draft
planetf1 wants to merge 3 commits into
Conversation
…ender_controls + set_request_adapter (Epic generative-computing#929 Phase 2) EmbeddedBinding gets one method, apply_activation(request, identity), for adapters embedded in the served base model (Granite Switch) — there is no weights lifecycle to prepare/activate/deactivate/release, only a field on the outgoing request. OpenAIBackend's inline isinstance(adapter, EmbeddedIntrinsicAdapter) block that wrote chat_template_kwargs.adapter_name and dropped the rewriter-set model param is now owned by the binding, reached through EmbeddedIntrinsicAdapter.weights. Removes the two dead activation methods the previous scope was built around (AdapterMixin.render_controls, AdapterMixin.set_request_adapter) along with OpenAIBackend's no-op render_controls override — neither had a working implementation or caller. adapter_scope() now rejects a non-WeightsBinding adapter with a clear TypeError instead of an AttributeError, since EmbeddedBinding has no activate()/deactivate() to scope. Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
generative-computing#1142) Adds an Embedded/Granite Switch construction example to docs/docs/advanced/intrinsics.md (Adapter(weights=EmbeddedBinding.from_base_model(backend))) alongside the existing LocalFileBinding one, plus a backend x reality support matrix. Adds a weights-binding shapes reference table to AGENTS.md Section 14, comparing LocalFileBinding's activate()/deactivate() lifecycle against EmbeddedBinding's single apply_activation(request, identity) request edit. Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
…ation Independent review (3 perspectives) found one BLOCKER and several correctness/design issues in the initial generative-computing#1142 implementation, all verified against the pinned source before fixing: - BLOCKER: apply_activation fired its two telemetry hooks via _run_async_in_thread from inside an already-running coroutine (OpenAIBackend._generate_from_intrinsic), spawning a throwaway asyncio loop + daemon thread per call that was never reclaimed (10 calls with metrics enabled leaked 20 threads, confirmed independently). Fixed by making apply_activation async and awaiting invoke_hook directly. - WARNING: apply_activation fired adapter_function_invocation_complete with outcome="success" hardcoded, before generation/parsing (which OpenAIBackend resolves lazily) could possibly have failed. Fixed by no longer firing invocation_complete from apply_activation -- only phase_complete (phase="activate"), which the method genuinely completes. Wiring a real invocation-complete signal in requires the caller to fire it once generation/parsing resolve; documented as a follow-up rather than solved here. - WARNING: EmbeddedActivationRequest, required to call the public apply_activation, wasn't exported from mellea.backends.adapters. Exported it and switched openai.py to import both it and EmbeddedBinding from the public package. - Hardened the openai.py activation branch with an explicit else: raise TypeError for a reassigned .weights (previously would have silently skipped activation), and wired base_model_name into EmbeddedBinding.source in OpenAIBackend.add_adapter (was always ""). - Added the missing adapter_scope TypeError regression test, fixed a _fire_phase_complete naming collision (same name, different first parameter, across two binding classes), corrected the docs' composable-construction example (built a backend it never bound, and implied backend support the code doesn't have), dropped a misapplied openai backend marker from a fully-mocked integration test, and removed a duplicate integration test. Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does
Granite Switch serves some LoRA/aLoRA adapters "embedded" — already baked into the model weights, turned on per-request by a control token in the chat template rather than loaded at runtime. Activating one of these had no proper home in the codebase: it was a hand-written
isinstancecheck inline insideOpenAIBackend, including a non-obvious step (dropping a straymodelparameter the rewriter config sets) that any new backend wanting the same support would have had to rediscover and copy by readingopenai.py. Meanwhile the class that should have owned this,EmbeddedBinding, was a stub that raisedNotImplementedErroron every method.This PR gives embedded-adapter activation a real, tested home —
EmbeddedBinding.apply_activation()— and removes two backend-mixin methods (render_controls,set_request_adapter) that were dead code from an earlier, since-abandoned design for the same job.Where this sits in Epic #929
Part of Epic #929 ("Adapter Function Lifecycle"), Phase 2 (Wave 4). Fixes #1142, which discussion #1486 rescoped from "implement the four
WeightsBindingverbs forEmbeddedBinding" down to "implement one method" once it became clear an embedded adapter has no weights lifecycle to manage — nothing to download, load, toggle, or unload; the only thing that varies per call is a field on the outgoing request.Adapter/Identity/IOContract/WeightsBinding), refactor(backends): AdapterMixin verb rename/narrow + resolve_model_options + IntrinsicMetricsPlugin (Epic #929 Phase 2) #1140 (Phase 1 — narrowedAdapterMixinverb contract).LocalFileBindingverbs for the LocalFile/PEFT reality; this is the Embedded/Granite Switch counterpart — same epic, same phase, different weights-binding shape.EmbeddedBindingagainstLocalHFBackend— the acceptance test for this shape working across backends) and refactor(intrinsics): remove deprecation shims; rewrite intrinsics_and_adapters.md; write 3 tutorials (Epic #929 Phase 4) #1144 (removing the deprecatedEmbeddedIntrinsicAdapter/IntrinsicAdaptershims, once nothing needs them).How this PR evolved
Initial implementation
EmbeddedBinding(mellea/backends/adapters/_core.py) gets one method,apply_activation(request, identity), and stops being aWeightsBindingsubclass — noprepare/activate/deactivate/release.EmbeddedIntrinsicAdapter.weights(the deprecated shimOpenAIBackendstill registers today) is now a realEmbeddedBindinginstead of a stub.OpenAIBackend._generate_from_intrinsic's inlineisinstance(adapter, EmbeddedIntrinsicAdapter)block is replaced with a call throughadapter.weights.apply_activation(...)— the backend now activates through the binding, not a copy-pasted snippet.AdapterMixin.render_controls/AdapterMixin.set_request_adapter(NotImplementedErrorstubs with no working implementation or caller) andOpenAIBackend's no-oprender_controlsoverride, plus their tests.AdapterMixin.adapter_scoperaises a clearTypeErrorfor a non-WeightsBindingadapter instead of anAttributeErrorthree lines later — it drives the lifecycle verbs, whichEmbeddedBindingno longer has.docs/docs/advanced/intrinsics.mdgets a composable-construction example;AGENTS.md§14 gets a weights-binding shapes reference table.After review
Ran this through 3 independent reviewers (exhaustive, pragmatic-senior-engineer, and pattern-matching perspectives). One BLOCKER and several smaller issues survived independent verification — including reproducing the BLOCKER myself before trusting the report — and are fixed here:
apply_activationfired its telemetry hooks via_run_async_in_thread— a bridge meant for calling async code from sync code — but its only caller,OpenAIBackend._generate_from_intrinsic, is already a coroutine. Called from inside an already-running event loop, that bridge spins up a brand-newasyncioloop and daemon thread per call and never reclaims them (a cross-threadloop.stop()doesn't reliably wake a loop blocked in its selector). Measured before the fix: 10 embedded-adapter calls with metrics enabled left 20 threads running after a 2-second settle; confirmed at 0 after.apply_activationis nowasync defand awaits the hook directly, matching every other async call site in the codebase.outcome="success"for calls that hadn't finished yet (WARNING, fixed by scope narrowing).apply_activationonly edits the outgoing request —OpenAIBackendresolves generation and parsing later, lazily, once the caller awaits the result. Firingadapter_function_invocation_completeat edit time meant guessing an outcome the method can't know, and it always guessed "success," so a Granite Switch call that failed or returned malformed JSON would still record success. Fixed by no longer firing that hook fromapply_activation; onlyadapter_function_phase_complete(phase"activate", which the method genuinely completes) fires now. Wiring a real invocation-complete signal in requires the caller to fire it once generation and parsing resolve — that's a bigger change than this issue's scope, so it's called out in the code (EmbeddedBinding.apply_activation's docstring) and tracked as Wire real adapter_function_invocation_complete outcome for Embedded activation (follow-up to #1142) #1560 rather than silently dropped.EmbeddedActivationRequestwasn't exported (WARNING, fixed). It's a required parameter type of the publicapply_activation, but was missing frommellea.backends.adapters.__all__— a third-party backend implementation had no supported way to import it. Now exported alongside its siblings;openai.pyimports both it andEmbeddedBindingfrom the public package instead of reaching into the private_coremodule..weights(fixed).EmbeddedIntrinsicAdapterpermits attribute mutation, so a caller reassigning.weightsafter construction would have silently skipped activation (request sent with noadapter_name). The activation branch inopenai.pynow has an explicitelse: raise TypeError(...), matching the fail-loud guard three lines above it, andOpenAIBackend.add_adapternow also threadsbase_model_nameinto the binding'ssourcefield (previously always empty, despite being documented for the futuremellea.adapter_function.sourcespan attribute).openaibackend marker from a fully-mocked integration test (test/README.mdscopes backend markers toe2e/qualitativeonly), added the previously-missing test foradapter_scope's newTypeError, renamed two tests whose names overpromised what they check, fixed the docs' composable-construction example (which built aLocalHFBackendit never used and implied aLocalFileBinding/EmbeddedBindingcombination that neither backend'sadd_adapteractually accepts yet — the example now usesbind_backend()and is explicit that the composedAdapterdataclass isn't wired into any backend's registration path yet, only the bindings themselves are), and resolved a naming collision between two same-named_fire_phase_completemethods with different parameter meanings.Test plan
test/backends/test_adapters/test_embedded_binding.py—apply_activationsetschat_template_kwargs.adapter_name, dropsmodel, preserves existingchat_template_kwargs, is stateless across calls, has noprepare/activate/deactivate/release, firesphase_completewithbinding_type="embedded", and does not fireinvocation_complete.test/backends/test_adapters/test_embedded_integration.py(integration) — a realOpenAIBackendactivates a realEmbeddedBindingthrough_generate_from_intrinsic, with the OpenAI async client mocked at the network boundary.test/backends/test_adapters/test_shims.py::test_adapter_scope_rejects_an_embedded_binding— pinsadapter_scope's newTypeErrorguard.test/backends/test_openai_intrinsics_unit.pyandtest/backends/test_openai_intrinsics.py(GPU e2e) continue to exercise the same observable request shape, unchanged.grep -rn "render_controls\|set_request_adapter" mellea/ test/ docs/— no hits.run_foreverthreads with metrics enabled) and confirmed it's gone after (threads stay at 2 across the same 10 calls).uv run pytest -m "not qualitative"— full suite passes.uv run ruff format --check ./uv run ruff check .— clean.uv run mypy .— clean (718 source files).tooling/docs-autogen/audit_coverage.py --quality --fail-on-quality --threshold 100— 0 issues.npx markdownlint-cli2on the touched doc pages — clean.Assisted-by: Claude Code