Skip to content

feat(backends): EmbeddedBinding implements apply_activation (Granite Switch path); remove render_controls + set_request_adapter (Epic #929 Phase 2) - #1559

Draft
planetf1 wants to merge 3 commits into
generative-computing:mainfrom
planetf1:issue-1142
Draft

Conversation

@planetf1

@planetf1 planetf1 commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

What this PR does

Granite Switch serves some LoRA/aLoRA adapters "embedded" — already baked into the model weights, turned on per-request by a control token in the chat template rather than loaded at runtime. Activating one of these had no proper home in the codebase: it was a hand-written isinstance check inline inside OpenAIBackend, including a non-obvious step (dropping a stray model parameter the rewriter config sets) that any new backend wanting the same support would have had to rediscover and copy by reading openai.py. Meanwhile the class that should have owned this, EmbeddedBinding, was a stub that raised NotImplementedError on every method.

This PR gives embedded-adapter activation a real, tested home — EmbeddedBinding.apply_activation() — and removes two backend-mixin methods (render_controls, set_request_adapter) that were dead code from an earlier, since-abandoned design for the same job.

Where this sits in Epic #929

Part of Epic #929 ("Adapter Function Lifecycle"), Phase 2 (Wave 4). Fixes #1142, which discussion #1486 rescoped from "implement the four WeightsBinding verbs for EmbeddedBinding" down to "implement one method" once it became clear an embedded adapter has no weights lifecycle to manage — nothing to download, load, toggle, or unload; the only thing that varies per call is a field on the outgoing request.

How this PR evolved

Initial implementation

  • EmbeddedBinding (mellea/backends/adapters/_core.py) gets one method, apply_activation(request, identity), and stops being a WeightsBinding subclass — no prepare/activate/deactivate/release.
  • EmbeddedIntrinsicAdapter.weights (the deprecated shim OpenAIBackend still registers today) is now a real EmbeddedBinding instead of a stub.
  • OpenAIBackend._generate_from_intrinsic's inline isinstance(adapter, EmbeddedIntrinsicAdapter) block is replaced with a call through adapter.weights.apply_activation(...) — the backend now activates through the binding, not a copy-pasted snippet.
  • Removed AdapterMixin.render_controls / AdapterMixin.set_request_adapter (NotImplementedError stubs with no working implementation or caller) and OpenAIBackend's no-op render_controls override, plus their tests.
  • AdapterMixin.adapter_scope raises a clear TypeError for a non-WeightsBinding adapter instead of an AttributeError three lines later — it drives the lifecycle verbs, which EmbeddedBinding no longer has.
  • Docs: docs/docs/advanced/intrinsics.md gets a composable-construction example; AGENTS.md §14 gets a weights-binding shapes reference table.

After review

Ran this through 3 independent reviewers (exhaustive, pragmatic-senior-engineer, and pattern-matching perspectives). One BLOCKER and several smaller issues survived independent verification — including reproducing the BLOCKER myself before trusting the report — and are fixed here:

  • Thread/event-loop leak (BLOCKER, fixed). The original apply_activation fired its telemetry hooks via _run_async_in_thread — a bridge meant for calling async code from sync code — but its only caller, OpenAIBackend._generate_from_intrinsic, is already a coroutine. Called from inside an already-running event loop, that bridge spins up a brand-new asyncio loop and daemon thread per call and never reclaims them (a cross-thread loop.stop() doesn't reliably wake a loop blocked in its selector). Measured before the fix: 10 embedded-adapter calls with metrics enabled left 20 threads running after a 2-second settle; confirmed at 0 after. apply_activation is now async def and awaits the hook directly, matching every other async call site in the codebase.
  • Telemetry reported outcome="success" for calls that hadn't finished yet (WARNING, fixed by scope narrowing). apply_activation only edits the outgoing request — OpenAIBackend resolves generation and parsing later, lazily, once the caller awaits the result. Firing adapter_function_invocation_complete at edit time meant guessing an outcome the method can't know, and it always guessed "success," so a Granite Switch call that failed or returned malformed JSON would still record success. Fixed by no longer firing that hook from apply_activation; only adapter_function_phase_complete (phase "activate", which the method genuinely completes) fires now. Wiring a real invocation-complete signal in requires the caller to fire it once generation and parsing resolve — that's a bigger change than this issue's scope, so it's called out in the code (EmbeddedBinding.apply_activation's docstring) and tracked as Wire real adapter_function_invocation_complete outcome for Embedded activation (follow-up to #1142) #1560 rather than silently dropped.
  • EmbeddedActivationRequest wasn't exported (WARNING, fixed). It's a required parameter type of the public apply_activation, but was missing from mellea.backends.adapters.__all__ — a third-party backend implementation had no supported way to import it. Now exported alongside its siblings; openai.py imports both it and EmbeddedBinding from the public package instead of reaching into the private _core module.
  • Defense-in-depth for a reassigned .weights (fixed). EmbeddedIntrinsicAdapter permits attribute mutation, so a caller reassigning .weights after construction would have silently skipped activation (request sent with no adapter_name). The activation branch in openai.py now has an explicit else: raise TypeError(...), matching the fail-loud guard three lines above it, and OpenAIBackend.add_adapter now also threads base_model_name into the binding's source field (previously always empty, despite being documented for the future mellea.adapter_function.source span attribute).
  • Test/docs accuracy fixes: removed a duplicate integration test, dropped a misapplied openai backend marker from a fully-mocked integration test (test/README.md scopes backend markers to e2e/qualitative only), added the previously-missing test for adapter_scope's new TypeError, renamed two tests whose names overpromised what they check, fixed the docs' composable-construction example (which built a LocalHFBackend it never used and implied a LocalFileBinding/EmbeddedBinding combination that neither backend's add_adapter actually accepts yet — the example now uses bind_backend() and is explicit that the composed Adapter dataclass isn't wired into any backend's registration path yet, only the bindings themselves are), and resolved a naming collision between two same-named _fire_phase_complete methods with different parameter meanings.

Test plan

  • test/backends/test_adapters/test_embedded_binding.pyapply_activation sets chat_template_kwargs.adapter_name, drops model, preserves existing chat_template_kwargs, is stateless across calls, has no prepare/activate/deactivate/release, fires phase_complete with binding_type="embedded", and does not fire invocation_complete.
  • test/backends/test_adapters/test_embedded_integration.py (integration) — a real OpenAIBackend activates a real EmbeddedBinding through _generate_from_intrinsic, with the OpenAI async client mocked at the network boundary.
  • test/backends/test_adapters/test_shims.py::test_adapter_scope_rejects_an_embedded_binding — pins adapter_scope's new TypeError guard.
  • Existing test/backends/test_openai_intrinsics_unit.py and test/backends/test_openai_intrinsics.py (GPU e2e) continue to exercise the same observable request shape, unchanged.
  • grep -rn "render_controls\|set_request_adapter" mellea/ test/ docs/ — no hits.
  • Independently reproduced the thread-leak BLOCKER before the fix (10 calls → 20 leaked run_forever threads with metrics enabled) and confirmed it's gone after (threads stay at 2 across the same 10 calls).
  • uv run pytest -m "not qualitative" — full suite passes.
  • uv run ruff format --check . / uv run ruff check . — clean.
  • uv run mypy . — clean (718 source files).
  • tooling/docs-autogen/audit_coverage.py --quality --fail-on-quality --threshold 100 — 0 issues.
  • npx markdownlint-cli2 on the touched doc pages — clean.

Assisted-by: Claude Code

…ender_controls + set_request_adapter (Epic generative-computing#929 Phase 2)

EmbeddedBinding gets one method, apply_activation(request, identity), for
adapters embedded in the served base model (Granite Switch) — there is no
weights lifecycle to prepare/activate/deactivate/release, only a field on
the outgoing request. OpenAIBackend's inline
isinstance(adapter, EmbeddedIntrinsicAdapter) block that wrote
chat_template_kwargs.adapter_name and dropped the rewriter-set model param
is now owned by the binding, reached through EmbeddedIntrinsicAdapter.weights.

Removes the two dead activation methods the previous scope was built around
(AdapterMixin.render_controls, AdapterMixin.set_request_adapter) along with
OpenAIBackend's no-op render_controls override — neither had a working
implementation or caller.

adapter_scope() now rejects a non-WeightsBinding adapter with a clear
TypeError instead of an AttributeError, since EmbeddedBinding has no
activate()/deactivate() to scope.

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
generative-computing#1142)

Adds an Embedded/Granite Switch construction example to
docs/docs/advanced/intrinsics.md (Adapter(weights=EmbeddedBinding.from_base_model(backend)))
alongside the existing LocalFileBinding one, plus a backend x reality
support matrix.

Adds a weights-binding shapes reference table to AGENTS.md Section 14,
comparing LocalFileBinding's activate()/deactivate() lifecycle against
EmbeddedBinding's single apply_activation(request, identity) request edit.

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
@github-actions github-actions Bot added the enhancement New feature or request label Aug 18, 2026
…ation

Independent review (3 perspectives) found one BLOCKER and several
correctness/design issues in the initial generative-computing#1142 implementation, all
verified against the pinned source before fixing:

- BLOCKER: apply_activation fired its two telemetry hooks via
  _run_async_in_thread from inside an already-running coroutine
  (OpenAIBackend._generate_from_intrinsic), spawning a throwaway asyncio
  loop + daemon thread per call that was never reclaimed (10 calls with
  metrics enabled leaked 20 threads, confirmed independently). Fixed by
  making apply_activation async and awaiting invoke_hook directly.
- WARNING: apply_activation fired adapter_function_invocation_complete
  with outcome="success" hardcoded, before generation/parsing (which
  OpenAIBackend resolves lazily) could possibly have failed. Fixed by no
  longer firing invocation_complete from apply_activation -- only
  phase_complete (phase="activate"), which the method genuinely
  completes. Wiring a real invocation-complete signal in requires the
  caller to fire it once generation/parsing resolve; documented as a
  follow-up rather than solved here.
- WARNING: EmbeddedActivationRequest, required to call the public
  apply_activation, wasn't exported from mellea.backends.adapters.
  Exported it and switched openai.py to import both it and
  EmbeddedBinding from the public package.
- Hardened the openai.py activation branch with an explicit
  else: raise TypeError for a reassigned .weights (previously would
  have silently skipped activation), and wired base_model_name into
  EmbeddedBinding.source in OpenAIBackend.add_adapter (was always "").
- Added the missing adapter_scope TypeError regression test, fixed a
  _fire_phase_complete naming collision (same name, different first
  parameter, across two binding classes), corrected the docs'
  composable-construction example (built a backend it never bound, and
  implied backend support the code doesn't have), dropped a misapplied
  openai backend marker from a fully-mocked integration test, and
  removed a duplicate integration test.

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(backends): EmbeddedBinding implements apply_activation (Granite Switch path); remove render_controls + set_request_adapter (Epic #929 Phase 2)

1 participant