Skip to content

OpenConceptLab/ocl_online#551 | The Mapper handles recommendations from a decision model - #126

Draft
filiperochalopes wants to merge 2 commits into
OpenConceptLab:mainfrom
filiperochalopes:feature/decision-models
Draft

filiperochalopes wants to merge 2 commits into
OpenConceptLab:mainfrom
filiperochalopes:feature/decision-models

Conversation

@filiperochalopes

@filiperochalopes filiperochalopes commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

PR Notes — feature/decision-models

TL;DR: Makes the Mapper handle recommendations from a decision model (no rationale, no match_strength, its own ai_assistant.decision_calls allowance). Part of OpenConceptLab/ocl_online#551 (decision models for the AI Assistant's LLM-as-judge); this is the OCL Mapper piece of changes that also land in oclapi2 and ocl-ai-assistant. It is not the whole ticket, so there is no closing keyword.

Draft: it depends on the AI Assistant and oclapi2 pieces below, and no decision template exists yet on any environment.


Related PRs

Repo Link Status
oclapi2 OpenConceptLab/oclapi2#976 Draft: registers the ai_assistant.decision_calls capability
ocl-ai-assistant https://github.com/OpenConceptLab/ocl-ai-assistant/pull/240 Draft, stacked on #239 (LiteLLM bump): decision template type, family allowlist, metering by (action_type, inference_type)

Main Changes

Counts from the PR's own file list on GitHub (13 files, +149/−19), against OpenConceptLab/oclmap main.

Area Files Description
Display map-projects/AICandidatesAnalysis.jsx (+4/-0) No empty narrative block when an analysis has no narrative
Log / Discuss map-projects/aiVisibility.js (+24), map-projects/MapProject.jsx (+16/-13), map-projects/Discuss.jsx (+4/-2) An analysis without rationale is logged as <recommendation> <primary code>, never "AIRecommendation: undefined". Discuss also guards older entries saved with no description
Auto Match skip map-projects/autoMatchRows.js (+13/-2), map-projects/MapProject.jsx hasCurrentAnalysis also compares the template key and version. Two call sites pass the run's template
Default template map-projects/aiVisibility.js, map-projects/constants.jsx (+3), map-projects/MapProject.jsx A project that names no template runs the chat default, or PROMPTS_DECISION_KEY_DEFAULT for a user entitled to decision calls and not generative ones
Quota errors common/quotaErrors.js (+2), common/limitAction.js (+2), map-projects/previewLimits.js (+6) ai_assistant_decision_calls_{limit_reached,not_entitled} are AI preview / quota errors and reuse the existing "OCL AI Assistant calls" meter and copy (no new strings in the 4 locales). limitAction.js names the ai_assistant.decision_calls capability, like every other refusal, when asking the Online service for a plan action
Tests aiVisibility.test.js (+35/-1), autoMatchRows.test.js (+26), previewLimits.test.js (+9), quotaErrors.test.js (+5/-1) Written first and failing (7 failures; aiVisibility did not even load), then green

No change needed: the export. __oclai_rationale__, __oclai_confidence_score__ and __oclai_oos_suggestions__ already resolve to null when the field is absent, and the title only appends match_strength when it exists.

What a decision answer looks like in the log

Analysis output Log description
{rationale: 'because'} (generative) because (unchanged)
{recommendation: 'RECOMMEND', primary_candidate: {canonical_reference: {code: 'E11.9'}}} (decision) RECOMMEND E11.9
{recommendation: 'REJECT'} REJECT
{} empty; Discuss shows just AIRecommendation

Key Decisions

Decision Choice Rationale
Skip rule Pool fingerprint and template key + version A chat answer must not stand in for a decision one, nor one version for another. The model is not part of the rule
Missing version Redo An analysis with no recorded version, or a run with no known one, cannot be compared. Side effect: analyses saved without a template version are redone once at the next Auto Match (one-time AI cost on existing projects)
Default template Chosen by entitlement, only when the project names none The server stays the authority (no fallback between chat and decision); this only avoids offering a template the user cannot run. -1 (kill switch) counts as not entitled
Quota copy Reuse the AI Assistant calls meter To the user a decision call is an AI Assistant call; avoids 4 locales of new strings
PROMPTS_DECISION_KEY_DEFAULT match-recommend-decision The template is created by hand against the AI Assistant, and its key must match this constant

Verification Checklist

npm run test
NODE_ENV=development API_URL=http://localhost:8000 npm run build
  • npm run test: 667 tests, 667 pass, 0 fail (after merging upstream main; 577/577 before the merge)
  • npm run build: webpack compiled successfully
  • .jsx files changed parse (Babel, jsx plugin): AICandidatesAnalysis, Discuss, MapProject, constants
  • Regression: generative rationale logs and the export are unchanged (no code change on those paths)
  • Render check in a browser: the unit tests cannot mount JSX, so the narrative block and Discuss line were only verified by syntax
  • Local end-to-end with a real decision response (needs the AI Assistant branch and a decision template)
  • limitAction.js asks the Online service about ai_assistant.decision_calls; not checked whether ocl_online's /billing/plan/ knows that capability (an unknown one would simply show no action)
  • An analysis without rationale stays the same after save and reload
  • Not done: restricting the staff model selector to decision models for a decision template (would duplicate the AI Assistant's model families in the Mapper). Today a staff user who picks a chat model for a decision template gets the server's error

Heads-up for review

filiperochalopes and others added 2 commits November 3, 2026 13:14
… in the Mapper

A decision model returns no rationale, match_strength or confidence_level, and runs on its own
ai_assistant.decision_calls allowance. The Mapper now copes with that:

- AICandidatesAnalysis: no empty narrative block when there is no narrative.
- Log/Discuss: an analysis without a rationale is logged as "<recommendation> <primary code>", never
  "AIRecommendation: undefined" (getAIRecommendationDescription; Discuss also guards older entries).
- Auto Match skip: hasCurrentAnalysis also compares the template key and version, so a chat answer
  does not stand in for a decision one (or one version for another). An analysis with no recorded
  version, or a run with no known one, is redone.
- Default template: a project that names none runs the chat default, or the decision default
  (PROMPTS_DECISION_KEY_DEFAULT) for a user entitled to decision calls but not generative ones.
  Entitlement is still enforced by the server; there is no fallback between the two.
- ai_assistant_decision_calls_{limit_reached,not_entitled} are AI preview/quota errors and reuse the
  existing "OCL AI Assistant calls" meter and copy.

The export needed no change: the rationale, confidence and out-of-scope columns already resolve to null
when absent. Not done: restricting the staff model selector to decision models for a decision template.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The branch was cut from a stale fork main (73 commits behind OpenConceptLab/oclmap main). Conflicts:
- MapProject.jsx: import line; kept both PROMPTS_DECISION_KEY_DEFAULT and applyTemplateColumns.
- previewLimits.test.js: kept both tests (decision-call copy, and upstream's every-refusal-has-a-message).

Follow-ups for upstream changes this branch had not seen:
- limitAction.js (new upstream): map ai_assistant_decision_calls_{limit_reached,not_entitled} to the
  ai_assistant.decision_calls capability, as every other refusal is, so the Online plan action can be asked.
- getQuotaError no longer returns usage: drop it from the decision-call expectation.

npm run test: 667 pass, 0 fail. npm run build: compiled successfully.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant