[harshach]

Fixes #35981

Concept lifecycle status and ontology publication status are derived from captured releases. The API returns those read-only fields and records a release change summary without adding another stored lifecycle.

The published stack targets release 2.1.0. Each linked Shipping issue is In Progress with Product, P1 and Governance metadata.

main → PR1 #35990 → PR2 #35994 → PR3 #35997 → PR4 #35995. Two chains start at PR4:

• PR4 #35995 → PR5 #36183 → PR6 #36187 → PR7 #36188 → PR8 #36190, open as drafts.
• PR4 #35995 → PR10 #35991 → PR12 #35996 → PR15 #35989.

This PR targets PR1 #35990. Its child is PR3 #35997. PR9, PR11, PR13, PR14 and PR16 are not published yet.

Current assessment for 20e50e5e265, recorded 2026-10-28.

Verdict. The independent review of the review-bot round passed at 20e50e5e265, this head, with no findings. The ledger records its base as PR1 eaf87167807 and its stable patch-id as bf0177fe84b. No carry is needed.

Test evidence at this head. The reviewer ran 24 tests in OntologyReleaseMemberQueryTest and OntologyDefinitionStatusResolverTest, with no failures. The storage tests OntologyConceptStatusStorageTest and OntologyDerivedStatusStorageTest need a database and were not rerun by the reviewer.

CI. Every check at this head passes except maven-collate-ci. That job fails with HTTP 422 (No ref found for: harshach/ontology-studio/01-review-rule). The Collate build is pointed at this PR's base branch, and the Collate repository has no branch of that name. That is a limit of the CI setup for stacked PRs, not a fault in this PR's code. PR1, whose base is main, passes it.

Still owed. Docker is down on the build machine, so none of these has run. Run each IT on -Pmysql-elasticsearch and then -Ppostgres-opensearch, and confirm a Tests run: line.

• The full merge-queue IT suite, which the impact planner selects because the stack changes ListFilter.java (area core). Run first IndependentOntology*IT, Ontology*ResourceIT, TaskResourceIT, TaskChangeEventIT, and WorkflowDefinitionResourceIT in the global-state lane.
• The container-backed unit classes OntologyConceptStatusStorageTest and OntologyDerivedStatusStorageTest.
• Changed-class coverage, which needs the ITs above.
• The plan's live and performance lanes for this PR.

Not verified yet. This PR is verified only when its unit, live and performance boxes in the plan are all checked. The operator lands the stack.

Earlier description and verification records

These records preserve the earlier design notes, test results and attachments. Earlier statements about the current head or readiness apply to their recorded revisions. The integration assessment above supersedes those statements.

Describe your changes:

Fixes #35981

This PR adds a decision record, docs/decisions/2026-10-21-ontology-definition-status-is-derived-from-releases.md. It states that definition status is computed from releases and never stored, so it adds no second lifecycle next to the release review (ADR:2026-10-14-ontology-reviews-bind-workflow-tasks-to-revisions).

Ontology Studio needs a status pill per concept and a publication summary per ontology. Without a server-side source, the UI would invent a lifecycle that can disagree with the releases that hold the concepts.

What changed:

• GET/list /v1/ontologyConcepts?fields=lifecycleStatus returns Draft, InReview or Approved. A concept is Approved when it matches its member in the active or the latest approved release, InReview when it matches its member in the release under review, and Draft otherwise.
• GET/list /v1/ontologies?fields=publicationStatus returns the active release (revision, semantic version, activation time), the latest release and its stage (Draft, InReview, ChangesRequested, Approved, Published), and unpublishedChanges, the row count of the ontology's working diff.
• A captured release stores changeSummary (added, changed, removed) against the release that was active at capture. Older releases have none.
• Clients cannot write these fields. The repositories strip them from storage and version history and reject patches that set them. The SDK drops them before it builds a patch.
• Both fields map to ViewBasic, so a basic viewer can list concepts with the pill. They expose membership and a count, never member content.

Deviation from the plan: a concept matches a member by governed content, with an equal entity version as a shortcut, not by version equality alone. Owner edits and an edit that is reverted move the version without changing content, and the working diff does not list them. Version equality alone would show Draft while unpublishedChanges says 0. A moved version costs one payload read for those pairs. The ADR records this. The change summary type is OntologyReleaseChangeSummary because ChangeSummary already exists in type/change.

Query cost: a concept page runs the release-role query, one member-version query and at most one member-payload query, whatever its size. An ontology page runs a constant number of queries (state, roles, four working-version queries, member versions, and payload reads only for definitions whose version moved).

Leaves out: properties and axioms have no lifecycleStatus. No UI. No migration, since every statement uses an existing index.

Type of change:

• New feature

High-level design:

OntologyDefinitionStatusResolver holds the status table and the batched page read. OntologyPublicationStatusResolver holds the stage table. OntologyReleaseRoles finds the active, latest, latest approved and in-review release per ontology in one window-function query that runs on MySQL 8 and PostgreSQL. OntologyUnpublishedChanges counts working-diff rows from (id, version) pairs and digests content only where versions differ, through the same OntologyReleaseComparison.sameContent the diffs use. Activation time comes from the ACTIVATED event whose activationRevision equals the state's, because findActivation returns the earliest event and is wrong after a rollback.

Tests:

Use cases covered

• Every row of the status table and the stage table, including a concept edited after approval and a concept unchanged inside an in-review release.
• Edit, review, rejection, edit, review, approval, activation, with the status asserted at each step.
• Edit then revert stays Approved (payload fallback). An edit to an approved concept turns only that concept Draft.
• unpublishedChanges equals the rows of the working diff, and changeSummary equals the release diff against its base.
• lifecycleStatus is never stored or versioned, and a PATCH that sets it returns 400.
• Release reads stay constant for a 4 versus a 40 concept page, and for 1 versus 6 ontologies.

Unit tests

• mvn test -pl openmetadata-service -Dtest='OntologyDefinitionStatusResolverTest,OntologyDerivedStatusStorageTest,OntologyConceptStatusStorageTest,RdfOntologyContractTest,OntologyReleaseCaptureTest': 53 tests, the storage tests on real MySQL 8 and PostgreSQL 15 containers. All passed before the rebase. After the rebase I re-ran OntologyDefinitionStatusResolverTest (21 passed).

Backend integration tests

• IndependentOntologyStatusIT, 7 tests, passed on -Pmysql-elasticsearch and on -Ppostgres-opensearch.
• The neighbouring ontology ITs (see the java-affected-tests block) ran after the rebase on both profiles.

Ran with the lane parallel: IndependentOntologyStatusIT, IndependentOntologyReleaseIT, IndependentOntologyWorkingDiffIT, IndependentOntologyActivationIT, OntologyConceptResourceIT, OntologyResourceIT. plan_local_java_tests.py --check-branch reports the impact map complete. The full *Ontology*IT area was not run.

Ingestion integration tests

• Not applicable (no ingestion changes).

Playwright (UI) tests

• Not applicable (no UI changes). The TypeScript types are regenerated.

Manual testing performed

• The ITs drive the embedded server over HTTP with the SDK client. No separate stack was started. The coordinator's live and perf lanes run at the code-ready SHA.

UI screen recording:

Not applicable. This is backend-only work with no UI change.

---

Summary by Gitar

• Derived status fields:
  • Added read-only lifecycleStatus on concepts and publicationStatus on ontologies, computed from releases
  • Captured releases store changeSummary with added, changed and removed counts
• Unpublished changes count:
  • unpublishedChanges counts working diff rows using batched membership and content reads
• Review and vote handling:
  • Added listReviews endpoint and OntologyReleaseReviewReader to list open release reviews per caller
  • Votes are durably recorded through TaskReviewVotes and the workflow handler
• Approval rule:
  • Added OntologyApprovalRule (AnyReviewer or AllReviewers) frozen into review tasks
  • Reviewers and rule are locked while a release is in review

_{This will update automatically on new commits.}

Confidence Score: 5/5

The reviewed changes appear safe to merge; the previous large-page query failure is fixed.

Summary

This PR derives concept and ontology statuses from captured releases and stores change counts when a release is captured.

• Concepts show whether their current content is approved, in review, or still a draft.
• Ontology reads show the active release, latest stage, and count of unpublished changes.
• Each captured release records how its members differ from the active release.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Requested status fields] --> B[Read release roles]
  B --> C[Read member versions in bounded chunks]
  C --> D{Versions match?}
  D -->|Yes| E[Use release membership]
  D -->|No| F[Read payloads in bounded chunks]
  F --> G[Compare governed content]
  E --> H[Return derived status]
  G --> H

Loading

_{Reviews (7) · Last reviewed commit: "Keep release and member ID binds within ..." · Reviewed by Greptile}

[github-actions]

✅ PR checks passed

The linked issue has a description and all required Shipping project fields set. Thanks!

[github-actions]

Jest test Coverage

UI tests summary

Lines	Statements	Branches	Functions
	80.14% (132405/165197)	66.74% (46891/70257)	69.79% (26589/38098)

[gitar-bot]

Code Review ✅ Approved 4 closed / 4 findings

🟡 Medium risk · Deriving ontology approval/publication status and change counts from release membership could misreport which definitions are approved or still unpublished.

Derives concept lifecycleStatus and ontology publicationStatus from captured releases, with a stored changeSummary on each release and no second lifecycle. Four open findings from the earlier review were resolved: the membership match is now defined, unpublishedChanges no longer caps at one page, the publicationStatus shape is documented in the API, and the review list no longer does a full scan with per-release task reads. No open issues remain.

✅ 4 closed

✅ Edge Case: "Matches its member" is undefined, so status can contradict the diff

📄 docs/decisions/2026-10-21-ontology-definition-status-is-derived-from-releases.md:22-24
The Approved/InReview/Draft rule depends on a concept "matching its member", but the record never says how that comparison is made. A release member has two identities: the source entity version and payload digest bound in the manifest, and the semantic comparison the diffs use, which "exclud[es] operational and derived fields" (ADR:2026-10-14-ontology-releases-seal-canonical-sql-members). Suppose an implementation compares entity versions or digests. Then an edit to an operational field only, such as owner or audit fields, turns an approved concept into Draft, while unpublishedChanges (from the semantic working diff) stays at 0. The same ontology would then report two different states. The record should name the comparison, preferably the same semantic member comparison as the diff endpoints, so the two values cannot disagree.

✅ Edge Case: unpublishedChanges counts a paged working diff, so it caps at a page

📄 docs/decisions/2026-10-21-ontology-definition-status-is-derived-from-releases.md:26-27 📄 docs/decisions/2026-10-21-ontology-definition-status-is-derived-from-releases.md:36-37
The record defines publicationStatus.unpublishedChanges as "the number of rows the working diff returns". GET /v1/ontologies/{id}/working/diff returns "a bounded, revision-bound page of working changes" (ADR:2026-10-19-native-ontology-imports-use-working-drafts), and ontologyWorkingDiff.json limits data to maxItems: 1000 with a cursor. Read literally, the rule makes the count stop at one page. Counting every page instead would walk the cursor for each ontology, which breaks the Consequences claim that a list page reads the status "with a fixed number of SQL statements whatever its size". The record should say the value is the total number of member changes between the working revision and the active release, computed without paging, at the working revision in the same ontology_state read. It should also say this comparison uses the same semantic member comparison as the working diff.

✅ Quality: publicationStatus shape is stated only in the PR description

📄 docs/decisions/2026-10-21-ontology-definition-status-is-derived-from-releases.md:18-20 📄 docs/decisions/2026-10-21-ontology-definition-status-is-derived-from-releases.md:26
The description says fields=publicationStatus "returns its active release, its latest release and that release's stage, and the number of unpublished working changes." The record states only unpublishedChanges. It does not state that publicationStatus also carries the active release, the latest release and that release's stage, or how "latest" is chosen. This is a REST/JSON contract that the UI must agree with, and the squash-merge drops the description. There is a second unrecorded choice: concept lifecycleStatus uses InReview, but the release's stage comes from EntityStatus (type/status.json), which spells it In Review. So one response would carry both spellings. Add a sentence to the Decision like: "publicationStatus carries the active release, the latest release (highest release revision, per ADR:2026-10-19-ontology-releases-allocate-semantic-versions) and that release's EntityStatus stage, plus unpublishedChanges." Also state whether lifecycleStatus is a new enum or reuses EntityStatus values.

✅ Performance: Review list does a full scan with per-release task reads on every page

📄 openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/OntologyReleaseRepository.java:237-251 📄 openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/OntologyReleaseRepository.java:296-306 📄 openmetadata-service/src/main/java/org/openmetadata/service/resources/ontology/OntologyResource.java:339-353 📄 openmetadata-service/src/main/java/org/openmetadata/service/ontology/OntologyReleaseReviewReader.java:53-62
listReviews reads every IN_REVIEW release in the whole deployment on every request, including requests for page 2 and later, because it has to compute total and skip to the offset. For each release it runs a policy authorization (canViewRelease), a full TaskRepository.get with the assignees,reviewers,createdBy relationship fetches, and a SubjectContext lookup. So one call to GET /v1/ontologies/reviews?limit=25 costs several queries per open review across all ontologies, not a bounded number per page. Paging through N open reviews costs O(N²/limit) task reads. The PR description says release reads should stay constant regardless of page size, and this endpoint breaks that. Fix: bulk-load the review tasks for each 1000-row batch (one setFieldsInBulk/getByIds call per batch rather than one get per release), and resolve the caller's SubjectContext once outside the loop. Better still, push the assigned/created filter into SQL, for example by joining on the task's assignee/createdBy relationships, so fewer rows are scanned at all.

Verification

Rules No rules evaluated

Functional validation Not enabled

Set up

Auto-approval Not enabled

Set up

_{For more verification details, comment gitar display:verbose.}

Options

Display: compact → Showing compact verification results.

Comment with these commands to change the behavior for this request:

Compact

gitar display:verbose

_{Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source}

[sonarqubecloud]

Quality Gate passed for 'open-metadata-ui'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[sonarqubecloud]

Quality Gate passed for 'open-metadata-ingestion'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud