Skip to content

Fixes #36714: Project concepts onto bound assets in Data mode - #36724

Merged
harshach merged 331 commits into
mainfrom
harshach/ontology-studio/13-data-mode
Oct 30, 2026
Merged

harshach merged 331 commits into
mainfrom
harshach/ontology-studio/13-data-mode

Conversation

@harshach

@harshach harshach commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

The stack targets release 2.1.0.

PR1 #35990, PR2 #35994, PR3 #35997, PR4 #35995, PR5 #36183, PR6 #36187, PR7 #36188, PR8 #36190, PR9 #36721, PR10 #35991, PR11 #36723, PR12 #35996 and PR15 #35989 are merged.

This PR targets main. PR14 #36725 builds on this branch. PR16 is not published yet.

Current assessment for 727d5a0f91c, recorded 2026-10-30.

Verdict. PASS. The independent review of the last fix round passed at bebd9843d11. c2feefb2bcd merges PR11's reviewed head without conflict, and 727d5a0f91c catches up with main after PR11 landed. PR13's own lines are unchanged.

What moved since the last assessment. gitar-bot's review raised two findings, and both are fixed. gitar-bot now approves with no open thread.

  • The bindings page size and the stale-reload budget had no record. The bindings decision record gains a v6 amendment. Every bindings request asks for 25 rows, and the reason for 25 was not written down when it was chosen, so the amendment states what it bounds. A 412 gets one automatic reload in each load generation. A second 412 shows the stale error with Retry, and a completed selection resets the count. NATIVE_ASSET_PAGE_SIZE cites the record.
  • Some native rows and chrome overrode core component styling. The legend and the AI-mode heading are now core Cards with Card.Content. The logo tile takes the card's surface colour, so in dark mode it no longer shows the page colour. Muted text uses Typography color="secondary". The Load more strip moves its border and padding onto a plain div.
  • What core cannot express stays at the call site under a new exception record, Data mode asset rows are styled at the call site. The styling record gains a v4 amendment that links it, as v2 and v3 did. It covers the full-bleed asset row and card header Buttons and three text colours that TypographyColor lacks. These call sites move to core when core ships a list-row variant and those colours.
  • The legend and logo tile changes also reach the glossary data graph, which shares those components.

PR11 then took its own review fixes and main with #36721 landed, and c2feefb2bcd merges PR11 at 4f9d666e3cf. No file is in both PR11's new changes and PR13's diff. Every PR13 file keeps its stable patch-id. The total changed because PR9's decision record and one comment in useOntologyChangeSetApply.ts moved into the base.

PR11 landed as 5bf55a51aa0, on top of #36726, which changes only backend repositories and their tests. 871d8e90961 merges 9ac5b217711, the commit before PR11's squash, without conflict. 727d5a0f91c then merges main and keeps PR13's tree, because main's tree equals PR11's final head merged with #36726. PR13's stable patch-id is 644efbd4ce8 against main and against PR11's final head, and no UI file differs from c2feefb2bcd.

The reviews declared these minor issues, which do not block.

  • If a card leaves the graph and returns within 100 ms of its last scroll event, a pending settle timer restores its old scroll position. A lineage edge can then leave the wrong row until the card's next scroll. This predates the fourth round, which narrowed it.
  • In AI mode, the top-centre loading status and read-failure alert can draw over the heading card. The status reaches the card on a canvas narrower than about 984 px, and the alert below about 1,340 px. This is inferred from the classes and was not rendered.
  • In AI mode, the opaque heading card can hide the top row of concept cards after the view resets, and clicks pass through it to those cards. This is inferred from the classes and was not rendered.
  • No test drives the AI concept picker's selection in the all-ontologies Data mode. Removing its selection handler fails no test.
  • The bindings decision record's Guard line, on main, still names OntologyConceptDataGraph.test.tsx, which PR13 deletes. The v6 amendment names the suites that now guard it.
  • The new AI-mode heading test checks the card's surface class and text, not its position, so an overlap with the mode tabs needs the live check.
  • Three older overrides in the same module predate PR13 and no record covers them. They are the glossary Load more box, the cluster card's shadow and the semantic-edge badge's colours.

Test evidence. The gates ran at c2feefb2bcd, whose UI tree equals this head's. Jest passes 1,673 of 1,673 tests in 162 suites, covering every Studio suite, the placeholder wrappers and the import guard test. TypeScript reports only main's error at IngestionWorkflowForm.tsx(413,9). ESLint, Prettier and organize-imports leave no change on PR13's UI and Playwright files. The import guard is clean, the Playwright impact map is up to date, and the decision record check is clean. The last fix round's review ran the Data mode suites at bebd9843d11 and passed 93 of 93.

CI. This head was pushed on 2026-10-30, and CI is running. With main as the base, GitHub links Fixes #36714, so Validate PR Metadata and maven-collate-ci can pass.

Still owed. None of these has run yet. Run each IT on -Pmysql-elasticsearch and then -Ppostgres-opensearch, and confirm a Tests run: line.

  • Playwright OntologyNativeDataMode.spec.ts and OntologyNativeOperations.spec.ts against a live stack built from this head. Then the specs that the impact map routes the changed files to, OntologyNativeStudio, OntologyNativeAuthoring, OntologyNativeRelations, OntologyStudioE2E and OntologyNativeTreeShell. They drive classic mode.
  • PR12's data graph ITs, which serve PR13's Data mode. Run NativeOntologyDataGraphPageIT first, then NativeOntologyDataGraphIT, NativeOntologyDataGraphContextIT, NativeOntologyDataGraphBindingScanIT, NativeOntologyDataGraphEdgeScanIT and OntologyConceptBindingsIT. PR13 changes no Java.
  • A live check of the restyled pieces in both themes. The logo tile is white with a gray-200 border in light mode and gray-800 with a faint border in dark mode, on rows and on the metric tile. The legend sits level with the zoom controls in the native and glossary scopes. The AI-mode heading card does not cover the mode tabs or the search, and the pointer passes through to the canvas.
  • A live check with AI mode on, on a canvas about 900 px wide. Trigger a reload and a failed read, and check whether the status and the alert cover the heading card. After a refresh, check the top-left concept cards under the heading.
  • Browser checks that Jest cannot make. Use real momentum scrolling on a card that leaves and returns, Show less with a real scroll, and page zoom at a fractional pixel ratio.
  • The plan's ten live lanes, the performance lane and the review media.

AI mode. main's AI-mode restyle #36433 gates new screens on AI mode. Paths are under openmetadata-ui/src/main/resources/ui/src/components/. No PR13 control is missing from AI mode, because Data mode renders in code that both modes share. main's AI mode tabs now call PR13's changeGraphMode, which mounts the Data surface. main's AI-only Data mode pieces moved to their match in the new code. The backdrop is in governance/ontology/OntologyGraphView/OntologyNativeDataMode.tsx, the controls surface is in OntologyExplorer/OntologyDataGraph.tsx, and the details-card style is on the heading in OntologyExplorer/OntologyDataGraphChrome.tsx. In the all-ontologies view with no concept chosen, AI mode shows main's concept picker and classic mode shows PR13's prompt. The single-ontology concept picker no longer renders, because Data mode now shows every bound concept of the ontology. The possible overlap of the status and the heading card is listed above.

Not verified yet. This PR is verified only when its unit, live and performance boxes in the plan are all checked. The operator lands the stack.

Describe your changes

Fixes #36714

Data mode on a native ontology now shows every bound concept as a card on one graph, fed by PR12's paged native data graph. Before, it showed the assets of one selected concept.

  • Each card shows the concept and the number of assets bound to it that the user can view.
  • Each asset row shows the connector logo, the table name and connector · N cols, and it opens the asset.
  • A card shows about four rows and scrolls inside itself. Load more (N more) fetches the next page with the cluster's cursor, and Show less returns the card to its first rows.
  • A dashed edge with a type chip joins the cards of concepts related by an axiom, and a solid edge shows observed lineage between rows. Lineage between two tables in one card shows as a note on each row. The legend and the scope text explain both edge kinds, and the old text that denied inferred edges is gone.

The graph loads concepts in pages. At 60 concepts the server reports that the view is full, so the graph disables loading more concepts and says that the view holds its maximum. If the bound data changes while a page loads, the graph says so and offers Retry. Data mode reloads after a model write or a binding write, and it stays mounted while Model mode shows, so switching back keeps the selection and the expanded card. The inspector's more assets link opens this view. The all-ontologies view shows the same graph for the chosen concept's ontology.

This is PR13 of the sixteen-part Ontology Studio program for release 2.1.0.

Type of change

  • New feature

High-level design

Of the 50 changed files, 46 are UI files. The others are three decision records and the generated Playwright impact map. PR13 adds no endpoint. useNativeOntologyDataGraph calls POST /api/v1/ontologies/{id}/dataGraph/pages through getNativeOntologyDataGraphPage in rest/ontologyAPI.ts. Each request sends the loaded concept prefixes and asks for one page of clusters, one page of a concept's bindings, or the next topology page. The hook follows a non-null cursor even after a page that permission filtering left empty, because the server bounds each scan and an empty page is not the end. It asks only for the cluster offsets that the server advertised.

OntologyNativeDataGraph.utils.ts turns those pages into the graph model, with its clusters, rows, edge kinds and cursors. OntologyDataGraph.tsx already drew glossary data graphs, and its native variant now draws this model. A card renders only the rows in its scroll window. It moves that window when a scroll crosses a row, and it stores the exact position 100 ms after scrolling stops. When a lineage row leaves a card's view, its edge anchors on the card, and edges between the same two cards merge. The dashed relation edge between those cards then moves once per such crossing, which this PR accepts. PR13 deletes OntologyConceptDataGraph.tsx and its test.

The paging contract, the 60-concept limit and the rule that a reader sees only assets they can view are in the bindings decision record, which PR12 amended. PR13 adds its v6 amendment for the page size and the stale-reload budget. The exception record lists the core styling Data mode overrides until core ships a list-row button and two text colours, and the styling record links it as v4.

Tests

Use cases covered

The tests cover these cases.

  • The graph model built from a fixed data graph, with edge kinds and cursors.
  • Paging clusters and bindings, an empty page with a non-null cursor, a full context, a changed binding revision, and a failed read with Retry.
  • Load more, Show less, logos, the legend, lineage edges and row notes.
  • The scroll window, its settle timer, and a card that leaves and returns.
  • The Data tab and the all-ontologies Data mode in both modes.

Unit tests

New suites cover OntologyNativeDataGraph.utils, useNativeOntologyDataGraph and OntologyNativeDataMode. OntologyDataGraph, the graph view, the explorer page, rest/ontologyAPI and OntologyConceptBindings.utils gain native cases. Mutation runs during review confirmed that the scroll settle and the reconciliation of a returning card each fail a test when changed. See Test evidence above for the gate counts at this head.

Backend and ingestion integration tests

Not applicable. PR13 changes no backend or ingestion code. The ITs behind the endpoint it calls are listed under Still owed.

Playwright and UI recording

playwright/e2e/Features/OntologyNativeDataMode.spec.ts binds 150 tables through the API, pages a cluster, checks that logos are not placeholders, and checks that edges join only related concepts. OntologyNativeOperations.spec.ts follows the new Data view. Neither has run against a stack built from this head. The UI recording and review screenshots are not made yet.

Performance

Not measured yet. The plan's lane times Data mode's first paint for one concept against trunk, the first paint for 20 clusters of 150 assets, and one Load more.

Checklist

  • I have read the CONTRIBUTING document.
  • My PR title is Fixes <issue-number>: <short explanation>.
  • My PR is linked to a GitHub issue via Fixes #36714 above.
  • I have commented on my code, particularly in hard-to-understand areas.
  • JSON Schema changes are not applicable, because PR13 changes no schema.
  • The PR build has not run in Docker with sample data yet, and no verified recording is embedded.
  • I have added tests and listed them above.
  • The final diff passed independent review, and the five minor issues it declared are listed above.
  • PR13 adds the exception record and amends the bindings and styling records, as linked above.

… canvas

Adds the 360 px New class panel (identity, definition, axioms, properties,
governance), a typed single change set built by OntologyNativeChangeSet.utils,
free-slot node placement, cancel-on-source-click for drawn relations, and
grouped class rows in the release panel. Translates the new locale keys.
Unlock the panel and the page after a rejected or applied attempt, and rebuild the
plan from a fresh working revision on the next Create. Restore the guard against
replaying an old Add concept request. Define the missing label keys and translate
them in every locale. Keep the canvas alive when a class is placed, and read and
write node centers consistently. Show the class change row with its local name,
parents and a pluralized property count, name property types, and manage focus.
…ypes

A user whose policy grants Create but not EditAll on relationshipType
could register a primary type and its inverse and then fail on the
back-link patch, which the server authorizes as EditAll. The new tests
expect the dialog to disable its inverse field for that user and the
flow to write a single type, plain or symmetric, with no patch. The
first test fails on the current component.
The inverse's back link is a PATCH of /inverse on the inverse type,
which the server authorizes as EditAll on relationshipType, and the
creator holds no owner right on a type it just made. The create dialog
now disables the inverse field for a user without that right and shows
the standard permission message under it, so a create-only user
registers one type, plain or symmetric, and never leaves a half-linked
pair. The decision record gains a v2 amendment with the rule.
A dry run counted a one-value enumeration, duplicate enum values, and a
one-item property chain as importable, and the draft then failed to apply.
The repository and the preview now read one rule set, so the preview names
the CSV row and column, or the RDF property, before Import is offered.
@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Jest test Coverage

UI tests summary

Lines Statements Branches Functions
Coverage: 80%
80.84% (136279/168573) 67.7% (48843/72144) 70.74% (27501/38872)

The placeholder is absolutely positioned and centred, so options taller than
the viewport spilled past both edges of an overflow-hidden page. The host now
scrolls, and the placeholder flows inside it, centred when it fits.

The icon colour test also inspects the glyph element, so a grey class on the
icon itself fails it.
The placeholder now sits in a centred core Card at the old 340px width, so the
copy no longer runs through the canvas dot grid. The overlay still lets pointer
events through, and only the card takes them.

The test names what it proves: the shared empty placeholder renders inside the
card. The wrapper guard itself is a lint rule, not a DOM assertion.
…mment

The record now says iri and namespace must parse as a URI and that the server
has no uniqueness check on namespace, so the duplicate refusal is client-only.
It no longer claims other code reads the onto/ shape, and the comment above
ontologyIris keeps only the record citation.
The wrapper now maps an optional per-feature action to a title button, which
the record said it never would. Add a v3 revision and a dated amendment, and
leave the Decision text unchanged.
A Load more pressed before the second 412 still sends its one queued request
afterwards, so the record no longer says nothing is sent. It also names the
hook's Generation a load generation, to keep it apart from the server's
topology generation, and says which tests guard the amendment now that
OntologyConceptDataGraph.test.tsx is gone.
…t it proves

The AI-mode heading is a surfaced card and the classic heading is not. The
logo tile test says it checks the card surface colour rather than visibility.
…tion

Core Button sizes set their own radius and padding and TypographyColor has no
primary or secondary-text step, so the full-bleed asset rows and three text
colours stay styled at the call site. A scoped record and a v4 amendment to the
styling decision say so, and the call sites cite it. The asset name keeps its
primary colour because the tertiary button would mute it. The Data layer title
drops its primary colour because the page text is already primary. The Load
more strip puts its border and padding on a plain div instead of a core Box.
The organize-imports and ESLint pass in CI inserts this blank line.
Takes the decision record's last wording fix from PR9's final head, which main now holds as #36721.
main's tree equals the class authoring head merged in the parent commit, so this merge records the squash and keeps the first-run tree as it is.
The glossary empty state now takes its muted colour from Typography's color prop, so the exception record's closing sentence holds for every Typography in Data mode. The bindings amendment names the PR by number and the view test that pins the stale message, and the AI-mode heading test resets the mode in afterEach.
… its scroll host

The placeholder kept flex-shrink 1 and min-h-full replaced its automatic
minimum, so it shrank to the host height and its centred content spilled
above the scroll origin. Use flex-1 with h-auto so it grows past the host
and stays centred when it fits. Put the no-concepts placeholder in
Card.Content so the card supplies the padding.
At 720x450 the headline must sit at or below the scroll host top and
Create from scratch must scroll into view.
…ding

A relative iri also gets 400, iriHash covers iri and not namespace, and
the context no longer says the server stores whatever it is sent. The
v3 amendment follows the v2 heading and lead-line style.
Without tw:h-auto the shell keeps its own h-full, shrinks to the scroll host and clips the headline again, so the scroll host test now asserts it.
Takes PR11 at 4f9d666: the empty states through the placeholder wrappers, the IRI record, the scrollable first-run screen, and main with #36721 landed.
Base automatically changed from harshach/ontology-studio/11-first-run to main October 30, 2026 18:43
Takes #36726, which changes only backend repositories and their tests.
main's tree equals the first-run head this branch already holds, merged with #36726 in the parent commit, so this merge records the squash and keeps the Data mode tree as it is.
@gitar-bot

gitar-bot Bot commented Oct 30, 2026

Copy link
Copy Markdown
Code Review ✅ Approved 2 closed / 2 findings

🟡 Medium risk · Adding a paged native ontology Data graph changes asset and lineage presentation; paging or topology errors could hide assets or misstate relationships.

Adds Data mode to native ontologies, showing every bound concept as a paged card on one graph with asset rows, lineage and relation edges, and Load more/Show less controls. Client page size and stale-retry budget are now recorded, and new native rows and chrome no longer conflict with core component styling. No open issues remain.

✅ 2 closed
✅ Quality: Client page size and stale-retry budget are not in any decision record

📄 openmetadata-ui/src/main/resources/ui/src/utils/governance/ontology/OntologyNativeDataGraph.utils.ts:35-37 📄 openmetadata-ui/src/main/resources/ui/src/hooks/governance/ontology/useNativeOntologyDataGraph.ts:268-282
The bindings decision record (docs/decisions/2026-10-19-native-concept-bindings-remain-operational-metadata.md:56) only gives the server caps: 12 clusters, 4 previews, at most 100 bindings per selection, 60 prefixes. Two values the client picks here are judgment calls that no record states. NATIVE_ASSET_PAGE_SIZE = 25 sets the bindings page size, which the record leaves to the client ("Clients advance through stored ranks by the requested page size"). failAttempt sets a retry budget: after a 412 it reloads once, and the next 412 surfaces as the stale error. The description gives that budget only in prose ("If bound data changes while a page loads, the graph says so and offers Retry"). The record would say: "Native Data mode requests bindings 25 at a time and reloads automatically after one 412 per generation; a second 412 is shown to the reader with Retry." Add that sentence to the existing record or to a new one.

✅ Quality: New native rows and chrome override core component styling

📄 openmetadata-ui/src/main/resources/ui/src/components/OntologyExplorer/OntologyDataGraph.tsx:274-282 📄 openmetadata-ui/src/main/resources/ui/src/components/OntologyExplorer/OntologyDataGraph.tsx:333-337 📄 openmetadata-ui/src/main/resources/ui/src/components/OntologyExplorer/OntologyDataGraphChrome.tsx:55-58 📄 openmetadata-ui/src/main/resources/ui/src/components/OntologyExplorer/OntologyDataGraphChrome.tsx:97-99
ADR:2026-10-11-migrated-antd-components-adopt-core-styling says callers must not set padding, radius, shadow or background on core components, and Box is layout only. The new code breaks this in several places. The asset-row Button sets tw:rounded-none tw:px-5 tw:py-3. DataEdgeLegend's Card sets tw:px-3 tw:py-2 on the root. DataLayerHeading in AI mode gives a Box tw:rounded-xl tw:border tw:bg-surface tw:p-3 tw:shadow-xs. AssetLogo gives a Box tw:bg-primary, which is the page colour in dark mode, so the tile disappears on a dark card. Typography colour also comes from tw:text-* classes instead of the color prop. Fixes: use Card size with Card.Content for padding, use Card instead of a styled Box for the heading panel, use tw:bg-surface/Card for the logo tile, and add a core list-row variant (or use the existing one) instead of overriding Button.

Verification

Rules No rules evaluated

Functional validation Not enabled

Set up

Auto-approval Not enabled

Set up

For more verification details, comment gitar display:verbose.

Options

Display: compact → Showing compact verification results.

Comment with these commands to change the behavior for this request:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source

@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

⚠️ UI Checkstyle passed — lint findings in changed files

🔍 ESLint findings in this PR's files — 0 error(s), 8 warning(s)

Errors block the build. Warnings do not yet — they are rules whose backlog is still
being worked down, listed so this PR does not add to it. See docs/ui-code-quality-gate.md.

0 error(s), 8 warning(s) across 4 changed file(s).

Count Rule
7 openmetadata-imports/review-sequential-api-calls
1 openmetadata-imports/no-api-calls-in-iteration
All findings
Location Rule Message
🟡 src/components/governance/ontology/OntologyGraphView/OntologyGraphView.tsx:573:9 openmetadata-imports/review-sequential-api-calls Review these sequential API requests. If they are independent, start them together with Promise.all/Promise.allSettled; keep sequencing only when data-dependent
🟡 src/hooks/governance/ontology/useOntologyConceptBindings.ts:157:26 openmetadata-imports/review-sequential-api-calls Review these sequential API requests. If they are independent, start them together with Promise.all/Promise.allSettled; keep sequencing only when data-dependent
🟡 src/rest/ontologyAPI.test.ts:253:12 openmetadata-imports/review-sequential-api-calls Review these sequential API requests. If they are independent, start them together with Promise.all/Promise.allSettled; keep sequencing only when data-dependent
🟡 src/rest/ontologyAPI.test.ts:261:12 openmetadata-imports/review-sequential-api-calls Review these sequential API requests. If they are independent, start them together with Promise.all/Promise.allSettled; keep sequencing only when data-dependent
🟡 src/rest/ontologyAPI.test.ts:292:12 openmetadata-imports/review-sequential-api-calls Review these sequential API requests. If they are independent, start them together with Promise.all/Promise.allSettled; keep sequencing only when data-dependent
🟡 src/rest/ontologyAPI.test.ts:314:5 openmetadata-imports/review-sequential-api-calls Review these sequential API requests. If they are independent, start them together with Promise.all/Promise.allSettled; keep sequencing only when data-dependent
🟡 src/rest/ontologyAPI.test.ts:450:24 openmetadata-imports/review-sequential-api-calls Review these sequential API requests. If they are independent, start them together with Promise.all/Promise.allSettled; keep sequencing only when data-dependent
🟡 src/rest/ontologyAPI.ts:129:28 openmetadata-imports/no-api-calls-in-iteration Avoid issuing one API request per item. Fetch at the data owner, use a bulk endpoint, or use useQueries with an intentional concurrency policy.

Fix locally (fast - only checks files changed in this branch):

make ui-checkstyle-changed

@sonarqubecloud

Copy link
Copy Markdown

@harshach
harshach merged commit d07d717 into main Oct 30, 2026
73 checks passed
@harshach
harshach deleted the harshach/ontology-studio/13-data-mode branch October 30, 2026 19:18
harshach added a commit that referenced this pull request Oct 30, 2026
Takes #36179 and #36727, which change JWT identity, MCP server and change request code and their tests. None of their files is in this branch's diff.
harshach added a commit that referenced this pull request Oct 30, 2026
main's tree equals the Data mode head this branch already holds, merged with #36179 and #36727 in the parent commit, so this merge records the squash and keeps the live data tree as it is.
harshach added a commit that referenced this pull request Oct 30, 2026
Takes 5fd577e, GitHub's merge of main after #36724 landed, and d81b8a0, the bot's impact map refresh. None of their files is in the last three commits.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend safe to test Add this label to run secure Github workflows on PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ontology Studio: project concepts onto bound assets in Data mode

1 participant