Skip to content

Vitest browser-mode UI integration tests (Pagination migration, marked) + CI - #35623

Draft
chirag-madlani wants to merge 11 commits into
mainfrom
poc/ui-integration-tests
Draft

chirag-madlani wants to merge 11 commits into
mainfrom
poc/ui-integration-tests

Conversation

@chirag-madlani

@chirag-madlani chirag-madlani commented Oct 16, 2026 •

Copy link
Copy Markdown
Collaborator

Describe your changes:

Draft, base of #36198. This adds UI integration tests in Vitest Browser Mode, with MSW mocking every /api call, to take over page-behaviour checks from heavy Playwright E2E specs. The layering rule is recorded in ADR:2026-10-28-page-behaviour-is-tested-in-vitest-browser-mode-and-e2e-tests-leave-by-marking.

What's here:

  • Harness: Vitest Browser Mode + MSW under src/test/browser/ (renderWithApp / renderApp, typed fixtures, paginationContract.ts). Any unmocked /api/* call fails the test, so no test can pass against a request that failed silently.
  • Migration: 35 of the 37 tests in playwright/e2e/Features/Pagination.spec.ts are now covered by 22 *.browser.test.tsx files (70 tests + 4 test.fails that pin real bugs). The Playwright tests are not deleted. Each one is tagged @migrated-browser with its replacement (playwright/constant/migration.ts) and keeps running until the browser tests are validated and PaginationIT covers every list endpoint. The mapping is in src/test/browser/PAGINATION_MIGRATION.md.
  • Harness fix: setup.ts now unmounts before resetting MSW handlers. vitest-browser-react only unmounts in the next test's beforeEach, so a still-mounted page refetched against reset handlers and threw unhandled errors under load.
  • Coverage: reportOnFailure: true, plus a json-summary reporter. The browser lcov is added to sonar.typescript.lcov.reportPaths next to Jest's.
  • CI, .github/workflows/ui-browser-tests.yml (new file; no existing workflow is changed):
    • runs only when the UI, the core components, or the workflow and its setup action change;
    • browser tests + coverage posts a Jest-style coverage + JUnit comment via MishaKav/jest-coverage-comment (same-repo PRs only) and uploads screenshots on failure;
    • no retries. The shuffled stability runs used while proving the suite were removed once they showed no flakes.

Type of change:

  • Improvement

High-level design:

  • Layering: pure logic stays in Jest. Page and feature behaviour runs in real Chromium with the real components, router, react-query, i18n and styles; only the network is faked (MSW). API contracts belong in Java ITs. Playwright keeps cross-feature journeys against a real server. See src/test/browser/README.md.
  • Speed so far: from main nightly timing artifacts, Pagination.spec.ts costs 274–495 s of CI worker time, plus the shared stack, build and fixture pipeline. Locally, on a heavily loaded M1 Pro (load 35–60 on 8 cores), the browser suite runs in ~2m14s without coverage, with no backend at all. This PR measures it on a CI runner.
  • Known gap before deleting the marked E2E tests: the cursor walk in PaginationIT is called only for tables. It should cover every list endpoint, so the server side of what moved here stays tested against a real DB.

Tests:

Use cases covered

Pagination across 22 list pages: Next/Previous with cursors and URL sync, deep links, reload, page-size change, search resetting to page 1, empty states, the deleted toggle, the Files ↔ Spreadsheets tab reset, client-side paging, and a visual snapshot of the schema table.

Unit tests

N/A: the change is test infrastructure. Jest is unaffected (*.browser.test.tsx is excluded from Jest).

Backend integration tests

  • Not applicable (no backend API changes).

Ingestion integration tests

  • Not applicable (no ingestion changes).

Playwright (UI) tests

  • Pagination.spec.ts: 35 tests are tagged @migrated-browser, and the 2 real-server smoke tests are untagged.

Manual testing performed

On macOS (M1 Pro):

  1. yarn test:ui:run --maxWorkers=3: 70 passed + 4 expected fail.
  2. yarn test:ui:coverage --maxWorkers=3: lcov.info and coverage-summary.json written even when a test fails.
  3. Under heavy CPU load the only failures were 15 s testTimeouts, a different test each run.
  4. Jest still skips the browser files.

UI screen recording:

Not applicable: test and CI infrastructure only, no product UI change (one ObservabilityAlertsTable row-header a11y fix).

Checklist:

  • I have read the CONTRIBUTING document.
  • My PR title is Fixes <issue-number>: <short explanation>: draft POC, no issue yet.
  • My PR is linked to a GitHub issue via Fixes #<issue-number> above: draft POC.
  • I have commented on my code, particularly in hard-to-understand areas.
  • I have added tests (unit / integration / Playwright as applicable) and listed them above.
  • Decision check: an ADR will be added if the approach is adopted.

🤖 Generated with Claude Code

…lure

vitest-browser-react unmounts in the next test's beforeEach, so the global
afterEach reset handlers and the URL under a still-mounted page, which then
refetched and got wrong-shaped responses (unhandled paging/permission errors).
Also write lcov when a test fails so one flake can't drop Sonar coverage.
…nd stability runs

Runs the browser suite with istanbul coverage and posts a Jest-style coverage
and JUnit comment, plus five shuffled no-retry runs whose failures are
summarized in a second comment. Drops the duplicate patch file.
@github-actions github-actions Bot added safe to test Add this label to run secure Github workflows on PRs UI UI specific issues labels Oct 16, 2026
Comment thread openmetadata-ui/src/main/resources/ui/playwright/e2e/Features/Pagination.spec.ts Outdated
Comment thread openmetadata-ui/src/main/resources/ui/src/test/browser/README.md
Comment thread .github/workflows/ui-browser-tests.yml Outdated
Comment thread .github/workflows/ui-browser-tests.yml
@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

UI browser tests: stability (5 shuffled runs, no retries)

Run Wall Passed Failed
1 190s 74 0
2 203s 74 0
3 147s 74 0
4 194s 74 0
5 202s 74 0

No test failed in any run.

@github-actions

github-actions Bot commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

UI browser test coverage

UI browser tests summary

Lines Statements Branches Functions
Browser coverage: 32%
32.62% (13802/42307) 16.03% (4070/25376) 20.36% (2504/12297)

UI browser tests (Vitest)

Tests Skipped Failures Errors Time
74 0 💤 0 ❌ 0 🔥 7m 48s ⏱️

…s on failure

main now calls /metadata/types/workflows/approvalEnforcedEntityTypes and
/changeRequests from entity pages, which the strict unmocked-API check caught.
The workflow now records wall time and uploads screenshots even when tests fail.
chirag-madlani and others added 6 commits October 26, 2026 14:57
# Conflicts:
#	openmetadata-ui/src/main/resources/ui/playwright/e2e/Features/Pagination.spec.ts
#	openmetadata-ui/src/main/resources/ui/yarn.lock
main removed it with the antd Form/Select/Input migration (#35509); the app
no longer mounts it.
# Conflicts:
#	openmetadata-ui/src/main/resources/ui/yarn.lock
The shuffled no-retry runs showed the suite stable; the workflow now runs
the browser tests once, with coverage, on UI and core-component changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…owser

The 35 Pagination E2E tests covered by browser tests keep running, tagged
with their replacement, until the browser tests are validated and
PaginationIT walks cursors for every listed entity type. Records the
layering rule as a decision.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Skips the coverage comment on fork PRs and lets it fail without failing
the job; formats three browser-test files.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gitar-bot

gitar-bot Bot commented Oct 28, 2026

Copy link
Copy Markdown
Code Review ✅ Approved 4 closed / 4 findings

🟡 Medium risk · Migrating pagination coverage to MSW-backed browser tests while retaining only two Playwright smoke tests could leave real-server cursor regressions undetected.

Vitest Browser Mode and MSW harness migrates pagination UI behaviour out of heavy Playwright specs, with a new browser-test CI workflow. Resolved: the stability run no longer loses wall time or counts when a test fails, and the E2E pagination specs are restored and tagged @migrated-browser rather than deleted, pending a PaginationIT cursor walk across all listed entity types. The test-layering decision is now recorded as an ADR cited from the browser test README. No open issues remain.

✅ 4 closed
✅ Bug: Stability run loses wall time and counts whenever a test fails

📄 .github/workflows/ui-browser-tests.yml:42-45 📄 .github/workflows/ui-browser-tests.yml:88-92 📄 .github/workflows/ui-browser-tests.yml:121-126
GitHub runs run: scripts with bash -e. When yarn test:ui:run exits non-zero, the script stops before the line that writes timing.json. The report's jq -rs ... results.json timing.json then fails, and the row prints n/a | no results instead of the run's pass/fail counts and wall time. So any run with a failing test, which is exactly the run you want to inspect, shows no numbers. The same thing happens in browser-tests: the seconds output is never written on failure. Capture the exit status, write the timing file, then exit with that status.

✅ Quality: E2E pagination deleted while real-server cursor paging stays untested

📄 openmetadata-ui/src/main/resources/ui/playwright/e2e/Features/Pagination.spec.ts:28-38 📄 openmetadata-ui/src/main/resources/ui/src/test/browser/PAGINATION_MIGRATION.md:37-42
This PR cuts 35 Playwright tests that paged through ~18 list endpoints (roles, policies, bots, alerts, data models, stored procedures, drive files, etc.) against a real backend. Their replacements are MSW tests, and those can only check that the UI agrees with fixtures/paging.ts#cursorPage, which the test author wrote. They cannot check what the server actually returns. The PR's own PAGINATION_MIGRATION.md (the * gap note) says the server-side replacement isn't there yet, and the repo confirms it: testPaginationForEntityType is only called for "table" (PaginationIT.java:83-85), and BaseEntityIT only checks that a list is non-empty. So once this merges, a cursor regression on any of those endpoints, such as a wrong before cursor or a duplicated last row, would pass every suite. The description puts this under "Known gap before deleting more E2E", but this PR already does the deleting. Either keep the Playwright cases until the IT cursor walk covers each listed entity type, or add that @ParameterizedTest in this PR.

✅ Quality: Test-layering decision is not in docs/decisions/

📄 openmetadata-ui/src/main/resources/ui/src/test/browser/README.md:13-20 📄 openmetadata-ui/src/main/resources/ui/sonar-project.properties:12 📄 openmetadata-ui/src/main/resources/ui/playwright/e2e/Features/Pagination.spec.ts:28-37
This PR makes a lasting rule and records it only in the PR body and in src/test/browser/README.md. No record in docs/decisions/ states it. A record would say something like: "Per-page list UI behaviour (cursor/URL sync, reload, page size, search reset, deleted toggle) is tested in Vitest browser mode with MSW (*.browser.test.tsx). Playwright keeps only real-server smoke tests for it, and every unmocked /api/* call fails a browser test." That rule comes from the description's "Layering: pure logic stays in Jest. Page and feature behaviour runs in real Chromium … Playwright keeps cross-feature journeys against a real server." Separately, merging the browser lcov into the Sonar gate (sonar.typescript.lcov.reportPaths) changes how the coverage gate is computed, which is a second decision. The description says an ADR will follow if the approach is adopted. It needs to land in this PR, because this PR already deletes the E2E tests and changes the Sonar inputs.

✅ Bug: Comment step failure skips failure-screenshot upload; forks fail

📄 .github/workflows/ui-browser-tests.yml:46-60 📄 .github/workflows/ui-browser-tests.yml:140-141
The steps Upload failure screenshots and Fail if tests failed use if: steps.tests.outcome == 'failure' with no status function, so GitHub treats them as success() && .... If Coverage comment fails, both steps are skipped. That happens when coverage-summary.json is missing because vitest crashed, or on a fork PR, where GITHUB_TOKEN is read-only and pull-requests: write is not granted. Failure screenshots are then never uploaded. On fork PRs the job and the stability report comment step also go red even when every test passes. Fix: add always() to the failure-handling steps, and skip comment posting for fork PRs.

Verification

Rules No rules evaluated

Functional validation Not enabled

Set up

Auto-approval Not enabled

Set up

For more verification details, comment gitar display:verbose.

Options

Display: compact → Showing compact verification results.

Comment with these commands to change the behavior for this request:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Powered by Gitar — free for open source

@chirag-madlani chirag-madlani changed the title POC: Vitest browser-mode UI integration tests (migrate Pagination.spec.ts) + CI Vitest browser-mode UI integration tests (Pagination migration, marked) + CI Oct 28, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

safe to test Add this label to run secure Github workflows on PRs UI UI specific issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant