Skip to content

feat(dashboards): support constant values and render modes for filters#2383

Open
alex-fedotyev wants to merge 7 commits into
mainfrom
alex/HDX-4404-dashboard-filter-modes
Open

feat(dashboards): support constant values and render modes for filters#2383
alex-fedotyev wants to merge 7 commits into
mainfrom
alex/HDX-4404-dashboard-filter-modes

Conversation

@alex-fedotyev
Copy link
Copy Markdown
Contributor

@alex-fedotyev alex-fedotyev commented May 30, 2026

Linear: HDX-4404.

Summary

  • Adds two fields to DashboardFilterSchema: constant: boolean (locks the saved default; the viewer cannot clear it) and renderMode: 'editable' | 'readonly' | 'hidden' (controls how the chip appears in the filter bar).
  • One dashboard can be cloned and re-pointed by saving a different default value per copy, instead of hand-coding the scope into every tile's WHERE.
  • Filter editor UI presents a single "Visibility" select with three presets (Editable / Read-only / Hidden) that map to the orthogonal (constant, renderMode) pair. The schema still admits all combinations so MCP and external API callers can express future variants.
  • When Visibility is Read-only or Hidden, the editor exposes a "Default value" TagsInput so the author can set the locked value directly. Editable filters keep using the existing chip + Save default flow unchanged.
  • External API FilterInput / Filter OpenAPI schemas and the MCP mcpDashboardFilterSchema carry both new fields; openapi.json regenerated.
  • Hook (useDashboardFilters) accepts savedFilterValues and overlays constant filter values on every read, regardless of URL state; setFilterValue is a no-op for constant expressions. Constant values are kept out of the URL on hydration so the lock doesn't leak into shared links, and handleSaveQuery preserves constant entries when saving so the editable Save default flow doesn't clobber them.
  • Out of scope (tracked as follow-ups in the Linear issue): URL-parameterized constants (?scope=...), inline operator/values on the filter definition, customer-facing docs in clickhouse-docs.
image

Test plan

  • Hook unit tests: constant value injection, setFilterValue no-op, sibling editable filters still work, getFilterQueriesForSource honors appliesToSourceIds for constant filters, hidden + constant resolves the saved value, missing saved value returns nothing.
  • Zod schema tests: accepts constant + renderMode in every valid combination, rejects unknown renderMode, rejects null for either field.
  • External API round-trip integration test: POST + GET + PUT with constant and renderMode set; absent fields stay absent on read; unknown renderMode returns 400.
  • MCP hyperdx_save_dashboard round-trip integration test: same coverage as the external API test.
  • UI smoke (Playwright + manual screenshots, light + dark):
    • Edit a filter, switch Visibility to Read-only, type a value into Default value, save. Verify the chip shows disabled with a lock icon and tiles include the locked value in WHERE.
    • Switch Visibility to Hidden. Verify the chip is removed from the bar; tiles still include the locked value.
    • Clear the Default value (TagsInput empty), save. Verify tiles are no longer scoped on that expression.
    • Hard-reload with an empty URL. Verify the locked value still applies (sourced from savedFilterValues).
    • Open a shared link with a stale value for the locked expression. Verify the locked value still wins.
  • Predicted tier: review/tier-4 (touches packages/api/src/routers/external-api/v2/dashboards.ts, expected per the implementation plan).

Tagging as draft so the UI smoke can be done against this branch's build.

@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented May 30, 2026

🦋 Changeset detected

Latest commit: d27e41e

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages
Name Type
@hyperdx/common-utils Minor
@hyperdx/api Minor
@hyperdx/app Minor
@hyperdx/otel-collector Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@vercel
Copy link
Copy Markdown

vercel Bot commented May 30, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hyperdx-oss Ready Ready Preview, Comment Jun 1, 2026 4:02pm
hyperdx-storybook Ready Ready Preview, Comment Jun 1, 2026 4:02pm

Request Review

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 30, 2026

E2E Test Results

All tests passed • 192 passed • 3 skipped • 1330s

Status Count
✅ Passed 192
❌ Failed 0
⚠️ Flaky 3
⏭️ Skipped 3

Tests ran across 4 shards in parallel.

View full report →

@alex-fedotyev alex-fedotyev force-pushed the alex/HDX-4404-dashboard-filter-modes branch from 1572113 to 6946b0a Compare May 30, 2026 00:54
@alex-fedotyev alex-fedotyev marked this pull request as ready for review May 30, 2026 01:01
@github-actions github-actions Bot added the review/tier-4 Critical — deep review + domain expert sign-off label May 30, 2026
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 30, 2026

🔴 Tier 4 — Critical

Touches auth, data models, config, tasks, OTel pipeline, ClickHouse, or CI/CD.

Why this tier:

  • Critical-path files (2):
    • packages/api/src/routers/external-api/v2/dashboards.ts
    • packages/api/src/routers/external-api/v2/utils/dashboards.ts
  • Cross-layer change: touches frontend (packages/app) + backend (packages/api) + shared utils (packages/common-utils)

Review process: Deep review from a domain expert. Synchronous walkthrough may be required.
SLA: Schedule synchronous review within 2 business days.

Stats
  • Production files changed: 16
  • Production lines changed: 1330 (+ 1255 in test files, excluded from tier calculation)
  • Branch: alex/HDX-4404-dashboard-filter-modes
  • Author: alex-fedotyev

To override this classification, remove the review/tier-4 label and apply a different review/tier-* label. Manual overrides are preserved on subsequent pushes.

@github-actions
Copy link
Copy Markdown
Contributor

<!-- deep-review -->

Deep Review

🔴 P0/P1 -- must fix

  • packages/api/src/mcp/tools/dashboards/saveDashboard.ts:53 -- hyperdx_save_dashboard's MCP inputSchema exposes id/name/tiles/tags/containers/filters but not savedFilterValues, while mcpDashboardFilterSchema tells the agent that constant: true takes its locked value from dashboard.savedFilterValues; the safeParse payloads at :161 and :354 therefore never receive savedFilterValues, so a constant filter created via MCP locks to nothing and scopes no tile.
    • Fix: Add savedFilterValues: z.array(z.object({ type: z.enum(['lucene','sql']), condition: z.string() })).optional() to the tool inputSchema, thread it through to the create/update body schemas, and add an example pairing constant: true with a matching entry to mcpFiltersParam.
    • agent-native

🟡 P2 -- recommended

  • packages/app/src/DBDashboardPage.tsx:1445 -- savedFilterValues is keyed by filter expression, so when two filter definitions share an expression (one constant, one editable) getDefaultValuesForExpression seeds the editable form with the constant's locked value; if the author clears the picker and saves the editable, handleSaveFilter deletes the entry for that expression and silently breaks the sibling constant's lock.

    • Fix: Key savedFilterValues by filter id (and update the hook overlay + URL strip + save preservation to match), or reject sibling definitions where one is constant and one is editable on the same expression.
    • adversarial, correctness
  • packages/app/src/DashboardFiltersModal.tsx:304 -- DashboardFilterEditForm recomputes initialDefaultValues whenever the prop savedFilterValues changes and its useEffect([filter, initialDefaultValues, reset]) calls reset(...) on the change, so a background refetch that lands while the modal is open wipes any in-progress edit to the defaultValues field.

    • Fix: Snapshot initialDefaultValues once via a ref (or gate the reset on !formState.dirtyFields.defaultValues) so subsequent savedFilterValues arrivals don't clobber the author's typed values.
    • reliability, julik-frontend-races
  • packages/app/src/hooks/useDashboardFilters.tsx:102 -- inside the iteration, valuesForExistingFilters[expression] = match is keyed by raw expression, so two filter definitions sharing one expression with mixed constant/editable last-writer-wins: the editable's URL value overwrites the constant's locked value, breaking the lock guarantee for the in-scope tile.

    • Fix: Resolve values per-definition (e.g. key by filter.id and merge in getFilterQueriesForSource), or refuse to overwrite a key already populated by a constant entry.
    • adversarial, correctness
  • packages/api/openapi.json:2366 -- the new property schemas declare "default": false / "default": "editable" (mirrored in the swagger JSDoc at packages/api/src/routers/external-api/v2/dashboards.ts:1382), but the Zod schema is .optional() with no .default() and the new API test explicitly asserts the omitted fields return undefined; generated SDKs will surface a default the wire format never produces.

    • Fix: Drop the default keys from both property schemas in openapi.json and the JSDoc; describe the implied default in the description text instead.
    • api-contract
  • packages/app/src/DBDashboardPage.tsx:1669 -- the new constant-preservation merge in handleSaveQuery, the URL-strip block at :1581, and the savedFilterValues upsert in handleSaveFilter (:1445) are all non-trivial and entirely uncovered; future edits to any of the three inline implementations can silently regress the "Save default doesn't clobber constants" contract.

    • Fix: Extract each merge into a pure helper (mergeConstantFiltersForSave, stripConstantsFromUrl, upsertSavedDefault) and unit-test the matrix: constants present + URL collisions, no constants, constants without saved entries, bracket-notation expressions.
    • testing, maintainability
  • packages/app/src/hooks/__tests__/useDashboardFilters.test.tsx:355 -- every new constant-filter test uses simple dot-notation expressions (environment, service.name); bracket-notation (SpanAttributes['k8s.pod.name']) flows through normalizeKey in the hook overlay, the URL strip, save preservation, and getDefaultValuesForExpression in the modal but has zero coverage.

    • Fix: Add a constant: true test with a bracket-notation expression and assert the locked value resolves through filterValues, blocks setFilterValue, and survives handleSaveQuery.
    • testing, correctness
🔵 P3 nitpicks (12)
  • packages/app/src/DBDashboardPage.tsx:1583 -- constantExpressions is rebuilt inline in the URL-hydration effect, again in handleSaveQuery, and a third time as a memo inside useDashboardFilters, each repeating parseKeyPath(...).join('.').

    • Fix: Promote normalizeExpression(k) and buildConstantExpressionSet(filters) into a shared module and call from all three sites.
    • maintainability, correctness
  • packages/common-utils/src/types.ts:1162 -- the new DashboardFilterRenderMode Zod enum is exported but every consumer in the app package uses the literal strings 'editable' / 'readonly' / 'hidden', including a hand-rolled FilterVisibility union and a getFilterVisibility parameter type.

    • Fix: Import the enum (or z.infer<> it into a type) and replace the magic strings so a future rename breaks the type-checker.
    • maintainability, kieran-typescript
  • packages/app/src/DashboardFiltersModal.tsx:87 -- applyFilterVisibility('editable') returns { constant: undefined, renderMode: undefined }; spreading that into the saved filter leaves the keys present (with undefined values) in memory, so structural equality against a JSON-roundtripped server copy sees a spurious difference even though the wire format is identical.

    • Fix: Return {} for the editable case so the keys are genuinely absent in memory.
    • kieran-typescript
  • packages/app/src/DashboardFiltersModal.tsx:412 -- the defaultsChanged comparison runs sanitizedDefaults.some((v, i) => v !== initialNormalized[i]), so reordering the same set fires a spurious save write.

    • Fix: Replace the positional compare with a set-equality check.
  • packages/app/src/DashboardFiltersModal.tsx:195 -- NEVER_USED_RANGE is wrapped in useMemo with an empty deps array inside the component.

    • Fix: Hoist it to module scope as a plain const.
  • packages/app/src/DashboardFiltersModal.tsx:188 -- the as DashboardFilter cast on filtersForQuery is redundant; the Pick<> type is already structurally assignable because the omitted fields are optional, so the cast just hides whether the assignment is sound.

    • Fix: Drop the cast (or widen the prop type) so the type relationship is explicit.
  • packages/app/src/hooks/usePresetDashboardFilters.tsx -- the preset handleSaveFilter keeps its single-argument signature while the modal's onSaveFilter prop now accepts an optional { defaultValues } second arg; safe today because supportsConstantFilters=false on presets, but it silently drops the arg if that flag is ever flipped.

    • Fix: Widen the signature to (filter, _options?: { defaultValues?: string[] }) => void.
  • packages/app/src/DBDashboardPage.tsx:170 -- the new parseQuery import uses a relative ./searchFilters path while every other call site in the package uses the @/searchFilters alias.

    • Fix: Switch to the alias for consistency.
  • packages/app/src/DBDashboardPage.tsx:1622 -- dashboard?.filters was added to the hydration effect's deps but no body code closes over it; the initializedDashboard.current === dashboard.id guard already protects against re-runs, so the extra dep only widens the re-execution surface.

    • Fix: Drop dashboard?.filters from the dep array and add a one-line eslint-disable explaining why.
    • reliability, julik-frontend-races
  • packages/app/src/DBDashboardPage.tsx:1470 -- handleRemoveFilter deletes a filter from dashboard.filters but leaves any savedFilterValues entry keyed by that filter's expression; recreating a filter on the same expression later silently re-locks to the orphaned value.

    • Fix: Strip the matching savedFilterValues entry alongside the filter delete.
    • correctness
  • packages/common-utils/src/types.ts:1183 -- the Zod schema admits renderMode: 'hidden' (or 'readonly') with constant: false; that combination produces a chip-less filter that never applies (no constant overlay) and is reachable from external API and MCP callers.

    • Fix: Either add a Zod refine rejecting non-editable renderMode without constant: true, or document the combo as a no-op in the field description.
    • correctness
  • packages/api/src/routers/external-api/__tests__/dashboards.test.ts:961 -- the HDX-4404 round-trip test uses toMatchObject on the filter payload, which lets unexpected extra fields slip through silently.

    • Fix: Use toEqual on the filter objects (or assert the full expected key set) for both POST and PUT responses.
    • testing

Reviewers (11): correctness, testing, maintainability, project-standards, security, learnings-researcher, api-contract, reliability, adversarial, kieran-typescript, julik-frontend-races, agent-native.

Testing gaps:

  • Page-level handleSaveQuery, handleSaveFilter, and URL-strip-on-hydration paths in DBDashboardPage.tsx have no direct coverage.
  • Bracket-notation expressions are unverified across the hook overlay, URL strip, and save preservation paths.
  • No component test exercises DashboardFiltersModal's Visibility select or the default-value picker (the filter-visibility-select / filter-default-values-input testids are unused).
  • No test asserts that DashboardFilters actually omits chips for renderMode: 'hidden' from the DOM.
  • No test covers two filter definitions sharing an expression with mixed constant/editable flags.
  • No Playwright/E2E coverage for the locked-filter authoring flow.

alex-fedotyev added a commit that referenced this pull request May 30, 2026
E2E shard 2 was failing on three pre-existing dashboard tests because
`visibleFilters` in DashboardFilters.tsx was a new array reference every
render, churning useDashboardFilterValues' useQueries and leaving the
chip dropdown disabled while the values query never settled. Memoize
the array so a stable reference flows into the hook.

Deep review fixes:

- P0/P1: MCP hyperdx_save_dashboard inputSchema now exposes
  savedFilterValues, paired with a usage example in the filters param
  description so constant: true filters can actually lock to a value
  via MCP. The body schema already accepts it; only the tool input
  shape was missing.

- P2: DashboardFiltersModal snapshots initialDefaultValues via state
  keyed on filter.id, so a background savedFilterValues refetch no
  longer wipes in-progress edits in the default-value picker.

- P2: Schema-level coherence checks moved to common-utils via two
  reusable refinements applied at the external API boundary:
  refineDashboardFilterCoherence rejects readonly/hidden render modes
  without constant: true; refineDashboardFiltersConstantSiblings
  rejects two filter definitions sharing an expression that disagree
  on constant. Internal callers (MCP, future migrations) get the
  same protection through the shared schemas.

- P2: openapi.json + JSDoc no longer declare default: false /
  default: "editable" on the new constant / renderMode property
  schemas. The Zod schema is .optional() with no .default() so
  generated SDKs would otherwise surface a default the wire format
  never produces; the implied default now lives in the description
  text instead.

- P2: page-level merge logic for "Save default" / URL hydration /
  savedFilterValues upsert is extracted into dashboardFilterUtils
  (normalizeExpression, buildConstantExpressionSet,
  stripConstantsFromUrl, mergeConstantFiltersForSave,
  upsertSavedDefault, removeSavedDefaultForExpression). The matrix
  including bracket-notation expressions is covered by a new unit
  test. useDashboardFilters and DBDashboardPage call the helpers.

- P2: bracket-notation constant filter is now covered in the
  hook unit tests (SpanAttributes['k8s.pod.name'] resolves through
  filterValues, blocks setFilterValue, and survives via
  getFilterQueriesForSource).

P3 polish:

- handleRemoveFilter now strips the matching savedFilterValues entry
  alongside the filter delete so deleting + recreating a filter on the
  same expression doesn't silently re-lock to an orphaned value.
- applyFilterVisibility('editable') returns {} so the spread on save
  no longer leaves constant: undefined / renderMode: undefined keys
  in memory.
- defaultsChanged uses set equality so reordering the same values
  doesn't fire a spurious save write.
- NEVER_USED_RANGE hoisted to module scope.
- FilterVisibility uses z.infer<typeof DashboardFilterRenderMode>
  instead of a hand-rolled string union.
- as DashboardFilter cast dropped on filtersForQuery.
- usePresetDashboardFilters.handleSaveFilter widened to accept the
  optional defaultValues arg so a future supportsConstantFilters=true
  flip on presets doesn't silently drop the second arg.
- parseQuery import uses the @/searchFilters alias for consistency
  with sibling files.
- dashboard?.filters dropped from the URL hydration effect's deps
  (the initializedDashboard.current === dashboard.id guard already
  prevents re-runs).
- HDX-4404 API round-trip test uses toEqual instead of toMatchObject
  so unexpected extra fields fail the assertion.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@alex-fedotyev
Copy link
Copy Markdown
Contributor Author

Pushed 21880e4 with the deep-review fix-pack plus the e2e shard-2 root cause. Summary:

E2E shard 2 (3 failing tests)

Three pre-existing dashboard tests (should create and populate filters, should scope a filter to a specific source via "Applies to sources", should save and restore query and filter values) failed because visibleFilters in DashboardFilters.tsx was a new array reference every render, so useDashboardFilterValues's useQueries churned and the chip dropdown stayed disabled while the values query never settled. Memoizing the array fixes the regression. The Playwright error-context.md snapshot confirmed the failure mode (both filter chips rendered [disabled] even though neither had constant / renderMode set, and the underlying table tile had loaded its data).

Deep review

  • P0/P1 saveDashboard.ts:53 MCP inputSchema missing savedFilterValues. Fixed in 21880e4 — threaded savedFilterValues through the tool input, the create/update helpers, and added an example pairing constant: true with a savedFilterValues entry in mcpFiltersParam.
  • P2 DashboardFiltersModal.tsx:304 reset-wipes-edits race. Fixed in 21880e4 — snapshot initialDefaultValues via state keyed on filter.id; a background savedFilterValues refetch no longer fires the reset.
  • P2 useDashboardFilters.tsx:102 + DBDashboardPage.tsx:1445 sibling-expression collision. Fixed in 21880e4 — added a Zod refinement at the boundary (refineDashboardFiltersConstantSiblings) that rejects two filter definitions sharing a normalized expression where one has constant: true and another doesn't. The id-keyed wire-format refactor (the other option you suggested) is intentionally out of scope; I'll file a follow-up for that since it's a wire-format change.
  • P2 openapi.json:2366 defaults declaration. Fixed in 21880e4 — dropped "default": false / "default": "editable" from both property schemas and the JSDoc; implied default text moved into the description.
  • P2 DBDashboardPage.tsx:1669 page-level merge helpers uncovered. Fixed in 21880e4 — extracted normalizeExpression, buildConstantExpressionSet, stripConstantsFromUrl, mergeConstantFiltersForSave, upsertSavedDefault, removeSavedDefaultForExpression into dashboardFilterUtils.ts, with the matrix (constants + URL collisions, no constants, constants without saved entries, bracket-notation) covered by a new unit test.
  • P2 useDashboardFilters.test.tsx:355 bracket-notation untested. Fixed in 21880e4 — added a constant: true test with SpanAttributes['k8s.pod.name'] covering hook overlay, setFilterValue no-op, and getFilterQueriesForSource.

P3 nitpicks all addressed in the same commit:

  • DBDashboardPage.tsx:1583 constantExpressions rebuilt three times — now one helper.
  • types.ts:1162 enum unused — FilterVisibility now z.infer<typeof DashboardFilterRenderMode>.
  • DashboardFiltersModal.tsx:87 applyFilterVisibility('editable') returns {} instead of { constant: undefined, renderMode: undefined }.
  • DashboardFiltersModal.tsx:412 set equality instead of positional compare.
  • DashboardFiltersModal.tsx:195 NEVER_USED_RANGE hoisted to module scope.
  • DashboardFiltersModal.tsx:188 as DashboardFilter cast dropped.
  • usePresetDashboardFilters.tsx widened to accept _options?: { defaultValues?: string[] }.
  • DBDashboardPage.tsx:170 parseQuery import uses the @/searchFilters alias.
  • DBDashboardPage.tsx:1622 dashboard?.filters dropped from deps (with an eslint-disable explaining why).
  • DBDashboardPage.tsx:1470 handleRemoveFilter strips the matching savedFilterValues entry.
  • types.ts:1183 renderMode: 'readonly' | 'hidden' without constant: true rejected by refineDashboardFilterCoherence.
  • dashboards.test.ts:961 toMatchObject swapped to toEqual so unexpected fields fail the assertion.

make ci-lint + 45 touched unit tests pass locally. Waiting on CI for the e2e shard-2 retry.

alex-fedotyev added a commit that referenced this pull request May 30, 2026
Two fixes for #2383 CI:

1. DashboardFiltersModal default-value picker no longer issues a query
   when the resolved table name is empty. A metric source with no
   metricType picked yet resolves `getMetricTableName` to
   `source.from.tableName`, which is typically empty on a metric source.
   Firing the autocomplete in that state produces a malformed
   `DESCRIBE {db}.{}` (empty Identifier substitution) that the
   ClickHouse proxy rejects with a 500. The e2e shard 2 trace shows
   exactly this 500 firing during filter editing, which left the
   dashboard filter chip dropdowns disabled long enough to time out the
   click-the-option waits. Gating on `tableName` keeps the picker idle
   until the form is configured enough to name a real table.

2. The external API HDX-4404 round-trip test asserted
   `whereLanguage: 'sql'` on the response for filters whose input payload
   omits `whereLanguage`. The Zod schema is `.optional()` with no
   default, so the wire format is undefined-in / undefined-out. The
   previous expectation made the test fail by asserting a default the
   schema never emits. Dropped the field from those three filter
   expects and from the PUT-flip filter expect; added a comment so the
   omission is intentional.
@alex-fedotyev
Copy link
Copy Markdown
Contributor Author

Pushed 73d370d with two CI fixes after digging into the shard 2 trace:

Integration test: dashboards.test.ts:965 was asserting whereLanguage: 'sql' on filters whose input payload omits whereLanguage. The Zod schema is .optional() with no default, so the wire format is undefined-in / undefined-out. Dropped the field from three filter expects in the round-trip test plus the PUT-flip expect; added a comment so the omission is deliberate.

E2E shard 2: trace.network shows a 500 on clickhouse-proxy at the moment of filter editing:

Code: 457. DB::Exception: Empty Identifier part after parameter `HYPERDX_PARAM_0` substitution.

That HYPERDX_PARAM_0 is the hash of an empty string. The SQL body is DESCRIBE {db}.{} with an empty Identifier substitution. The FilterDefaultValueSelect I added inside the modal fires useDashboardFilterValues while editing, and for a metric source with no metricType picked yet, getMetricTableName(source, undefined) resolves to source.from.tableName, which is empty on metric sources. That's what put the request together and got a 500. The 500 left the chip dropdowns disabled long enough to time out the option clicks.

Gated filterForValueQuery on tableName resolving non-empty so the picker stays idle until the form is configured enough to name a real table.

Local make ci-lint passes, make ci-unit passes (1782 / 1782; one ENOMEM in TraceRedirectPage.test.tsx that's environment, unrelated). Waiting on CI shard 2.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 30, 2026

Deep Review

Intent (per author): add constant + renderMode to dashboard filters so a saved value can be locked into every read, and so a chip can render as editable, read-only, or hidden. External API + MCP both round-trip the new fields. Read path overlays constants over URL state; URL never reflects constants.

Scope: ~2500 LOC across 23 files. Base 2a68145.

🔴 P0/P1 — must fix

  • packages/app/src/DBDashboardPage.tsx:1706handleRemoveSavedQuery unconditionally writes draft.savedFilterValues = [], which silently drops every constant filter's locked value while filter.constant === true remains; the chip still renders the lock icon, setFilterValue still no-ops, but tiles now run with no scope on the previously-locked expression — exactly the failure mode the feature exists to prevent.
    • Fix: preserve savedFilterValues entries whose normalized expression matches a constant: true filter (mirror mergeConstantFiltersForSave with an empty URL input) instead of clearing the array.
    • correctness, adversarial

🟡 P2 — recommended

  • packages/app/src/DBDashboardPage.tsx:1450handleSaveFilter upserts under filter.expression but never removes the old expression's savedFilterValues entry, so renaming a constant filter's expression leaves an orphaned saved value behind and the renamed filter resolves to undefined.

    • Fix: in handleSaveFilter, detect when the saved filter's expression differs from the existing filter and call removeSavedDefaultForExpression(draft.savedFilterValues, oldExpression) before the upsertSavedDefault for the new one.
  • packages/api/src/routers/external-api/v2/utils/dashboards.ts:1155buildDashboardBodySchema only cross-checks sibling coherence; it does not require constant: true filters to have a matching savedFilterValues entry, so the clone-and-relock template path silently ships a "locked" dashboard with no WHERE scope when the caller forgets the saved value.

    • Fix: add a superRefine that, for every filter with constant: true, asserts at least one savedFilterValues entry whose Lucene condition's normalized key matches the filter's expression — reject with a clear path so MCP callers see the error.
    • adversarial, agent-native
  • packages/api/src/mcp/tools/dashboards/schemas.ts:743mcpFiltersParam is a bare z.array(mcpDashboardFilterSchema) with no array-level refinement; the external API rejects mixed constant: true + editable siblings on the same expression, but the MCP tool path only catches it later as a Validation error JSON inside an isError: true tool response, which contradicts the schema description and gives agents an inconsistent rejection surface across the two write paths.

    • Fix: apply .superRefine((filters, ctx) => refineDashboardFiltersConstantSiblings(filters as DashboardFilter[], ctx)) to mcpFiltersParam, or share buildDashboardBodySchema across the MCP and external-API entry points.
    • api-contract, kieran-typescript
  • packages/api/src/routers/api/dashboards.ts:47 — the internal POST/PATCH routes validate via DashboardWithoutIdSchema / DashboardSchema.partial(), neither of which carries refineDashboardFilterCoherence or refineDashboardFiltersConstantSiblings; any server-internal or non-MCP/non-external-API caller can persist renderMode: 'readonly' without constant: true, or mixed constant/editable siblings, and the subsequent GET returns them as if valid.

    • Fix: wrap the internal body schema in the same refinements buildDashboardBodySchema applies, or expose a shared validated dashboard-body schema both routes import.
    • kieran-typescript, api-contract
  • packages/app/src/DashboardFiltersModal.tsx:75getFilterVisibility collapses constant: true with renderMode undefined or 'editable' to the 'readonly' preset; applyFilterVisibility('readonly') then writes renderMode: 'readonly' back on save, so an MCP-authored filter with only constant: true silently mutates its wire shape the first time a human opens and re-saves it in the editor.

    • Fix: either preserve the inbound renderMode value on save when the user did not change the Visibility select, or document the deliberate normalization and add a round-trip test pinning the post-save shape.
  • packages/app/src/hooks/useDashboardFilters.tsx:130 — during first render after a stale shared link, dashboard?.filters may still be loading while the URL already populated filterQueries; the memo computes an empty constantExpressions, treats the URL entry as editable, and emits the URL-derived WHERE into tile queries before constants hydrate, so React Query can cache a tile result scoped to the stale URL value rather than the locked saved value.

    • Fix: short-circuit the overlay (return an empty valuesForExistingFilters) when filters is empty and the dashboard load has not yet resolved, or gate getFilterQueriesForSource on a dashboardReady flag in DBDashboardPage so tiles do not query until both filters and savedFilterValues have arrived.
  • packages/api/src/routers/external-api/__tests__/dashboards.test.ts — the new round-trip test does not cover renderMode: 'readonly'/'hidden' without constant: true (coherence rejection), mixed constant/editable siblings (array-level rejection), or constant: true paired with a savedFilterValues entry on the same expression; the MCP test file covers all three, so a regression that drops a refinement from the external API path would not be caught here.

    • Fix: port the three corresponding MCP test cases (should reject renderMode without constant: true, should reject mismatched sibling constants, and the savedFilterValues + constant filter round-trip) into the external-API test suite.
🔵 P3 nitpicks (14)
  • packages/app/src/hooks/usePresetDashboardFilters.tsx:31 — the preset hook never threads savedFilterValues, so any constant: true filter declared on a preset dashboard resolves to undefined and applies no WHERE scope.

    • Fix: either reject constant: true at the preset definition boundary or thread a savedFilterValues source through to useDashboardFilters from the preset hook.
  • packages/app/src/DBDashboardPage.tsx:1603 — the hydration block runs setFilterQueries(stripConstantsFromUrl(...)) whenever dashboard.savedFilterValues is truthy, including for an empty array with no constants declared, producing a spurious URL write on every dashboard load.

    • Fix: gate the hydration block on dashboard.savedFilterValues.length > 0 (not just truthiness).
  • packages/app/src/dashboardFilterUtils.ts:93mergeConstantFiltersForSave re-encodes preserved constant saved entries through parseQuery + filtersToQuery, which coerces any original type: 'sql' saved condition to type: 'lucene' on every save.

    • Fix: preserve the original constant entries verbatim instead of round-tripping them, or document the coercion explicitly.
  • packages/app/src/hooks/useDashboardFilters.tsx:130 — two filters whose raw expressions differ but normalize the same (Attr['k'] vs Attr.k) both write into valuesForExistingFilters and produce duplicate WHERE clauses per tile.

    • Fix: dedupe the overlay write by normalized expression before populating valuesForExistingFilters.
    • correctness, adversarial
  • packages/app/src/hooks/useDashboardFilters.tsx:221hasScrubbedConstantsRef is a one-shot guard that does not re-fire when constantExpressions grows mid-session (filter flipped to constant: true at runtime) or when the dashboard id changes inside the same mount, leaving stale URL entries until the next setFilterValue write.

    • Fix: key the scrub guard on the dashboard id (or on a hash of constantExpressions) so a changed constant set re-runs the URL scrub.
    • correctness, julik-frontend-races
  • packages/app/src/hooks/useDashboardFilters.tsx:110savedFilterValues whose Lucene condition fails parseLuceneFilter fall into passthroughFilters and bypass the constant overlay entirely, so a constant filter whose saved value happens to be unparseable resolves silently to no value.

    • Fix: validate savedFilterValues conditions at the API boundary and reject unparseable entries, or surface a user-visible warning when a constant resolves to undefined despite having a saved entry.
  • packages/app/src/hooks/useDashboardFilters.tsx:124 — the inline constantByNormalized loop rebuilds exactly the set already memoized as constantExpressions higher in the same hook.

    • Fix: reuse the existing constantExpressions reference instead of constructing the parallel set.
  • packages/app/src/DashboardFiltersModal.tsx:151getDefaultValuesForExpression inlines parseKeyPath(expression).join('.') rather than calling the normalizeExpression helper this same PR extracts into dashboardFilterUtils.ts.

    • Fix: import normalizeExpression from @/dashboardFilterUtils and replace the inline parseKeyPath(...).join('.') calls.
  • packages/app/src/DashboardFilters.tsx:39isLocked and visibleFilters predicates are re-derived inline at the chip-row level alongside getFilterVisibility in the modal, so a future renderMode value would have to be added in three places.

    • Fix: export isFilterLocked(f) and isFilterVisible(f) from dashboardFilterUtils.ts and consume them from both files.
  • packages/app/src/DashboardFiltersModal.tsx:266 — the { defaultValues?: string[] } options bag is duplicated inline at five call sites with no named type, so a future field addition has no compiler-enforced propagation.

    • Fix: export a FilterSaveOptions type from DashboardFiltersModal.tsx and use it at all onSave / onSaveFilter / handleSaveFilter sites.
  • packages/api/openapi.json:6info.version stays at 2.0.0 despite the additive growth (constant, renderMode, lucene added to SavedFilterValue.type).

    • Fix: bump to 2.1.0 so generated SDKs and pinning tools can detect the additive extension.
  • packages/app/src/DashboardFiltersModal.tsx:1 — the file grows to ~967 lines (pre-existing 551, this PR adds ~416), pushing further past the project's 300-line component guideline; pre-existing violation worsened by this diff.

    • Fix: extract the new Visibility/default-value editor surface into a sub-component (e.g. FilterVisibilityForm.tsx).
  • packages/app/src/DashboardFiltersModal.tsx:202filtersForQuery is recomputed as [filterForValueQuery], so any change to filterForValueQuery (a fresh object identity per dep change) re-keys useOptimizedKeyValuesCalls and triggers a ClickHouse subscription churn while the modal is open.

    • Fix: key the memo on the primitive fields filterForValueQuery depends on, or pass filterForValueQuery to the hook directly without the wrapping array memo.
  • packages/app/src/DBDashboardPage.tsx:1623 — the initialization effect lists dashboard?.savedFilterValues in deps but reads dashboard?.filters inline and disables the exhaustive-deps lint; the initializedDashboard.current === dashboard.id guard keeps behavior correct, but the dep mismatch is a maintenance footgun.

    • Fix: drop savedFilterValues from the dep array (or move both reads behind a ref) and leave a one-line comment naming the dashboard-id guard as the intended re-run signal.

Reviewers (11): correctness, testing, maintainability, project-standards, agent-native, learnings, security, api-contract, adversarial, kieran-typescript, julik-frontend-races.

Testing gaps:

  • No hook test asserts result.current.filterQueries (the array consumed by tile WHERE) for any constant-filter case — only filterValues and getFilterQueriesForSource are exercised.
  • No hook test covers savedFilterValues: null (or option omitted) paired with constant: true, nor a runtime flip of editable -> constant: true after first mount (the hasScrubbedConstantsRef path).
  • No hook test for constant: true + renderMode: 'editable' — the combination the schema accepts but the UI does not surface.
  • No test for handleRemoveSavedQuery against a dashboard with constant filters (the P1 cascade above).
  • No test for the constant-filter expression-rename path (orphaned savedFilterValues entry).
  • No external API + MCP test for an originally type: 'sql' constant saved entry surviving the round-trip without language coercion.
  • No regression test pinning that ?filters= URL overrides cannot win against a constant: true filter on the same expression (server-side / read-side contract).
  • No round-trip test for an MCP-created filter with only constant: true and no renderMode, asserting the wire shape after a UI re-save.

alex-fedotyev and others added 6 commits June 1, 2026 15:28
E2E shard 2 was failing on three pre-existing dashboard tests because
`visibleFilters` in DashboardFilters.tsx was a new array reference every
render, churning useDashboardFilterValues' useQueries and leaving the
chip dropdown disabled while the values query never settled. Memoize
the array so a stable reference flows into the hook.

Deep review fixes:

- P0/P1: MCP hyperdx_save_dashboard inputSchema now exposes
  savedFilterValues, paired with a usage example in the filters param
  description so constant: true filters can actually lock to a value
  via MCP. The body schema already accepts it; only the tool input
  shape was missing.

- P2: DashboardFiltersModal snapshots initialDefaultValues via state
  keyed on filter.id, so a background savedFilterValues refetch no
  longer wipes in-progress edits in the default-value picker.

- P2: Schema-level coherence checks moved to common-utils via two
  reusable refinements applied at the external API boundary:
  refineDashboardFilterCoherence rejects readonly/hidden render modes
  without constant: true; refineDashboardFiltersConstantSiblings
  rejects two filter definitions sharing an expression that disagree
  on constant. Internal callers (MCP, future migrations) get the
  same protection through the shared schemas.

- P2: openapi.json + JSDoc no longer declare default: false /
  default: "editable" on the new constant / renderMode property
  schemas. The Zod schema is .optional() with no .default() so
  generated SDKs would otherwise surface a default the wire format
  never produces; the implied default now lives in the description
  text instead.

- P2: page-level merge logic for "Save default" / URL hydration /
  savedFilterValues upsert is extracted into dashboardFilterUtils
  (normalizeExpression, buildConstantExpressionSet,
  stripConstantsFromUrl, mergeConstantFiltersForSave,
  upsertSavedDefault, removeSavedDefaultForExpression). The matrix
  including bracket-notation expressions is covered by a new unit
  test. useDashboardFilters and DBDashboardPage call the helpers.

- P2: bracket-notation constant filter is now covered in the
  hook unit tests (SpanAttributes['k8s.pod.name'] resolves through
  filterValues, blocks setFilterValue, and survives via
  getFilterQueriesForSource).

P3 polish:

- handleRemoveFilter now strips the matching savedFilterValues entry
  alongside the filter delete so deleting + recreating a filter on the
  same expression doesn't silently re-lock to an orphaned value.
- applyFilterVisibility('editable') returns {} so the spread on save
  no longer leaves constant: undefined / renderMode: undefined keys
  in memory.
- defaultsChanged uses set equality so reordering the same values
  doesn't fire a spurious save write.
- NEVER_USED_RANGE hoisted to module scope.
- FilterVisibility uses z.infer<typeof DashboardFilterRenderMode>
  instead of a hand-rolled string union.
- as DashboardFilter cast dropped on filtersForQuery.
- usePresetDashboardFilters.handleSaveFilter widened to accept the
  optional defaultValues arg so a future supportsConstantFilters=true
  flip on presets doesn't silently drop the second arg.
- parseQuery import uses the @/searchFilters alias for consistency
  with sibling files.
- dashboard?.filters dropped from the URL hydration effect's deps
  (the initializedDashboard.current === dashboard.id guard already
  prevents re-runs).
- HDX-4404 API round-trip test uses toEqual instead of toMatchObject
  so unexpected extra fields fail the assertion.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two fixes for #2383 CI:

1. DashboardFiltersModal default-value picker no longer issues a query
   when the resolved table name is empty. A metric source with no
   metricType picked yet resolves `getMetricTableName` to
   `source.from.tableName`, which is typically empty on a metric source.
   Firing the autocomplete in that state produces a malformed
   `DESCRIBE {db}.{}` (empty Identifier substitution) that the
   ClickHouse proxy rejects with a 500. The e2e shard 2 trace shows
   exactly this 500 firing during filter editing, which left the
   dashboard filter chip dropdowns disabled long enough to time out the
   click-the-option waits. Gating on `tableName` keeps the picker idle
   until the form is configured enough to name a real table.

2. The external API HDX-4404 round-trip test asserted
   `whereLanguage: 'sql'` on the response for filters whose input payload
   omits `whereLanguage`. The Zod schema is `.optional()` with no
   default, so the wire format is undefined-in / undefined-out. The
   previous expectation made the test fail by asserting a default the
   schema never emits. Dropped the field from those three filter
   expects and from the PUT-flip filter expect; added a comment so the
   omission is intentional.
…tion

The DashboardFiltersModal's FilterDefaultValueSelect fires useDashboardFilterValues
queries with a temporary 'new' filter ID. Since filterIds were computed inside
queryFn and cached, the shared React Query cache entry had filterIds: [['new']].
When the chip's useDashboardFilterValues hook (same source/expression/dateRange,
same queryKey) read from cache, filterValuesById.get(realUUID) returned undefined,
leaving the chip's VirtualMultiSelect disabled indefinitely.

Fix: compute filterIds outside queryFn (in a useMemo after useQueries returns) so
they are derived from the current caller's filter list at render time instead of
being frozen in the cache from whoever populated it first.

Also add missing await to DashboardPage.clickFilterOption's serviceFilter.click()
so the click is properly awaited before waiting for the dropdown option to appear.
Round 2 deep-review fix-pack.

**P0/P1**

- `DashboardFiltersModal.tsx`: `applyFilterVisibility('editable')` returns
  `{}`, so flipping a saved readonly/hidden filter back to "Editable"
  let `rest` carry the original `constant: true` and `renderMode`
  through the spread. The persisted filter stayed locked while the
  visibility UI claimed it was editable. Destructure `constant` and
  `renderMode` out of `rest` alongside `visibility` / `defaultValues`
  so the `visibilityFields` spread is authoritative.
- `DashboardFiltersModal.tsx`: the default-value picker's
  `filterForValueQuery` memo depended on raw `useWatch` of CodeMirror-
  backed inputs, firing a ClickHouse autocomplete request per
  keystroke with mid-edit WHERE strings that the proxy rejects. Pipe
  `watchedExpression` / `watchedWhere` / `watchedWhereLanguage`
  through `useDebouncedValue(..., 300)` (matches
  `MetricAttributeHelperPanel`) so the picker only re-queries once
  the author pauses.

**Correctness (P2)**

- `useDashboardFilters.tsx`: scrub stale `constant` expressions out
  of `filterValues` inside `setFilterValue` before
  `filtersToQuery`, AND add a one-shot hydration scrub on first
  non-empty URL snapshot. Without this, a viewer landing with a
  stale shared link that carries a value for an expression now
  locked by `constant: true` would re-publish the locked scope to
  the URL the next time any sibling filter wrote.
- `useDashboardFilters.tsx`: aggregate by normalized expression in
  the overlay loop so legacy data with mixed `constant: true` +
  editable siblings on the same expression resolves
  deterministically (any sibling lock means the saved value wins
  for every sibling).
- `openapi.json` + JSDoc: widen `SavedFilterValue.type` enum to
  `[sql, lucene]`. The UI's "Save default" flow writes
  `type: 'lucene'` and the runtime overlay only matches when the
  caller follows that shape; the previous spec advertised `[sql]`
  only.
- `common-utils/types.ts`: import `parseKeyPath` for
  `refineDashboardFiltersConstantSiblings` so the schema
  normalization matches the runtime overlay
  (`useDashboardFilters -> parseKeyPath`) and the filter helpers
  (`filters.ts -> parseKeyPath`). Move `parseKeyPath` to a new
  `core/keyPath.ts` leaf module to avoid forming a circular import
  (`types.ts -> metadata.ts -> clickhouse -> guards.ts -> types.ts`);
  `core/metadata.ts` re-exports for existing call sites.

**Correctness (P3)**

- `DBDashboardPage.tsx`: `handleRemoveFilter` no longer strips the
  shared `savedFilterValues` entry when another sibling filter
  still references the same normalized expression (two
  `constant: true` siblings scoped to different `appliesToSourceIds`
  are schema-legal and share one saved entry).
- `DashboardFilters.tsx`: rename `isReadOnly` -> `isLocked` and
  document the upstream `visibleFilters` exclusion so the name no
  longer conflates "chip is rendered locked" with "chip is locked
  but possibly hidden upstream".

**API contract**

- `mcp/tools/dashboards/schemas.ts`: apply
  `refineDashboardFilterCoherence` directly on
  `mcpDashboardFilterSchema` and document the sibling-agreement
  rule on the `.describe` block; LLM callers now hit the coherence
  rule at the input boundary instead of via downstream server
  rejection.
- `external-api/v2/dashboards.ts`: document the sibling-agreement
  rule on `FilterInput.constant` and note that `savedFilterValues`
  is overwritten as a whole on update (drop orphan entries).
- `mcp/tools/dashboards/saveDashboard.ts`: matching note on the
  `savedFilterValues` describe.

**Testing**

- `common-utils/dashboardFilter.test.ts`: 14 new tests on the two
  refinement helpers (`refineDashboardFilterCoherence` and
  `refineDashboardFiltersConstantSiblings`), including
  bracket-vs-dot normalization and nested bracket cases.
- `app/hooks/useDashboardFilters.test.tsx`: assert
  `ignoredFilterExpressions === []` on saved-vs-URL collision;
  empty `getFilterQueriesForSource` when a constant has no saved
  value; URL scrubbing on `setFilterValue` writes; aggregated-overlay
  behavior on legacy mixed siblings.
- `api/mcp/__tests__/dashboards.test.ts`: `savedFilterValues`
  round-trip with `toEqual` (catches schema drift in either
  direction); reject mismatched-sibling-constants; reject
  `renderMode: readonly` without `constant: true` on the MCP
  schema.
…xtraction

850ac84 moved parseKeyPath from metadata.ts into a leaf core/keyPath.ts
module to break a metadata -> clickhouse -> guards -> types -> metadata
cycle. External call sites kept working because metadata.ts re-exports the
name, but metadata.ts itself still uses parseKeyPath inline at the
getKeyValues body (line 1353). A re-export does not bind the name in the
local module scope, so the dts build fails with TS2304:
"Cannot find name 'parseKeyPath'" once make ci-build runs. The regular
tsc --noEmit on @hyperdx/app missed it because that workspace consumes
common-utils through its pre-built dist.

Add the matching import at the top and collapse the bottom re-export to
plain export { parseKeyPath } so the file binds and re-exports the same
symbol.

Verified locally with make ci-build, make ci-lint, and the common-utils
metadata test suite (50/50).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

review/tier-4 Critical — deep review + domain expert sign-off

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant