Skip to content

fix: validate a filter's operator against the column type by evaluating it - #2099

Open
hamodywe wants to merge 2 commits into
supabase:mainfrom
hamodywe:fix/validate-filter-operator-at-subscribe-time
Open

hamodywe wants to merge 2 commits into
supabase:mainfrom
hamodywe:fix/validate-filter-operator-at-subscribe-time

Conversation

@hamodywe

Copy link
Copy Markdown

Closes #2098.

The problem

subscription_check_filters answers "can this operator be applied to this column type?" with a pg_operator lookup. check_equality_op answers it by building SQL and letting the planner resolve it. Those two answers differ, in both directions.

  • varchar carries no ~~ of its own — it resolves one through text — so oprleft = col_type fails and like/ilike/match/imatch are refused on varchar columns that handle them fine.
  • bytea carries ~~ but no ~~*, and the guard checks ~~ for both operators, so ilike on a bytea column is accepted and raises at WAL time.
  • The comparison operators are not checked at all. The else branch only verifies the value casts, so eq on a json, xml or point column registers and raises inside apply_rls.

The last one is the expensive case. is_visible_through_filters is evaluated in apply_rls' subscription loop, so the raise aborts the whole list_changes call: every subscription on the tenant loses the batch — other tables and other roles included — and because the slot's confirmed_flush advances during decoding, the batch is not replayed. One client subscribing to body=eq.x on a json column stops postgres_changes for the entire project. That is the same failure shape as #2093, reached through the filter path rather than the RLS path, and it is not covered by the containment in #2096 because is_visible_through_filters runs outside that block.

The change

Stop asking the catalog and ask the operator. The branch now probes check_equality_op — the exact call apply_rls will make — inside its own block:

begin
    perform realtime.check_equality_op(filter.op, col_type, filter.value, filter.value, filter.negate);
exception when others then
    raise exception 'operator % is not supported on column % of type %: %',
        filter.op::text, filter.column_name, col_type::text, sqlerrm;
end;

Because the probe is the runtime path, subscribe-time acceptance and WAL-time evaluation cannot drift apart — which no catalog predicate can guarantee, since operator resolution goes through implicit and binary-coercible casts that pg_operator does not record. The two pg_operator guards are removed as a result; the is keyword checks and the eager regex compile stay, the latter so a bad pattern keeps its own precise error instead of arriving as "unsupported operator".

The client now gets, at subscribe time:

operator eq is not supported on column body of type json: operator does not exist: json = json

Verification

I swept every operator against a table covering 17 column types, comparing what subscription_check_filters accepts against what check_equality_op can evaluate.

before after
accepted at subscribe, raises at WAL time 21 0
refused at subscribe, evaluates fine 4 0

The 21: json and xml on eq neq lt lte gt gte isdistinct, point on eq lt lte gt gte isdistinct, bytea on ilike. The 4: varchar on like ilike match imatch.

Also verified end to end, on supabase/postgres:17.6.1.127 with the tenant migrations applied:

Tests added to subscriptions_test.exs cover all three: the json rejection, the bytea ilike rejection, varchar like being accepted, and a varchar like subscription gating rows end to end through list_changes.

The tenant dumps are in this PR for the same reason as #2097: Realtime.Tenants.Migrations.load_db_dump/1 provisions a new tenant from priv/repo/tenant_db_dump_<major>.sql and then records every migration as applied without replaying any, so a stale dump would leave new tenants with the old trigger and the bug intact. update-tenant-db-snapshots.yml is gated to same-repo PRs so it will not run here; I edited the three snapshot files by hand and checked the result rather than the edit — a fresh database loaded from the edited tenant_db_dump_17.sql reports 82 migrations, matching migrations.ex, and re-running the full matrix against it gives 0 and 0. Please regenerate them properly before merge if you would rather not carry a hand-edit.

I could not run mix test locally — Windows box with no Elixir toolchain, and the suite provisions tenant databases through its own Docker backend. Everything above was verified at the SQL level instead; the Elixir tests are written to the file's existing patterns and run on CI.

One note on ordering: this and #2097 both add a line to the migration list in lib/realtime/tenants/migrations.ex and both touch the dumps, so whichever merges second needs a trivial rebase.

…ng it

subscription_check_filters decided whether an operator applies to a column
type by looking it up in pg_operator. That answer is wrong in both directions.

varchar carries no `~~` entry of its own but resolves one through text, so
`like` and `ilike` were refused on varchar columns that handle them fine.
bytea carries `~~` but no `~~*`, and the guard checks `~~` for both, so
`ilike` on a bytea column was accepted and then raised at WAL time. The
comparison operators were not checked at all, so `eq` on a json, xml or point
column registered without complaint and raised inside apply_rls.

is_visible_through_filters is evaluated in apply_rls' subscription loop, so
that raise aborts the whole list_changes call: every subscription on the
tenant loses the batch - other tables and other roles included - and the WAL
is not replayed. One client subscribing to `body=eq.x` on a json column is
enough to stop postgres_changes for the whole project.

Probe check_equality_op instead, the same call apply_rls makes, so acceptance
at subscribe time and evaluation at WAL time cannot disagree. Sweeping a
(column type x operator) matrix over 17 column types and 11 operators goes
from 21 accepted-but-raises and 4 rejected-but-works to none of either.

The bundled tenant dumps carry the trigger, and a new tenant is provisioned
from them without replaying migrations, so they are updated here too.
@edgurgel

edgurgel commented Sep 8, 2026

Copy link
Copy Markdown
Member

Hey team we are just in the process of moving tests we had on https://github.com/supabase/walrus to here so that we only change Realtime for PG changes. Once this has landed we can definitely accept this change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

subscription_check_filters accepts filter operators apply_rls cannot evaluate, and refuses some it can

2 participants