objectstack-ai
diff --git a/‎.changeset/engine-dotted-projection-refused.md‎
Lines changed: 55 additions & 0 deletions b/‎.changeset/engine-dotted-projection-refused.md‎
Lines changed: 55 additions & 0 deletions
diff --git a/‎content/docs/protocol/objectql/types.mdx‎
Lines changed: 10 additions & 3 deletions b/‎content/docs/protocol/objectql/types.mdx‎
Lines changed: 10 additions & 3 deletions
diff --git a/‎docs/protocol-upgrade-guide.md‎
Lines changed: 7 additions & 0 deletions b/‎docs/protocol-upgrade-guide.md‎
Lines changed: 7 additions & 0 deletions
@@ -0,0 +1,55 @@
+---
+"@objectstack/objectql": minor
+---
+
+<!-- adr-0087: registered engine-dotted-projection-refused -->
+
+fix(objectql)!: `engine.find` / `engine.findOne` refuse a dotted projection instead of widening the response to every field (#7589)
+
+`engine.find()` and `engine.findOne()` are a **public API**, and a `fields`
+entry carrying a dotted path (`['name', 'account.name']`) — which used to
+answer 200 with **every** column, byte-identical to no projection at all —
+now **throws `400 INVALID_FIELD`**.
+
+#7532 (PR #7588) closed this at the REST ingress
+(`assertProjectionFieldsExist`), covering everything that reaches `findData`.
+A caller reaching the engine directly passed through none of it, and that
+caller set was measured, not assumed (#7589): a flow `get_record` node's
+authored `fields: ['name', 'account.name']` parses (`GetRecordConfigSchema`
+restricts nothing), travels verbatim into `data.find(...)` /
+`data.findOne(...)`, cleared the engine's head-only projection filter on its
+head segment (`account` IS a field), and reached the driver as a projection
+column — where SQL renders `"account"."name"` against a table that was never
+joined, the DB answers `no such column`, and the driver's #3821 recovery
+ladder retries `select('*')`. The caller asked to narrow and silently
+received everything, pointing away from both FLS and data minimisation. A
+saved report's `query.fields` (`plugin-reports` forwards it verbatim) reached
+it the same way.
+
+The head-only check was justified by its own comment: "the engine will
+resolve those via populate". **No populate step exists** — #7601 measured it,
+and this comment was the last place in the repo asserting dotted-path
+resolution does. The comment and the check it explained are gone together;
+what is removed is not a working feature but a path to widening, kept alive
+by a false premise.
+
+**FROM → TO**: a direct-engine caller (flow `get_record` `fields`, saved
+report `query.fields`, hook code) projecting `account.name` reads the related
+record with `expand` (`{ expand: { account: { object: '<target>', fields:
+['name'] } } }`) while keeping the reference column itself in `fields`, or
+denormalises the value onto the queried object (a stored field, written when
+the source changes) and names that. A plain reference column (`fields:
+['account']`) still projects.
+
+**Deliberately KEPT** (same ruling, 2026-08-12): the unknown-PLAIN-column
+tolerance — an unknown plain name is still dropped silently and an
+all-unknown projection still falls back to `*`, because the "no records
+exist" failure that tolerance prevents is real. A registry-less host (no
+field map) gets **no** verdict, exactly as the ingress gate returns early
+there; for that host the driver-side #3821 ladder remains the documented
+backstop, and a driver-side carve-out stays measured-need only. One path is
+observable rather than refused: a dotted `fields` inside a nested `expand`
+raises this refusal inside `expandRelatedRecords`' pre-existing
+graceful-degradation `catch`, so it logs a warning naming the field and the
+fix and retains the raw foreign keys — the same posture the sort axis (#7095)
+records for the same catch.
@@ -605,11 +605,18 @@ account_id:
 
 **Storage:** Stores `id` of referenced record
 
-**Query behavior:**
+**Query behavior:** `expand` is the door for related data. A dotted `fields`
+entry (`'account_id.company_name'`) is **refused** (`400 INVALID_FIELD`) — no
+driver resolves one, at the REST ingress since #7532 and on direct
+`engine.find` / `engine.findOne` calls since #7589. Keep the reference column
+itself in the projection: the relation is carried by `account_id`, and
+projecting it away leaves the expansion nothing to resolve.
+
 ```typescript
-// Expand the account lookup
+// Read a column of the related account
 const opportunities = await engine.find('opportunity', {
-  fields: ['name', 'account.company_name']  // Expands account
+  fields: ['name', 'account_id'],
+  expand: { account_id: { object: 'account', fields: ['company_name'] } },
 });
 ```
 
 
@@ -394,6 +394,13 @@ What makes this one cheaper to meet than its two siblings, and worth saying beca
 - **`driver-sql-distinct-bare-filter-typed`** — `SqlDriver.distinct() third argument — any value` → a bare FilterCondition (@objectstack/spec/data) — the same value find() carries under query.where, never a query envelope
   - Why not automatic: This entry records a TYPE being added, not a surface being withdrawn, and it says so up front because the distinction decides who has to do anything. `distinct` is not declared on `IDataDriver`, so #5181 / #6075 never reached it and it kept `filters?: any` while its body said something far more specific — `applyFilters(builder, filters)` is handed the ARGUMENT ITSELF, never a `.where` off it. ⚠️ RUNTIME BEHAVIOUR IS UNCHANGED by this entry's change: not one statement moved, so no upgrade breaks at run time and nothing that answered correctly stops. What the annotation removes is a compile-time hole, measured rather than assumed: a truthy NON-OBJECT third argument — `distinct('orders', 'product', 'completed')` — used to type-check and resolve the UNFILTERED set, because `applyFilters` emits no predicate at all for a truthy non-object, non-array filter. A call meaning "which products among completed orders" answered with EVERY product, silently. That spelling is now TS2345 at the call site. This is a driver CALL ARGUMENT — code, never stack metadata — so there is no source for the D2 chain to rewrite and deliberately no schema tombstone, the disposition `data-driver-find-stream-retired` (#4484), `storage-service-list-retired` (#5540), `actor-user-roles-to-positions` (#6011) and `driver-aggregate-undeclared-key-aliases-removed` (#6321) already carry. ⚠️ It differs from those four in ONE measured way a reader should not have to infer: because nothing changed at run time, an untyped JS caller is not affected BY THE UPGRADE at all. The entry is here for a different reason — such a caller is exactly the one tsc can never reach, and the silent widening above is a defect they may ALREADY be sitting on, before and after this major. The generated upgrade guide is the only channel that reaches them, which is why the fix is written down rather than left to the compiler. ⛔ The reverse mismatch is NOT closed and no type can close it: `FilterCondition` is an open map (`[key: string]: any`) because a filter key IS a field name, so a query envelope `{ object, where }` is structurally a valid filter — one constraining columns named `object` and `where` — and so is a FilterArray. Both reach `distinct` type-checked and are refused at run time, loudly, with INVALID_FILTER / 400. `driver-memory`'s opposite half — where the BARE spelling returns the unfiltered set in silence — stays open under the #5499 freeze (#6320). ADR-0087, #6320.
   - Done when: No caller passes a non-object to `distinct()`'s third argument. A scalar there is now a compile error (`TS2345: Argument of type 'string' is not assignable to parameter of type 'FilterCondition'`); rewrite it as the bare filter it was always meant to be — `'completed'` becomes `{ status: 'completed' }`. ⚠️ That is NOT an equivalent rewrite: the old spelling returned the UNFILTERED set, so the answer changes once fixed, and the changed answer is the one the call always meant. An untyped JS caller gets no compile error and no behaviour change — for them this entry is the only notice that the spelling never filtered anything. A query envelope or a FilterArray in that slot still compiles and is rejected at run time with INVALID_FILTER / 400.
+- **`engine-dotted-projection-refused`** — `engine.find(object, { fields }) and engine.findOne(object, { fields }) carrying a dotted entry (`account.name`) — the direct engine path, not the REST ingress` → read the related record with `expand` (`{ expand: { account: { object: '<target>', fields: ['name'] } } }`), keeping the reference column itself in `fields` — the relation is carried by that column and projecting it away leaves expansion nothing to resolve (#7537); or denormalise the value onto the queried object (a stored field, written when the source changes) and name that — the same remedy the REST ingress has prescribed since #7532, and the sort axis since #6924
+  - Why not automatic: #7532 (PR #7588) closed the PROJECTION axis' dotted leg at the REST ingress (`assertProjectionFieldsExist`, `400 INVALID_FIELD`), which covers everything reaching `findData`. A caller reaching `engine.find()` / `engine.findOne()` DIRECTLY passed through none of it, and that caller set was measured, not assumed (#7589): a flow `get_record` node's authored `fields: ['name', 'account.name']` parses (`GetRecordConfigSchema` restricts nothing), travels verbatim into `data.find(...)`, cleared the engine's head-only projection filter on its head segment (`account` IS a field), and reached the driver as a projection column — where SQL renders `"account"."name"` against a table that was never joined, the DB answers `no such column`, and the driver's #3821 recovery ladder retries `select('*')`. The caller asked to narrow and silently received EVERY field, byte-identical to no projection at all, pointing away from both FLS and data minimisation.
+
+Ruled 2026-08-12 on #7589 (Option B): a dotted entry the engine cannot resolve is refused loudly at the engine's own head-only projection filter, covering every caller that reaches the engine. The check it replaces was justified by a comment claiming the engine resolves relationship paths "via populate"; #7601 measured that NO populate step exists — after PR #7617 that comment was the last place in the repo asserting dotted-path resolution does — so what was removed is not a working feature but a path to widening, kept alive by a false premise. The unknown-PLAIN-column tolerance is explicitly KEPT by the same ruling (an unknown plain name still drops silently; an all-unknown projection still falls back to `*`), a registry-less host gets no verdict (the driver-side #3821 ladder remains its documented backstop, and a driver-side carve-out is measured-need only), and a dotted `fields` inside a nested `expand` degrades to an observable warning rather than a refusal — `expandRelatedRecords`' pre-existing graceful-degradation `catch` swallows every expand failure, the same posture the sort axis (#7095) records for the same catch.
+
+This is a CODE-path API, not stored metadata, so — like `engine-find-formula-order-by-refused` at this step — there is no `sys_metadata` row for the D2 chain to rewrite and the ledger entry is the notification channel. No mechanical rewrite exists: the platform cannot decide between `expand` and denormalisation for the caller, and it must not resolve the path itself — no driver ever did, and inventing a join here is a feature decision, not a migration. #7589, #7532, #7601, #3821, #5918, ADR-0112.
+  - Done when: No `engine.find` / `engine.findOne` call site passes a dotted `fields` entry, no flow `get_record` config authors one, and no saved report's `query.fields` names one — grep flow definitions and report definitions for a `fields` entry containing a `.`, and rewrite each to `expand` (keeping the reference column projected) or to a denormalised stored column. Reads complete with no `INVALID_FIELD` whose message says "follows the relationship" or "a dotted path", and no "Failed to expand relationship field" warning whose error text does.
 - **`engine-find-formula-order-by-refused`** — `engine.find(object, { orderBy }) and engine.findOne(object, { orderBy }) naming a `formula` field — the direct engine path, not the REST ingress` → denormalise the value onto the object (a stored field, written when the source changes) and sort by that — the same remedy the REST ingress has prescribed since #6924 / #6994; a `summary` field is unaffected and still sorts, because it gets a real maintained column
   - Why not automatic: #4226 / #4256 / #6994 closed the SORT axis at the REST ingress (`assertSortFieldsExist`, `400 INVALID_SORT`), which covers everything reaching `findData`: the list route, `POST /data/:object/query`, the export route and the RPC dispatcher. A caller reaching `engine.find()` / `engine.findOne()` DIRECTLY passed through none of it, and a `formula` ORDER BY there was dropped in silence. Measured on a real driver: `asc` and `desc` came back BYTE-IDENTICAL, in insertion order, under a success, with the rows carrying the very values they were asked to be ordered by. No column exists to order by (a formula is computed on read, so no driver materialises one), so the ORDER BY reached the driver, found nothing, and the unknown-column backstop returned the rows unordered.