Skip to content

fix(data): decorrelate the 30-case demo volume generator - #1671

Merged
os-steve merged 1 commit into
mainfrom
claude/issue-1659-decorrelate-case-generator
Sep 6, 2026
Merged

fix(data): decorrelate the 30-case demo volume generator#1671
os-steve merged 1 commit into
mainfrom
claude/issue-1659-decorrelate-case-generator

Conversation

@os-steve

@os-steve os-steve commented Sep 6, 2026

Copy link
Copy Markdown
Collaborator

Description

The 30-case demo volume generator in src/data/service.seed.ts read one
counter for every axis it varies. crm_account and status both took i % 5,
and priority, type and origin all took i % 4. Equal-length lists walked
by the same index advance in lockstep, so each group came out perfectly
correlated. A generator whose declared purpose is volume with variety was
producing volume with none.

This decorrelates every axis onto a stated distribution, and reports the
measurement rather than an impression.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)

Related Issues

Closes #1659

The measurement — account x status cross-tab

Computed from the real seed by importing cases.records and cross-tabbing the
30 generated rows, before and after.

Before — 5 of 25 cells occupied, a perfect one-to-one map:

account closed escalated in_progress new resolved total
Acme Corporation . . . 6 . 6
Globex Industries . . 6 . . 6
Initech Solutions 6 . . . . 6
Stark Medical . 6 . . . 6
Wayne Enterprises . . . . 6 6
total 6 6 6 6 6 30

After — 25 of 25 cells occupied, every account showing all five statuses:

account closed escalated in_progress new resolved total
Acme Corporation 1 1 1 2 1 6
Globex Industries 1 1 2 1 1 6
Initech Solutions 2 1 1 1 1 6
Stark Medical 1 2 1 1 1 6
Wayne Enterprises 1 1 1 1 2 6
total 6 6 6 6 6 30

The other axes — status was not the only correlated pair

priority, type and origin all read i % 4, so they were a three-way
lockstep, not merely correlated with each other in passing: every low case was
a question raised by email, every medium a bug by phone, every high a
problem from web, every critical a feature_request from chat. The
generated subjects said so out loud — Demo case 02 — medium bug repeated every
four rows.

All ten pairwise cross-tabs, cells occupied:

pair before after
account x status 5 / 25 (one-to-one) 25 / 25
account x priority 20 / 20 19 / 20
account x type 20 / 20 20 / 20
account x origin 20 / 20 20 / 20
status x priority 20 / 20 20 / 20
status x type 20 / 20 18 / 20
status x origin 20 / 20 18 / 20
priority x type 4 / 16 (one-to-one) 16 / 16
priority x origin 4 / 16 (one-to-one) 16 / 16
type x origin 4 / 16 (one-to-one) 16 / 16
total 137 / 193 188 / 193

created_date was checked too. ageDays = 1 + (i % 30) over 30 rows makes the
day a bijection with the row index, so every axis is trivially a function of it;
the artifact that matters is a short fixed period. Before, all five axes
walked the calendar on a constant stride (accounts and statuses every 5th day,
priorities/types/origins every 4th). After, only crm_account does — see the
trade-off below.

Changes Made

  • status, priority, type and origin each advance one step per row plus
    a rotation
    , so no two axes stay in step. crm_account remains the anchor.
  • account x status is complete by construction, not by luck: account a
    takes rows a, a+5 … a+25, whose block numbers run 0…5, so its six statuses
    are (a + b) % 5 over b = 0…5 — all five. The 30 rows walk the whole 5 x 5
    grid, 25 cells once and five of them twice.
  • The priority / type / origin rotations are tuned, not derived. They
    were chosen by computing all ten cross-tabs and keeping the assignment with
    the fewest empty cells. The block comment says so, and says to re-measure if
    any list's length changes — there is no gate holding this (AGENTS.md:431).
  • Four duplicated index expressions collapsed into the locals now computed once
    (subject, crm_account, type, origin, sla_due_date all read them).
  • One changeset. Demo-data distribution is user-visible.

Why this is not just another coincidence

Two schemes that read as more principled than they measure were tried and
rejected on the numbers, and are recorded in the block comment so they are not
re-derived:

  • "Rotate the 5-lists on the 4-block and the 4-lists on the 5-block, the
    lengths are coprime so they cannot resonate."
    Measured: it empties one
    status completely (marginals 8,8,7,7,0) and drops status x priority to
    4 of 20 cells — worse than the defect.
  • "Rotate crm_account too, so it stops landing on every fifth day."
    Measured: costs twelve further empty cells (5 → 17 missing) across the other
    pairs. The every-fifth-day beat is the deliberate price of anchoring — with
    one row per day, some axis has to be the fixed one the other four are measured
    against. Stated beside the code per the deliberate-deviation rule.

Marginals are as flat as 30 rows allow and, for the two 5-element axes,
unchanged from before: 6/6/6/6/6 by account and by status, 8/7/7/8 by priority,
type and origin.

Second-order effect the card called out

Acme's related list held 8 open cases against a hand-written description
speaking of one worked ticket plus one billing dispute. Verified: 8 Acme rows, 2
hand-authored + 6 generated, and all 6 generated were new by construction.
After, Acme has 7 open cases spanning five different statuses; one generated row
is now legitimately closed. The hand-authored description is not touched —
it is true of the hand-authored set, and PR #1657 deliberately kept it.

Testing

  • Unit tests pass — pnpm verify green end to end
  • Linting passes
  • Build succeeds
  • New tests added — deliberately none (no new gate, per the card)

pnpm verify = validate && typecheck && lint && lint:i18n-gate && hygiene && hygiene:tokens && build && test, all eight stages green:

✓ Validation passed (1341ms)
✓ i18n lint gate: 0 `i18n/missing-*` issues
✓ source hygiene clean
✓ source token ratchet clean
✓ Build complete (1404ms)   — 5 author-time warnings, all pre-existing and unrelated
  Test Files  161 passed (161)
       Tests  3402 passed | 1 skipped (3403)

Partitions re-measured after the change, as the card requires:

test/seed-consistency.test.ts
test/seed-validation-warnings.test.ts
test/territory-seed-coverage.test.ts   (#638)
test/activity-seed-coverage.test.ts    (#671)
  before: Test Files 4 passed (4) · Tests 71 passed (71)
  after:  Test Files 4 passed (4) · Tests 71 passed (71)

No partition moved. The named assertions pass by name — partitions the seeded accounts across NA, EU and neither, populates each of the 30 / 60 / 90-day bands, stratifies the three churn tiles instead of repeating one number. The
stronger statement: the diff touches src/data/service.seed.ts and nothing
else, and inside it only the generated-case block, so the accounts, events and
tasks those partitions read are byte-identical to main. That also covers the
csm_flagged_accounts grouping PR #1657 just changed — its input,
src/data/sales.seed.ts, is untouched.

Additional Notes

Generated demo-case subjects change, because they name the row's priority and
type and those now vary independently (Demo case 06 — high feature_request
where it read medium bug). Subject is the upsert identity for these rows, so a
demo database seeded before this change keeps the old rows alongside the new
ones until pnpm demo:reset. Called out in the changeset.

is_sla_violated moves from 1 generated row to 2. The rule is unchanged; the
count moves because which rows are settled moved. seed-consistency's breach
assertion — flagged only when open and genuinely past due — still passes.

Checklist

  • I have added a changeset.changeset/decorrelate-the-case-volume-generator.md
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings
  • New and existing unit tests pass locally with my changes

🤖 Generated with Claude Code

https://claude.ai/code/session_018xtjdpZFjgWh4Ad9Wcx68J


Generated by Claude Code

The generator read ONE counter for every axis it varies: `crm_account`
and `status` both took `i % 5`, and `priority`, `type` and `origin` all
took `i % 4`. Equal-length lists walked by the same index advance in
lockstep, so account and status came out perfectly correlated — 5 of the
25 cross-tab cells occupied — and priority, type and origin were a
three-way lockstep holding 4 of 16 cells apiece. A generator whose whole
purpose is volume WITH variety was producing volume with none: filtering
the case list by account returned a single status every time.

Each list now advances one step per row PLUS a rotation, so no two axes
stay in step. `crm_account` is the anchor; `status` rotates one step per
pass through the accounts, which makes account × status complete by
construction rather than by luck — account `a` takes rows a, a+5 … a+25,
so its six statuses are `(a + b) % 5` over b = 0…5, all five of them.
The `priority`, `type` and `origin` rotations are tuned, not derived:
they were chosen by computing all ten pairwise cross-tabs and keeping
the assignment that left the fewest empty cells. The block comment
records that, and says to re-measure if a list's length changes.

Measured over the shipped lists: pairwise cell occupancy 137/193 before,
188/193 after; no cell holds more than 3 of the 30 rows; marginals stay
flat at 6/6/6/6/6 by account and by status and 8/7/7/8 by priority, type
and origin. Every account now shows all five statuses.

The eight hand-authored cases are untouched, as are the seeded accounts,
events and tasks. The #638 territory partition and the #671 activity
bands are unmoved.

Co-authored-by: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018xtjdpZFjgWh4Ad9Wcx68J
@vercel

vercel Bot commented Sep 6, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
hotcrm Ignored Ignored Sep 6, 2026 8:48am UTC

Request Review

@github-actions github-actions Bot added the metadata Declarative metadata — schema, security posture, UI surfaces label Sep 6, 2026
@os-steve
os-steve marked this pull request as ready for review September 6, 2026 08:53
@os-steve
os-steve added this pull request to the merge queue Sep 6, 2026
Merged via the queue into main with commit fa637d9 Sep 6, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

metadata Declarative metadata — schema, security posture, UI surfaces

Projects

None yet

Development

Successfully merging this pull request may close these issues.

The 30-case demo generator indexes accounts and statuses with the same i % 5, so every account has exactly one status — no account shows a mix

2 participants