feat(schema): ClickHouse index algorithms and engine SETTINGS

[lohanidamodar]

Summary

Adds two ClickHouse-only DDL features to the Schema builder so real ClickHouse-backed services can express the table layout they need:

1. Index algorithm selection. Every ClickHouse INDEX is parameterised by a TYPE (minmax / set / bloom_filter / ngrambf_v1 / tokenbf_v1 / inverted) and a GRANULARITY. The schema previously hard-coded TYPE minmax GRANULARITY 3 for every index() call, which is the wrong choice for any high-cardinality column. Table::index() now accepts an optional algorithm, algorithmArgs, and granularity so callers can pick the correct algorithm per column.

2. Engine SETTINGS clause. CREATE TABLE … SETTINGS k=v, … was unreachable. Real ClickHouse usage needs index_granularity, allow_nullable_key, etc. — Table::settings() now emits this after TTL in the rendered DDL.

The pattern follows engine() / ttl() precisely: ClickHouse-only DDL features live on Table and are silently ignored by the other dialect compilers.

Why

Both utopia-php/audit and utopia-php/usage currently hand-roll their ClickHouse setup() routines, emitting INDEX … TYPE bloom_filter GRANULARITY 1 and SETTINGS index_granularity = 8192, allow_nullable_key = 1 directly. Without these features the schema builder cannot express what those services already produce, so they cannot delegate setup() here.

API

use Utopia\Query\Schema\ClickHouse as Schema;
use Utopia\Query\Schema\ClickHouse\IndexAlgorithm;
use Utopia\Query\Schema\Table;

(new Schema())->create('events', function (Table $table) {
    $table->bigInteger('id')->primary();
    $table->string('user_id');
    $table->string('country');
    $table->string('text');

    // No algorithm specified → default `TYPE minmax GRANULARITY 3`
    $table->index(['id']);

    // BloomFilter — high-cardinality strings with `=` / `IN`
    $table->index(['user_id'], algorithm: IndexAlgorithm::BloomFilter);

    // Set(N) — small fixed value sets, custom granularity
    $table->index(['country'], algorithm: IndexAlgorithm::Set, algorithmArgs: [100], granularity: 4);

    // NgramBloomFilter(n, size_bytes, hashes, seed) — text search
    $table->index(['text'], algorithm: IndexAlgorithm::NgramBloomFilter, algorithmArgs: [4, 1024, 3, 0]);

    $table->settings([
        'index_granularity' => 8192,
        'allow_nullable_key' => true, // booleans become 1 / 0
    ]);
});

// CREATE TABLE `events` (
//   `id` Int64, `user_id` String, `country` String, `text` String,
//   INDEX `idx_id` `id` TYPE minmax GRANULARITY 3,
//   INDEX `idx_user_id` `user_id` TYPE bloom_filter GRANULARITY 1,
//   INDEX `idx_country` `country` TYPE set(100) GRANULARITY 4,
//   INDEX `idx_text` `text` TYPE ngrambf_v1(4, 1024, 3, 0) GRANULARITY 1
// )
//   ENGINE = MergeTree() ORDER BY (`id`)
//   SETTINGS index_granularity = 8192, allow_nullable_key = 1

Indexes can also be added in alter() callbacks — they render as ALTER TABLE … ADD INDEX …. MinMax and Inverted reject algorithmArgs (no parenthesised args in DDL); float algorithmArgs and settings values use sprintf('%F', ...) so 1.0e-5 becomes 0.00001 instead of the rejected 1.0E-5.

Files

• src/Query/Schema/ClickHouse/IndexAlgorithm.php — new enum (6 algorithms).
• src/Query/Schema/Index.php — adds optional algorithm, algorithmArgs, granularity fields. Validates the no-args-on-MinMax/Inverted rule, granularity ≥ 1, and at-least-one-column. The strict identifier regex on name only fires when algorithm is set, since other dialects quote index names (so hyphens / dots are valid there).
• src/Query/Schema/Table.php — index() and uniqueIndex() accept the new fields. Auto-generated index names sanitise non-identifier characters in column names, so event-type produces idx_event_type. Adds settings() with a conservative key/value allow-list and float-safe formatting.
• src/Query/Schema/ClickHouse.php — single compileSkipIndex(Index) helper used by both create() and alter(). SETTINGS rendered after TTL. alter() rejects SETTINGS (ClickHouse doesn't accept it in ALTER TABLE; callers must emit ALTER TABLE … MODIFY SETTING directly).

Test plan

• composer lint passes
• composer check (PHPStan max) passes
• 16 new ClickHouse schema tests covering each algorithm shape, composite columns, alter ADD INDEX, all validation paths, float formatting, the algorithm-scoped name regex, and SETTINGS edge cases
• Full Query test suite: 5086/5086 pass

🤖 Generated with Claude Code

[github-actions]

📊 Coverage

Metric	PR	Baseline	Δ
Lines	91.89% (7046/7668)	91.92%	-0.03%
Methods	84.70% (1041/1229)	84.80%	-0.10%
Classes	62.50% (105/168)	62.87%	-0.37%

Full per-file breakdown in the job summary.

[greptile-apps]

Greptile Summary

This PR adds two ClickHouse-only DDL features to the schema builder: per-index algorithm selection (minmax / set / bloom_filter / ngrambf_v1 / tokenbf_v1 / inverted with optional args and granularity) and a SETTINGS k=v, … clause rendered after TTL. All previously flagged issues from earlier review rounds have been addressed in this revision — the name-regex is correctly gated on algorithm !== null, no-arg algorithms reject algorithmArgs, float formatting uses sprintf('%F') throughout, alter() guards against non-skip index types and rejects SETTINGS, and index() retains its static return type for fluent chaining.

Confidence Score: 5/5

Safe to merge — no P0 or P1 issues found; all previously flagged defects have been resolved.

Every concern from prior review rounds is addressed in the current diff. The single remaining finding is a minor README/code discrepancy on the string-value regex quantifier (* vs +), which is P2-only and does not affect runtime behaviour.

No files require special attention.

Important Files Changed

Filename	Overview
src/Query/Schema/ClickHouse.php	compileSkipIndex() consolidates CREATE and ALTER paths; null-algorithm falls back to GRANULARITY 3; float args use sprintf('%F'); ALTER correctly rejects non-skip index types and SETTINGS.
src/Query/Schema/Table.php	index() extended with algorithm/algorithmArgs/granularity and kept returning static; autoIndexName() sanitises column names; settings() validates keys/values and stores pre-formatted strings.
src/Query/Schema/Index.php	Adds optional algorithm/algorithmArgs/granularity fields with correct validation: name regex gated on algorithm!=null, no-arg algorithms reject algorithmArgs, granularity >= 1 enforced.
src/Query/Schema/ClickHouse/IndexAlgorithm.php	New enum with 6 ClickHouse skip-index algorithm values; straightforward and correct.
src/Query/Schema/Column.php	Forwards new index() parameters and adds settings() proxy to Table; consistent with existing delegation pattern.
tests/Query/Schema/ClickHouseTest.php	16 new tests covering all algorithms, composite columns, ALTER ADD INDEX, validation paths, float formatting, name-regex scoping, and SETTINGS edge cases.
README.md	Documents new API accurately; minor discrepancy: string-value pattern shown as * (allows empty) but code enforces + (requires non-empty).

_{Reviews (14): Last reviewed commit: "fix(schema): restore missing brace in se..." | Re-trigger Greptile}

[abnegate]

@copilot Rebase

[Copilot]

@copilot Rebase

Done — merged main into the branch in 34b2353. All merge conflicts resolved (the $blueprint → $table rename from main, the fluent builder methods, and the updated index() signature).

[abnegate]

@copilot address the greptile comment

[Copilot]

@copilot address the greptile comment

Fixed in a3fcc2b — restored ): static { return type and return $this; on Table::index() so it matches uniqueIndex(), fulltextIndex(), and spatialIndex() and doesn't break fluent chaining.

[abnegate]

@copilot address the greptile comment

[Copilot]

@copilot address the greptile comment

Fixed in b1df44e — both the compileAlter and compileCreate index loops now guard against non-IndexType::Index entries, throwing UnsupportedException instead of silently rendering wrong DDL for uniqueIndex(), fulltextIndex(), or spatialIndex() calls.

[abnegate]

@copilot Fix the readme to use builder style for new additions instead of callback

[Copilot]

@copilot Fix the readme to use builder style for new additions instead of callback

Updated in d60af4c — both the skip-index and SETTINGS code blocks now use $schema->table('events')->...->create() builder style, consistent with the engine and TTL examples above them.

[abnegate]

@copilot fix unit test and PHPstan checks

[Copilot]

@copilot fix unit test and PHPstan checks

Fixed in 16ddd87 — all skip-index and SETTINGS tests were using a non-existent callback API ($schema->create('events', function (Table $table) {...})). Rewrote them to use the fluent builder style ($schema->table('events')->...->create()), which resolves both the PHPStan errors (undefined method, unknown class) and the unit test failures.