Skip to content

Optimizes database, enhances indexing, and streamlines 4.x upgrades - #37

Open
arotolo wants to merge 8 commits into
mainfrom
feature/optimize-sampling-clean
Open

arotolo wants to merge 8 commits into
mainfrom
feature/optimize-sampling-clean

Conversation

@arotolo

@arotolo arotolo commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

This release significantly improves database performance, upgrade reliability, and indexing behavior by addressing long-standing issues with planner statistics, introducing deferred optimization, and refining connection management.

Key Improvements:

  • Eliminates Planner Statistics for Stable Performance: Removes problematic SQLite planner statistics, which often led to performance regressions on large types. SQLite's default heuristics are now relied upon, providing consistent and fast query plans. Connect operations also no longer unnecessarily rewrite database page 1.
  • Introduces Deferred Optimization for 4.x Upgrades: Adds an autoOptimize constructor parameter (default true) and Optimize()/OptimizeAsync() methods. This allows developers to defer time-consuming one-time upgrade tasks (like dropping redundant 4.x indexes, rebuilding partially indexed ones, and compacting the database) to a background thread, preventing UI freezes on app launch.
  • Enhances Indexing Logic and Migration: CreateIndex now intelligently detects and rebuilds indexes when their DDL definition mismatches, rather than silently keeping stale 4.x indexes. It also correctly drops orphaned legacy indexes from 4.x rollbacks, improving query efficiency and reducing write overhead.
  • Refines Database Compaction with File Swap: The Cleanup(vacuum: true) and Optimize() methods now use a more robust VACUUM INTO strategy. This compacts the database by writing a single copy to a temporary file and atomically swapping it, significantly reducing the required free disk space compared to a full VACUUM and handling interruptions gracefully.
  • Improves Connection Lifecycle Reliability: Critical operations like Connect, Disconnect, Dispose, and Backup now acquire the connection gate, ensuring they run serially and prevent race conditions that could lead to connection issues or file locks.
  • Strengthens Database File Management: Ensures the connection pool is cleared before a database file is deleted (e.g., when rebuildCache is enabled), preventing issues where subsequent operations might incorrectly target a deleted file.
  • Updates Documentation and Versioning: Provides comprehensive updates to the changelog, README, and dedicated documentation files to clearly explain the new features, optimizations, and upgrade considerations for version 5.4.0.

@arotolo
arotolo requested a review from michaelstonis November 6, 2026 23:29
@arotolo arotolo added the enhancement New feature or request label Nov 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Development

Successfully merging this pull request may close these issues.

2 participants