Skip to content

enable cache_db auto compaction#158

Merged
wiz merged 1 commit into
mempoolfrom
mononaut/cache-db-compaction
Jul 12, 2026
Merged

enable cache_db auto compaction#158
wiz merged 1 commit into
mempoolfrom
mononaut/cache-db-compaction

Conversation

@mononaut

Copy link
Copy Markdown
Contributor

the shared rocksdb store constructor disables auto compaction to speed up initial indexing:

db_opts.set_disable_auto_compactions(true); // for initial bulk load

and then we have to explicitly re-enable it on each store instance later:

start_fetcher(self.from, &daemon, to_add)?
.map(|blocks| self.add(&blocks, Operation::AddBlocks));
self.start_auto_compactions(&self.store.txstore_db);

start_fetcher(self.from, &daemon, to_index)?
.map(|blocks| self.index(&blocks, Operation::AddBlocks));
self.start_auto_compactions(&self.store.history_db);

but when the cache_db was added in [2699560], apparently auto compaction was never re-enabled for that store.

this means

  • the rocksdb L0 SST files for the cache_db grow unbounded every time we flush the cache_db memtable to disk, as we do after every precache run.
  • point lookups for scripthashes not in the cache_db (i.e. most /address/:address queries) get more and more expensive, since they have to search every L0 SST file to confirm there are no hits.
    • lookups for scripthashes that are in the cache are still fast, since a) they're usually still in the memtable, and b) if not, the latest entry is probably in one the first SST files we check.

in normal operation this isn't very noticeable because the cache_db is small and rarely flushed, but if you did something silly like reboot electrs and re-run the precache every hour then read performance for less popular addresses could eventually become severely degraded.

@wiz wiz merged commit bf622e5 into mempool Jul 12, 2026
12 checks passed
@wiz wiz deleted the mononaut/cache-db-compaction branch July 12, 2026 03:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants