Skip to content

fix: release ProjectCache contents after snapshot load#214

Open
octo-patch wants to merge 1 commit intojustrach:mainfrom
octo-patch:fix/issue-210-project-cache-memory-leak
Open

fix: release ProjectCache contents after snapshot load#214
octo-patch wants to merge 1 commit intojustrach:mainfrom
octo-patch:fix/issue-210-project-cache-memory-leak

Conversation

@octo-patch
Copy link
Copy Markdown

Fixes #210

Problem

ProjectCache.get() in src/mcp.zig loads snapshots for non-default repos but never calls releaseContents() or releaseSecondaryIndexes() on the cached Explorer. This retains raw file contents in memory for the lifetime of each cache entry. On a ~22K file repo, this causes ~4.5GB RSS from a single cached project.

Solution

After loadSnapshot succeeds, release file contents that are not needed for query serving (the snapshot provides outlines + trigram index + word index, which is sufficient for all MCP query tools). Mirrors the identical pattern already used by the primary explorer in main.zig scanBg().

Testing

The fix mirrors the existing main.zig pattern exactly (fc > 1000 or CODEDB_LOW_MEMORY). Verified by code inspection that releaseContents and releaseSecondaryIndexes are safe to call after snapshot load — the snapshot contains outlines and indexes sufficient for all query tools.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a5ae293506

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +127 to +130
const fc = new_entry.explorer.outlines.count();
if (fc > 1000 or std.process.hasEnvVarConstant("CODEDB_LOW_MEMORY")) {
new_entry.explorer.releaseContents();
new_entry.explorer.releaseSecondaryIndexes();
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Initialize project root before dropping cached contents

This new releaseContents() path causes a functional regression for non-default projects: once contents are released, codedb_read/codedb_search must read files from disk, but ProjectCache.get() never sets new_entry.explorer's root directory. In Explorer.readContentForSearch, disk fallback uses self.root_dir or cwd, so for project values outside the server working directory, reads fail (file-not-found) or searches silently miss matches. This is triggered whenever fc > 1000 or CODEDB_LOW_MEMORY is set.

Useful? React with 👍 / 👎.

@justrach
Copy link
Copy Markdown
Owner

justrach commented Apr 8, 2026

Thanks for the PR. This is a useful fix.

There are still a few Codex review comments to address before merge.

Can you please go through that feedback and push the requested changes?

Once you have updated the PR, tag @codex so it can re-review the branch.

Repository owner deleted a comment from chatgpt-codex-connector bot Apr 8, 2026
Copy link
Copy Markdown
Owner

@justrach justrach left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. This is a useful direction for the ProjectCache retention issue.

One blocking change before merge: releaseContents() cannot drop cached file contents before the cache has initialized the project root on the embedded explorer. On the current head, once contents are released, codedb_read / codedb_search fall back to disk reads via an explorer whose root is still unset, so non-default project roots can read relative to cwd and fail or silently miss matches.

Please make sure the project root is set on the cache entry before the contents-release path can run, then tag @codex review again once that is updated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: ProjectCache retains raw file contents — 4.5GB leak on large repos

2 participants