fix: release ProjectCache contents after snapshot load#214
fix: release ProjectCache contents after snapshot load#214octo-patch wants to merge 1 commit intojustrach:mainfrom
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a5ae293506
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| const fc = new_entry.explorer.outlines.count(); | ||
| if (fc > 1000 or std.process.hasEnvVarConstant("CODEDB_LOW_MEMORY")) { | ||
| new_entry.explorer.releaseContents(); | ||
| new_entry.explorer.releaseSecondaryIndexes(); |
There was a problem hiding this comment.
Initialize project root before dropping cached contents
This new releaseContents() path causes a functional regression for non-default projects: once contents are released, codedb_read/codedb_search must read files from disk, but ProjectCache.get() never sets new_entry.explorer's root directory. In Explorer.readContentForSearch, disk fallback uses self.root_dir or cwd, so for project values outside the server working directory, reads fail (file-not-found) or searches silently miss matches. This is triggered whenever fc > 1000 or CODEDB_LOW_MEMORY is set.
Useful? React with 👍 / 👎.
|
Thanks for the PR. This is a useful fix. There are still a few Codex review comments to address before merge. Can you please go through that feedback and push the requested changes? Once you have updated the PR, tag |
justrach
left a comment
There was a problem hiding this comment.
Thanks for the PR. This is a useful direction for the ProjectCache retention issue.
One blocking change before merge: releaseContents() cannot drop cached file contents before the cache has initialized the project root on the embedded explorer. On the current head, once contents are released, codedb_read / codedb_search fall back to disk reads via an explorer whose root is still unset, so non-default project roots can read relative to cwd and fail or silently miss matches.
Please make sure the project root is set on the cache entry before the contents-release path can run, then tag @codex review again once that is updated.
Fixes #210
Problem
ProjectCache.get()insrc/mcp.zigloads snapshots for non-default repos but never callsreleaseContents()orreleaseSecondaryIndexes()on the cachedExplorer. This retains raw file contents in memory for the lifetime of each cache entry. On a ~22K file repo, this causes ~4.5GB RSS from a single cached project.Solution
After
loadSnapshotsucceeds, release file contents that are not needed for query serving (the snapshot provides outlines + trigram index + word index, which is sufficient for all MCP query tools). Mirrors the identical pattern already used by the primary explorer inmain.zigscanBg().Testing
The fix mirrors the existing
main.zigpattern exactly (fc > 1000 or CODEDB_LOW_MEMORY). Verified by code inspection thatreleaseContentsandreleaseSecondaryIndexesare safe to call after snapshot load — the snapshot contains outlines and indexes sufficient for all query tools.