feat: add volume workspace

[NeurekaSoftware]

Checklist

• This PR is not opened from my fork’s main branch
• All new user-facing strings are translated via Paraglide (m.*())

What This PR Implements

Brings the Projects-style file browser and text editor to Docker volumes. File changes are staged and saved together through the volume workspace.

Fixes #3247

Changes Made

• Added revision-safe APIs for browsing, reading, updating, and downloading volume files.
• Added create, edit, rename, move, upload, download, restore, and delete operations.
• Reused shared workspace components from Projects.
• Added rollback, permission checks, path safety, configurable tree limits, and edge-agent support.
• Added Paraglide messages and focused backend tests.

Testing Done

• Manually verified the feature in production.
• Frontend type checking and formatting passed.
• Frontend production build passed.
• Focused Go tests for the volume workspace and related helpers passed.

AI Tool Used (if applicable)

Codex 5.6 Sol (Extra High) — assisted with implementation, testing, code review, and drafting this PR description. The completed feature was manually verified in production.

Additional Context

None.

Disclaimer Greptiles Reviews use AI, make sure to check over its work.

To better help train Greptile on our codebase, if the comment is useful and valid Like the comment, if its not helpful or invalid Dislike

To have Greptile Re-Review the changes, mention greptileai.

Greptile Summary

The PR adds a revision-safe workspace for browsing and editing Docker volume files, reusing the project workspace UI and coordinating volume mutations through a per-volume lock.

• Adds recursive workspace APIs, transactional file operations, rollback, path validation, permissions, and configurable tree limits.
• Serializes workspace saves, legacy mutations, backup creation, and restores against the same volume.
• Adds shared frontend workspace components and a volume-specific editor workflow.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the previously reported revision, legacy-mutation, backup-consistency, and response-snapshot races are now serialized by the same per-volume lock over their full critical sections.

Reviews (6): Last reviewed commit: "fix(volumes): return workspace from lock..." | Re-trigger Greptile

Context used (5)

• Knowledge Base — Backend API Server
• Knowledge Base — Container & Image Lifecycle
• Knowledge Base — Backend Bootstrap, Configuration, and Dependency Injection
• Knowledge Base — Frontend application (SvelteKit)
• Knowledge Base — Shared Types (types/)

Disclaimer Greptiles Reviews use AI, make sure to check over its work.

To better help train Greptile on our codebase, if the comment is useful and valid Like the comment, if its not helpful or invalid Dislike

To have Greptile Re-Review the changes, mention greptileai.

Greptile Summary

The PR adds a revision-safe Docker volume workspace with shared browsing and editing behavior. The latest changes serialize workspace reads, updates, legacy mutations, backup creation, and restores around the same per-volume state.

• Adds volume workspace tree, content, download, and transactional update APIs.
• Reuses shared workspace types and frontend components across projects and volumes.
• Adds common per-volume read/write locking, rollback behavior, path validation, permissions, migrations, and focused tests.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the previously reported revision, legacy-mutation, backup-consistency, response-snapshot, and intermediate-read races are protected by the same per-volume lock over the relevant critical sections.

_{Reviews (10): Last reviewed commit: "Merge remote-tracking branch 'origin/mai..." | Re-trigger Greptile}

Context used (3)

• Knowledge Base — Backend API Server
• Knowledge Base — Container & Image Lifecycle
• Knowledge Base — Backend Bootstrap, Configuration, and Dependency Injection

[NeurekaSoftware]

@greptileai

[NeurekaSoftware]

@greptileai

[NeurekaSoftware]

@greptileai

[NeurekaSoftware]

@kmendell I'd be eternally grateful if you took a look at this when you have some extra time. :)

[kmendell]

this is a nit pick on my part but just have codex move tests into the same volumes_test file i dont like when there random files names , call me crazy :D

[NeurekaSoftware]

I have OCD, I get it. :)

[NeurekaSoftware]

@kmendell should the workspace service and workspace service tests also be moved into the volume service and volume tests?

[NeurekaSoftware]

@greptileai

[NeurekaSoftware]

@greptileai

[NeurekaSoftware]

@kmendell Codex is recommending we add a small native Rust-based scanner to the Arcane tools image.

The current BusyBox build runs stat for every entry and doesn’t have the batching features needed to make this fast. My larger volume takes around 6.5 seconds to scan and 13 seconds to save.

Saving scans twice: once to check that the volume hasn’t changed since it was loaded, and again to return the updated tree and revision after saving.

A native scanner could collect the same metadata in one process without changing the API or removing these safety checks.

[kmendell]

ill have to look and see whats out there.

[kmendell]

Im not a expert on rust i really only know how to read/write go code lol, ive dabbled in rust a bit but, ill see if i can somehow create a go version that walks directories faster and more efficiently

[NeurekaSoftware]

Im not a expert on rust i really only know how to read/write go code lol, ive dabbled in rust a bit but, ill see if i can somehow create a go version that walks directories faster and more efficiently

I get it, haha. I figured we could follow up in a second PR later to address performance.

For the moment, this is perfect for those that need to edit settings in a named volume. Thank you so much for being so open to contributions!

[NeurekaSoftware]

WIP: Fully Align Project and Volume Workspaces

Summary

• Make Workspace the canonical feature name across APIs, DTOs, services, handlers, frontend modules, permissions, query keys, activities, settings, and UI text.
• Remove the entire Volume /browse/* API. Its only production caller is Workspace download, which moves to the canonical Workspace route.
• Retain Build Workspace /builds/browse/* because it is actively used and is outside the Project/Volume refactor.
• Project Workspace manages auxiliary files only; Compose, override, and .env remain project configuration.
• Use multipart, revision-checked, multi-file operations for both domains.
• Replace old APIs and names immediately, without compatibility routes or field aliases.

Canonical API

Provide identical route structures:

• GET /api/environments/{id}/projects/{projectId}/workspace
• GET /api/environments/{id}/projects/{projectId}/workspace/file?relativePath=...
• GET /api/environments/{id}/projects/{projectId}/workspace/file/download?relativePath=...
• PUT /api/environments/{id}/projects/{projectId}/workspace
• Equivalent routes under /volumes/{volumeName}/workspace.

Both PUT endpoints accept multipart/form-data:

• Required manifest JSON part containing fileTreeRevision and fileChanges.
• Repeated files parts.
• create_file and update_file require a unique uploadIndex.
• Remove inline content; all writes use multipart file parts.
• restore_file and backupId remain a Volume-only extension.
• Reject missing, duplicate, unused, or out-of-range upload indices.

Convert POST /projects to atomic multipart creation:

• Required project JSON part containing name, composeContent, and optional envContent.
• Required manifest JSON part containing initial workspace operations.
• Repeated files parts mapped through uploadIndex.
• Validate Compose and every workspace operation before committing the directory or database row.

Remove:

• Project /files, /file, and /includes.
• Volume /files, /file, and every /browse/* route.
• Workspace fields from Project create/update/details DTOs.
• The special Project include-update DTO and handler.

Upload Limits and Behavior

Add paired environment variables:

• PROJECT_WORKSPACE_MAX_FILE_SIZE_MB
• VOLUME_WORKSPACE_MAX_FILE_SIZE_MB

Contract:

• Both default to 10.

• Values are interpreted as MiB-sized units: value × 1024 × 1024 bytes.

• Zero or negative values fall back to 10.

• Malformed non-numeric values follow normal configuration-load failure behavior.

• Each value controls its domain’s upload limit, editor content-loading limit, and read-only threshold.

• Backend enforcement remains authoritative.

• Expose effective values as read-only runtime settings:

  • projectWorkspaceMaxFileSizeMb
  • volumeWorkspaceMaxFileSizeMb

• The existing settings-loading path supplies these values to the frontend, including remote environments.

• New-project, Project Workspace, and Volume Workspace UIs use the effective runtime value instead of a hardcoded constant.

• Files must still be valid UTF-8 text without NUL bytes to be uploaded or edited.

• Existing binary, symlink, special, or oversized files remain visible and downloadable but read-only.

• Error messages display the configured domain limit dynamically.

Retain and rename the paired traversal settings:

• PROJECT_WORKSPACE_MAX_DEPTH, default 20
• PROJECT_WORKSPACE_MAX_ENTRIES, default 2000
• VOLUME_WORKSPACE_MAX_DEPTH, default 50
• VOLUME_WORKSPACE_MAX_ENTRIES, default 10000

Remove the old *_FILE_TREE_* environment names without aliases.

Canonical File Structure

Shared:

• C:\Users\Neureka\Git\GitHub\NeurekaSoftware\Arcane\types\workspace\workspace.go
• C:\Users\Neureka\Git\GitHub\NeurekaSoftware\Arcane\backend\pkg\workspace\workspace.go
• C:\Users\Neureka\Git\GitHub\NeurekaSoftware\Arcane\backend\api\handlers\workspace.go
• C:\Users\Neureka\Git\GitHub\NeurekaSoftware\Arcane\frontend\src\lib\types\workspace.ts
• C:\Users\Neureka\Git\GitHub\NeurekaSoftware\Arcane\frontend\src\lib\utils\workspace-files.ts
• C:\Users\Neureka\Git\GitHub\NeurekaSoftware\Arcane\frontend\src\lib\components\workspace-file-tree-panel.svelte

Project:

• C:\Users\Neureka\Git\GitHub\NeurekaSoftware\Arcane\types\project\workspace.go
• C:\Users\Neureka\Git\GitHub\NeurekaSoftware\Arcane\backend\api\handlers\project_workspace.go
• C:\Users\Neureka\Git\GitHub\NeurekaSoftware\Arcane\backend\internal\services\project_workspace_service.go
• C:\Users\Neureka\Git\GitHub\NeurekaSoftware\Arcane\backend\pkg\projects\workspace.go
• C:\Users\Neureka\Git\GitHub\NeurekaSoftware\Arcane\frontend\src\lib\types\project-workspace.ts
• C:\Users\Neureka\Git\GitHub\NeurekaSoftware\Arcane\frontend\src\lib\services\project-workspace-service.ts
• C:\Users\Neureka\Git\GitHub\NeurekaSoftware\Arcane\frontend\src\routes\(app)\projects\components\project-workspace-utils.ts

Volume:

• C:\Users\Neureka\Git\GitHub\NeurekaSoftware\Arcane\types\volume\workspace.go
• C:\Users\Neureka\Git\GitHub\NeurekaSoftware\Arcane\backend\api\handlers\volume_workspace.go
• C:\Users\Neureka\Git\GitHub\NeurekaSoftware\Arcane\backend\internal\services\volume_workspace_service.go
• C:\Users\Neureka\Git\GitHub\NeurekaSoftware\Arcane\backend\pkg\dockerutil\volume_workspace.go
• C:\Users\Neureka\Git\GitHub\NeurekaSoftware\Arcane\frontend\src\lib\types\volume-workspace.ts
• C:\Users\Neureka\Git\GitHub\NeurekaSoftware\Arcane\frontend\src\lib\services\volume-workspace-service.ts
• C:\Users\Neureka\Git\GitHub\NeurekaSoftware\Arcane\frontend\src\routes\(app)\volumes\components\volume-workspace-utils.ts

Delete or retire:

• types\volume\browse.go
• frontend\src\lib\types\project-files.ts
• frontend\src\routes\(app)\projects\components\project-file-tree-utils.ts
• backend\pkg\projects\project_files.go
• The unused GenericFileBrowser.svelte component after moving its actively used Build Workspace type/sort exports to a small non-component module.

Move the shared FileEntry DTO out of the Volume package so the actively used Build Workspace handler no longer imports a Volume Browser type.

Legacy Removal and Migration

Remove Volume Browser implementation:

• Delete directory-list, preview, download, upload, mkdir, and delete browser handlers.
• Delete their corresponding VolumeService methods and tests.
• Delete obsolete FileMetadata and browser request/response types.
• Delete volume.browse.* edge commands and tests.
• Remove the browser → workspace tab alias.
• Remove browser-only activities and stop emitting legacy Volume file events; retain historical event decoding for existing database records.

Clean up infrastructure naming while preserving the helper implementation used by Workspace and backup restore:

• Rename isBrowsableVolumeInternal to a workspace/helper-support validation name.
• Rename createTempContainerInternal to reflect acquiring a reusable Volume helper.
• Replace browser wording in errors, logs, comments, and scheduler descriptions.
• Rename setting key and DTO field from volumeBrowserHelperIdleTimeout to volumeHelperIdleTimeout.
• Add matching SQLite and PostgreSQL migration 071_rename_volume_workspace_legacy_keys.sql to preserve the stored timeout value under the new key.
• Keep detection and cleanup of old unlabeled helper containers, but rename it as unlabeled-helper cleanup rather than legacy-browser compatibility.

Remove the obsolete volumes:browse permission:

• Workspace list/read/download requires volumes:read, matching Projects.
• Volume mutations retain operation-specific volumes:upload, volumes:delete, and volumes:backup checks.
• Migrate persisted custom-role and API-key grants from volumes:browse to volumes:read, deduplicating grants.
• Remove the permission from the catalog, built-in role definitions, frontend checks, and matcher tests.

Other cleanup:

• Rename Project/Volume query keys from .files/.file to .workspace/.workspaceFile.
• Rename edge commands symmetrically to project.workspace.* and volume.workspace.*.
• Replace Project Files, Volume Browser, and generic file_browser_* UI keys with shared workspace_* keys or domain-specific projects_workspace_* / volumes_workspace_* keys.
• Replace project-specific variables and DOM IDs inside the shared panel with workspace-neutral names.
• Migrate the Swarm stack editor away from Project-named helper aliases to shared workspace helpers.
• Update the CLI update-includes implementation to use a revision-checked multipart Project Workspace update while retaining the include-focused command behavior.
• Leave Compose filesystem helpers such as ReadProjectFiles and ProjectFileCandidates unchanged because they describe actual Compose project files, not the removed Workspace feature naming.

Project Save Flow

Keep one Save action:

1. Validate all drafts.
2. Save Project Workspace changes first.
3. Rebase the returned workspace entries and revision immediately.
4. Save Project configuration through the normal JSON Project endpoint.
5. If workspace save fails, retain all drafts and do not save configuration.
6. If configuration save fails, retain only configuration drafts and report that workspace changes were already committed.

Project Workspace excludes protected Compose, override, and environment files from both its entries and revision hash, so configuration changes do not invalidate workspace revisions.

Test Plan

• Shared tests for multipart parsing, upload-index integrity, UTF-8/NUL validation, dynamic size limits, safe paths, revisions, collisions, and read-only classification.
• Configuration tests for defaults, positive overrides, zero/negative fallback, malformed values, schema generation, runtime-settings exposure, and remote-environment forwarding.
• Mirrored Project and Volume handler/service tests for list, read, download, multi-upload, create/update/folder/rename/move/delete, stale revisions, permissions, and truncation.
• Project tests for protected-file exclusion, multipart atomic creation, GitOps/archive restrictions, include updates through Workspace, and partial-save recovery.
• Volume tests for restore operations, binary/oversized download, helper reuse, and backup compatibility.
• Migration tests for the helper setting and volumes:browse role/API-key grants.
• Assert every removed Project/Volume legacy route is unregistered and every obsolete edge command is absent.
• Confirm Build Workspace /builds/browse/* remains registered and functional.
• Run backend, CLI, and shared-types Go tests; frontend check/format/build; test TypeScript checks; and manual development-environment verification with both default and overridden upload limits.

[NeurekaSoftware]

@kmendell I’m not sure how you feel about the proposed changes above. The commit can be reverted or adjusted if you see any issues with the approach.

I felt the existing implementation was becoming difficult to follow because we were duplicating functionality, and the API contract was starting to become somewhat messy as a result.

I know we all tend to have strong opinions when developing software, so I’ll leave the final direction up to your discretion. I’ll try to get the rest of my current work pushed shortly.

[NeurekaSoftware]

Summary

This commit makes Workspace the canonical file-management feature for both Projects and Volumes.

Highlights

• Added matching Project and Volume Workspace APIs for listing, reading, downloading, and multipart file operations.
• Added atomic multipart Project creation with revision-checked workspace changes.
• Added configurable upload limits:
  • PROJECT_WORKSPACE_MAX_FILE_SIZE_MB
  • VOLUME_WORKSPACE_MAX_FILE_SIZE_MB
  • Both default to 10 MiB and are exposed through runtime settings.
• Consolidated shared Workspace DTOs, utilities, components, query keys, permissions, activities, and edge commands.
• Removed legacy Project file/include routes and the Volume Browser /browse/* implementation.
• Removed volumes:browse, migrating existing grants to volumes:read.
• Renamed the Volume helper timeout setting and added SQLite/PostgreSQL migrations.
• Retained Build Workspace /builds/browse/*, which is outside this refactor.
• Updated the CLI update-includes flow to use the multipart Project Workspace API.

Breaking changes

Legacy Project/Volume routes, DTO fields, edge commands, frontend modules, permissions, and old *_FILE_TREE_* environment variables were removed without compatibility aliases.

Validation

Backend builds and focused Workspace, configuration, migration, authorization, and route tests passed. CLI compilation, shared-types tests, frontend checks, formatting, build, and test TypeScript checks also passed.

Full backend lint was waived because of an unrelated pre-existing shimbad finding in lifecycle_path_diagnostics_other.go.

[NeurekaSoftware]

@greptileai

[NeurekaSoftware]

@greptileai

[depot-code-access]

This pull request has merge conflicts. Please resolve the conflicts so the PR can stay up-to-date and reviewed.

[kmendell]

I am hoping to get this and the s3 PR merged this weekend , just been a bit busy.

[NeurekaSoftware]

I am hoping to get this and the s3 PR merged this weekend , just been a bit busy.

I haven't tested with the major refactoring I did because it requires a migration, which I wanted to avoid running on prod until I get the okay from you.

Let me know when you have time and I'll deploy the changes to prod to make sure there is no regression.

[kmendell]

Do not be alarmed about the conflicts that are about to happen on here lol, I will fix them there is going to be alot i restructured the netire backend to clean stuff up and make it easier to maintain

[depot-code-access]

This pull request has merge conflicts. Please resolve the conflicts so the PR can stay up-to-date and reviewed.

[kmendell]

LGTM, Thanks!

[kmendell]

I have a native Go binary that ill addto speed up the files stuff, but just wnated ot get this in first