Skip to content
AlexB8675Public

About

No description, website, or topics provided.

Resources

Stars

4 stars

Watchers

0 watching

Forks

Repository files navigation

DFS — Discord File System

A private cloud drive that stores encrypted file chunks as Discord attachments and keeps the real file system in PostgreSQL. See docs/DESIGN.md for the full design.

Status: DFS runs at https://dfs.xlestudio.it. Done: the web UI, the API (M2), Discord storage and the bot (M1), the admin console, the first deployment, durability (M4: the journal on Discord, dfs recover and a recovery drill that passed against production), previews of images, PDFs, text and code, and audio and video players, with other players reached through stream links. Left for later: nightly database snapshots and the scrubber. docs/DEPLOY.md is the runbook for the server.

Requirements

  • Node.js 24

  • Docker Desktop, for PostgreSQL and the integration tests

  • pnpm 10, via Corepack. Turborepo runs each package's scripts with pnpm, so pnpm itself must be on your PATH; corepack pnpm … alone is not enough. Set it up once:

    corepack enable pnpm

    If that fails because Node is installed in a protected folder, put the shims in a folder that is on your PATH instead, e.g. npm's global folder on Windows: corepack enable pnpm --install-directory "$env:APPDATA\npm" (PowerShell).

Getting started

pnpm install
pnpm dev

Open http://localhost:5173 and sign in with one of the demo accounts listed under the form; clicking one fills it in. demo is the owner and an admin, and taylor shows a first sign-in, where a new user chooses their own password. In development all data lives in your browser (reset it from the account menu).

The demo data includes share links to try, signed in or not:

Link What it shows
/s/demo-documents A folder to browse and download
/s/demo-lisbon A password-protected folder (lisbon)
/s/demo-resume A single file with a download limit
/s/demo-expired An expired link
/s/demo-revoked A link the owner turned off

Running the backend

pnpm dev also starts the API (http://localhost:3000) and the bot (port 3001). They need PostgreSQL, which runs in Docker:

pnpm db:up
pnpm db:migrate
pnpm dev

Press Ctrl+C once to stop all three services. The dev scripts run in parallel through pnpm, which avoids Turborepo waiting on Windows batch shells after the servers have stopped.

Without the database, both still start: /api/health answers 503 and the bot retries until Postgres is up. If the database restarts while the bot leads, the bot stops on purpose (another instance could take over); restart pnpm dev to bring it back. The database itself comes back on its own when Docker Desktop restarts.

pnpm db:up also starts the media service, which examines audio and video. Set VITE_API_MOCKS=off in apps/web/.env.local to use the real API, and on a fresh database make the first account with pnpm dfs owner --username <name>, which prints a temporary password. See docs/TESTING.md for setup, browser checks, and the live end-to-end tests.

Scripts

Run from the repository root. pnpm starts the dev services in parallel; Turborepo runs the build and check tasks across the workspace.

Command What it does
pnpm dev Starts the web app (with its mock API), the API and the bot
pnpm check Formatting, typecheck, lint and tests: run it before every push
pnpm build Production build of the web app
pnpm typecheck TypeScript checks
pnpm lint ESLint, including the React Compiler rules
pnpm test Unit and integration tests (Vitest); the latter need Docker
pnpm format Formats with Prettier
pnpm db:up Starts PostgreSQL 18 and the media service in Docker, on localhost only
pnpm db:down Stops them; the data stays in a Docker volume
pnpm db:migrate Applies database migrations
pnpm dfs Admin commands: owner, setup, master-key, recover, drill, reset-storage
docker/deploy.sh Sends the committed code to the VPS and restarts DFS there

Integration tests start their own PostgreSQL in Docker. Without Docker, point TEST_DATABASE_URL at a server where the user may create databases.

Layout

apps/web          React SPA (React 19 + React Compiler, Vite, Tailwind, shadcn/ui)
  src/app         router, query client, providers
  src/features    one folder per feature: auth, drive, drag, tree, uploads, live-events, admin, …
  src/layout      app shell: header, sidebar
  src/components  shared components; ui/ is generated by shadcn
  src/mocks       mock API (MSW) and demo data, dev only
apps/api          Fastify HTTP API, run by Node straight from its TypeScript sources
apps/bot          the Discord bot: storing, packing and deleting blobs, leader election
apps/media        the media service: ffprobe and ffmpeg for media info, subtitles and covers
packages/shared   API schemas (Zod), metrics and file-name rules, shared by the web app and the API
packages/config   settings from the environment (or secret files), with development defaults
packages/db       Drizzle schema, SQL migrations, metrics, and the integration test harness
packages/crypto   envelope encryption and the frame format
packages/storage  the blob stores: Discord, local and chaos
packages/contract the API's contract suite, run against the mock and the real API
docker/           production images, Compose, Caddyfile and deploy script; PostgreSQL for development
tools/            scripts for the user's own machine: dfs-reencode, for videos that seek fast
docs/DESIGN.md    design document
docs/TESTING.md   testing locally
docs/DEPLOY.md    the server: setup, deploying, looking after it

About

No description, website, or topics provided.

Resources

Stars

4 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages