Development and Testing

Development and Testing

WebSSH supports Python 3.11 and newer. The production container currently uses Python 3.14. The frontend is vanilla JavaScript and CSS with locally vendored dependencies.

Development setup

On Windows PowerShell:

python -m venv .venv
.\.venv\Scripts\python.exe -m pip install --require-hashes -r requirements-test.txt
$env:SECRET_KEY = '<strong local development secret>'
.\.venv\Scripts\python.exe start.py

On Unix-like systems, use the corresponding .venv/bin/python interpreter and environment-variable syntax.

Do not reuse production DATA_DIR, credentials, or secrets for development and tests.

Dependency management

Direct Python dependencies live in requirements.in; hash-locked runtime and test files are generated by:

.\scripts\lock_requirements.ps1

Verify locks without modifying them:

.\scripts\lock_requirements.ps1 -Check

Do not hand-edit generated lock files. A dependency change must update the input, both lock outputs as applicable, and relevant compatibility tests.

Python tests

Run the suite with:

.\.venv\Scripts\python.exe -m pytest tests --ignore=tests/integration -q -n 2 --dist=loadscope

The pytest harness uses a reduced bcrypt work factor to keep authentication-heavy tests fast. An isolated subprocess contract verifies that normal application processes retain bcrypt's production work factor. The maintained gates execute the full unit suite on Python 3.14 and a representative compatibility suite on Python 3.11. Separate jobs exercise Redis 7/8 rate limiting and disposable OpenSSH and encrypted SMB servers. Targeted tests are useful while developing, but they do not replace the relevant full gate before release.

Frontend dependencies

Pinned browser libraries are declared in package.json and copied into the repository through the vendor workflow. Current contracts include xterm.js, fit/search addons, Socket.IO client, highlight.js, and Material Icons.

npm ci
npm run vendor
npm run vendor:check

Do not add runtime CDN scripts, fonts, or styles. The application must remain offline-capable and satisfy CSP and vendor-integrity tests.

JavaScript and E2E tests

npm run lint:js
npm run test:js
npm run test:e2e

test:e2e always runs the complete suite. CI uses two isolated servers with one worker each and assigns tests by recorded duration:

npm run test:e2e:ci -- --shard=1/2
npm run test:e2e:ci -- --shard=2/2

The runner discovers the current suite before partitioning it. Every discovered test belongs to exactly one shard; new tests receive a fallback weight. Test identity is based on file and full title, so moving source lines does not change assignment. Shared account and session fixtures require one worker per server; increasing workers without isolating that state can invalidate results.

CI uploads test-results/e2e-timing-*.json from each shard, including failed runs. Download reports from a representative successful run into test-results/ and refresh the checked-in duration baseline with:

npm run test:e2e:timings:refresh
# Or pass the downloaded report paths explicitly:
npm run test:e2e:timings:refresh -- path/to/shard-1.json path/to/shard-2.json

Review the baseline diff alongside the suite change. Do not mix reports from different revisions when refreshing it. Timing weights affect scheduling only; they do not decide which tests run.

Success screenshots and animation frames are opt-in:

npm run test:captures

This sets WEBSSH_CAPTURE_ASSETS=1 for the capture suites. Their functional, accessibility and geometry assertions also run in normal CI. Failure screenshots and traces remain available without the flag. Captures used by the README or product site update their declared files below assets/; additional QA captures are written to the ignored .test-run.tmp/captures/ directory.

Playwright assumptions must be updated when a default frontend state changes. Test both the new default and the user's explicit override.

Container and runtime gates

Relevant release validation includes:

• container build and startup;
• Gunicorn gthread, exactly one worker, and thread/socket reserve;
• /health and /ready behavior;
• graceful shutdown and transfer cleanup;
• AMD64 and ARM64 images;
• SPDX SBOM and provenance;
• Trivy results for fixable High/Critical findings;
• CodeQL for Python, JavaScript/TypeScript, and Actions.

Always associate CI evidence with the exact commit under review. A green run from an older SHA is not evidence for a newer change.

Immutable image publication

Main pushes and version tags call the complete Tests workflow once, with the exact source SHA. Tests has no separate main-push trigger. PRs still test their own merge revision, and manual exact-SHA runs remain available, including the Dependabot vendor refresh workflow. Every required job must finish successfully; missing, skipped, cancelled, or failed jobs prevent publication.

At the same time, independent native AMD64 and ARM64 jobs build digest-only candidates on ubuntu-24.04 and ubuntu-24.04-arm. They retain package refreshes, SBOM and provenance, verify source and platform identity, scan the exact runtime digests for fixable High/Critical findings, and run compatible plus opt-in hardened startup, backup/restore and shutdown checks. PR image scans also use native runners. Superseded PR scans are cancelled; release runs remain separate.

Each platform uploads its candidate identity only after its scan and runtime checks pass. The handoff is bound to repository, source SHA and workflow run. Retrying failed jobs can reuse a successful platform from an earlier attempt of the same run; a successfully rebuilt platform replaces its own handoff. Different runs, revisions, future attempts and mismatched registry evidence are rejected.

The publication job waits for both platform jobs and the full test gate. It rechecks the immutable candidates and assembles their complete manifest descriptors, including attestations, under a run-specific candidate-* tag. Only the validated combined index is copied to the existing branch/version tags (main, latest, or the version and major.minor series). Every destination tag must resolve to that original index digest. Promotion never rebuilds the image.

Successful platform handoffs are retained for 14 days. Per-platform scan evidence and the successful image-release.json are retained for 90 days; the release record contains the tested revision, index digest, platform digests and verified tags. A retry after handoff expiration needs fresh platform builds.

Registry tag writes are not transactional: a registry/network failure can leave some tags updated, so a failed promotion requires checking all tags against the candidate digest before retrying. Publisher jobs are serialized, with up to 100 pending jobs queued. Queue arrival follows validation completion, so a main push also rechecks the current remote main SHA immediately before promotion. Superseded main candidates fail without updating release tags; version-tag releases remain eligible independently of the current main SHA.

PRs also exercise the promotion helper against a disposable loopback-only local registry, using tiny synthetic platform images with attestation payloads:

python scripts/check_release_promotion.py

This requires Docker and Buildx. It verifies native candidate assembly, whole-index preservation at each destination tag, and rejection of a different source SHA. It creates a uniquely named registry container and removes only that container and its anonymous volume after verifying its ownership label. This contract does not replace the real candidate scans or application runtime checks.

Test selection and required checks

The protected PR checks are all-tests and security-scan / image-security. They always report a result. The first requires the complete test job set, including SMB and the release contract; the second requires both native image scans. Missing, failed or cancelled jobs block their aggregate. Successful older runs cannot satisfy a newer revision.

Change or trigger	Tests	Container scans and publication
Code, tests, dependencies, workflow or mixed PR	Full Python 3.14 suite; Python 3.11 compatibility; Redis 7/8; OpenSSH; SMB; JavaScript, vendor and lint checks; both browser shards; runtime and release contracts	Native AMD64 and ARM64 scans; no publication
Documentation-only PR	Exact-revision classification, documentation contracts and both required aggregates	Both image scans deliberately skipped; no publication
Main push or version tag	Complete reusable Tests workflow once	Native candidates build alongside tests; publish only after all gates pass
Manual exact-SHA test run	Full tests, regardless of changed paths	No publication from the Tests workflow
Scheduled or manual Container Security run	Independent image validation	Both native architecture scans; no publication

scripts/ci_change_scope.py owns the documentation allowlist: README.md, Markdown files below docs/, and documentation media below docs/media/. Runtime assets under static/, tests, scripts, workflows, locks and unknown paths always select the full path. The classifier checks the exact checkout, compares the PR base to the tested merge revision, and includes deleted paths and both sides of renames. It reads the complete Git diff, without an API file limit. Missing history, an empty diff or uncertain Docker exclusions select full validation. Workflow summaries record the decision and its reason.

The existing documentation contracts run in the initial test job through python -m unittest discover -s tests -p test_documentation_surface.py -v. The 33 checks cover Wiki links, page coverage, README structure, documentation media, LDAP and Command Set instructions, runtime-version consistency and production Compose commands without installing application dependencies. Keep every test that reads README or allowed documentation paths in this module, including contracts that compare documentation with runtime or workflow files. Full pytest excludes that module to avoid running it twice.

The same initial job runs python scripts/check_repository_hygiene.py against the complete tracked tree. This blocks local agent instructions, internal review artifacts, transient development output, workstation-specific AI-tool paths and unreferenced files below assets/, including files force-added despite ignore rules. Run it locally before opening or updating a pull request.

Only a successful docs classification permits the named expensive jobs to be skipped. An unexpected result, a missing job or a missing scope fails the aggregate. Workflow-level path filters are intentionally avoided so required checks cannot remain pending merely because documentation changed.

Main and tag pushes always run the full release pipeline, including docs merges. Every published image therefore belongs to its fully tested source revision. This also preserves the current-main promotion guard when a documentation merge arrives during an earlier code release. Pages, Wiki publication and CodeQL retain their own triggers and do not use the PR documentation shortcut.

When adding a new test job, add it to all-tests.needs and the corresponding required set in scripts/check_release_gates.py. Keep selection conditions in sync and test both full and documentation modes. Do not solve a failing aggregate by accepting arbitrary skipped jobs or by removing a required check.

Storage and concurrency rules

When changing JSON persistence:

• hold storage_lock for the complete load-modify-save sequence;
• use atomic_write_json;
• preserve corrupt data for diagnosis rather than overwriting it;
• make schema migrations additive and backed up.

When changing SSH/SFTP or transfers:

• preserve user ownership checks;
• use existing Paramiko channel helpers;
• keep timeouts, cancellation, and quota admission;
• ensure quick connections outlive queued/active transfer references;
• test graceful shutdown and disconnect races.

Contributions

Keep changes focused and match the surrounding module's style and error contracts. Feature proposals belong in GitHub Discussions. Security vulnerabilities must follow the private process in SECURITY.md, not a public issue.

Before submitting a change, run the checks relevant to its actual blast radius and document any gate that could not be executed.