ci(e2e): retry reachability on empty results, upload diagnostics on failure

The e2e-reachability job intermittently fails with 'no components with
alerts in .socket.facts.json': the tier-1 reachability backend can
return empty results while the CLI reports success (ENG-5093), and the
same flake has hit unrelated PRs.

- Add a retry-probe hook to the e2e matrix: entries that define it get
  up to 3 scan attempts, retrying only when the probe says the output
  looks incomplete. Persistent failures still fail via the validate
  step. Each retry emits a warning annotation and a step-summary line
  so flake frequency stays visible.
- Add tests/e2e/reach-facts-probe.sh: exits 0 when the facts file has
  alerted components, non-zero (retry) when empty or missing.
- Upload /tmp/e2e-output.log, SARIF/GitLab outputs, and facts files as
  artifacts when any e2e job fails, so flakes are diagnosable without
  a re-run.

Also bump version to 2.6.2 (2.6.0 and 2.6.1 are being released ahead
of this PR).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Author: lelia

.github/workflows/e2e-test.yml

@@ -46,6 +46,11 @@ jobs:
               --enable-debug
             validate: tests/e2e/validate-reachability.sh
             setup-node: "true"
+            # The tier-1 reachability backend intermittently returns empty
+            # results while the CLI reports success (ENG-5093). The probe
+            # exits 0 when the facts file has alerted components; anything
+            # else is retried before validation fails the job.
+            retry-probe: bash tests/e2e/reach-facts-probe.sh tests/e2e/fixtures/simple-npm
 
           - name: gitlab
             args: >-
@@ -96,15 +101,54 @@ jobs:
       - name: Run Socket CLI
         env:
           SOCKET_SECURITY_API_KEY: ${{ secrets.SOCKET_CLI_API_TOKEN }}
+          RETRY_PROBE: ${{ matrix.retry-probe }}
         run: |
           set -o pipefail
-          socketcli ${{ matrix.args }} 2>&1 | tee /tmp/e2e-output.log
+          # Entries with retry-probe get up to 3 attempts: the probe exits 0
+          # when the scan output looks complete, and a run that fails it is
+          # re-run on the assumption of a transient backend failure. A
+          # persistent failure still reaches the validate step, which fails
+          # the job with full context. Retries are surfaced as warning
+          # annotations so flake frequency stays visible instead of being
+          # silently absorbed.
+          max_attempts=3
+          attempt=1
+          while :; do
+            socketcli ${{ matrix.args }} 2>&1 | tee /tmp/e2e-output.log
+            [ -z "$RETRY_PROBE" ] && break
+            if bash -c "$RETRY_PROBE"; then
+              break
+            fi
+            if [ "$attempt" -ge "$max_attempts" ]; then
+              echo "::warning title=e2e-${{ matrix.name }} incomplete results::output still fails the completeness probe after ${max_attempts} attempts; letting validation fail the job"
+              break
+            fi
+            echo "::warning title=e2e-${{ matrix.name }} transient retry::attempt ${attempt} failed the completeness probe (suspected backend transient, see ENG-5093); retrying"
+            echo "e2e-${{ matrix.name }}: retry after attempt ${attempt} — completeness probe failed (suspected transient)" >> "$GITHUB_STEP_SUMMARY"
+            attempt=$((attempt+1))
+            sleep 30
+          done
 
       - name: Validate results
         env:
           SOCKET_SECURITY_API_KEY: ${{ secrets.SOCKET_CLI_API_TOKEN }}
         run: bash ${{ matrix.validate }}
 
+      - name: Upload diagnostics on failure
+        if: failure()
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
+        with:
+          name: e2e-${{ matrix.name }}-diagnostics-attempt${{ github.run_attempt }}
+          path: |
+            /tmp/e2e-output.log
+            /tmp/*.sarif
+            tests/e2e/fixtures/simple-npm/.socket.facts.json
+            tests/e2e/fixtures/simple-pypi/.socket.facts.json
+            gl-*.json
+            license_output.json
+          if-no-files-found: ignore
+          retention-days: 14
+
   # Branch protection requires the e2e-* checks, but the `e2e` job above is
   # skipped on PRs that can't access repository secrets -- fork PRs and
   # Dependabot PRs. A job skipped via a job-level `if` never expands its

CHANGELOG.md

@@ -1,6 +1,6 @@
 # Changelog
 
-## 2.5.11
+## 2.6.2
 
 ### Fixed: SBOM fetch failures no longer produce empty reports
 

pyproject.toml

@@ -6,7 +6,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "socketsecurity"
-version = "2.5.11"
+version = "2.6.2"
 requires-python = ">= 3.11"
 license = {"file" = "LICENSE"}
 dependencies = [

socketsecurity/__init__.py

@@ -1,3 +1,3 @@
 __author__ = 'socket.dev'
-__version__ = '2.5.11'
+__version__ = '2.6.2'
 USER_AGENT = f'SocketPythonCLI/{__version__}'

tests/e2e/reach-facts-probe.sh

@@ -0,0 +1,19 @@
+#!/usr/bin/env bash
+# Exits 0 when the reachability facts file contains components with alerts.
+#
+# Used by the e2e workflow's retry-probe hook: a --reach run against the
+# known-vulnerable fixture that reports success but yields no alerted
+# components is the signature of a transient tier-1 backend failure
+# (ENG-5093), so the run is worth repeating before validation fails the job.
+set -euo pipefail
+
+TARGET="${1:?usage: reach-facts-probe.sh <target-path>}"
+
+uv run python - "$TARGET" <<'PY'
+import sys
+
+from socketsecurity.core.alert_selection import load_components_with_alerts
+
+components = load_components_with_alerts(sys.argv[1], ".socket.facts.json")
+sys.exit(0 if components else 1)
+PY