Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,8 +196,8 @@ alone.
**Presentation options**:
The policy input controlling how one snapshot acquisition becomes a public projection.

**Snapshot presentation facet**:
The host-side owner of the neutral acquisition-to-presentation contract and TypeScript policies.
**Snapshot policy facet**:
The host-side owner of neutral snapshot policy: presentation, freshness, timeout and overlay.
Platform acquisition supplies raw facts and a fold policy; runner-side Swift presentation remains
separate across the process boundary.

Expand Down
59 changes: 55 additions & 4 deletions docs/adr/0004-ios-snapshot-backend-strategy.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,10 +78,61 @@ capture-plan tier before the process boundary. The contract fixture under
`contracts/fixtures/snapshot-presentation-conformance.json` is the shared proof between those
runtimes; it does not imply that Swift and TypeScript share an implementation.

This is the first ownership slice of the Wave 4 debt tracked by #1983. Freshness recovery,
timeout evidence, and screenshot-overlay policy retain their existing daemon adapters until their
neutral host seams are extracted; new consumers must use the facet rather than add another daemon
presentation path.
The same split now holds for the three remaining Wave 4 policies tracked by #1983, so
`src/snapshot/` is the host-side owner of snapshot policy generally rather than of presentation
alone:

- **Freshness recovery.** The freshness window, the Android staleness classification and its
thresholds, and the retry loop live in `src/snapshot/snapshot-freshness/`. The loop is
parameterized by a classifier and a retry schedule, so "how long may a backend lag behind a real
transition" is a policy input rather than a constant the loop owns. The schedule is stated as a
duration budget; the loop derives the deadline from the window's `markedAt` itself, so the
budget is always spent from the action and a caller has no absolute instant it could get wrong.
`src/daemon/session-snapshot-freshness.ts` keeps only what needs a session: reading and retiring
the window on store-owned `SessionState`, and choosing the comparison baseline from snapshot
lineage. It remains the declared R7 owner of `androidSnapshotFreshness`.
- **Timeout evidence.** Whether a capture failed because the hierarchy never arrived is decided
once, at the deepest boundary that has the evidence, from machine-defined values only:
`snapshot-capture-failure-reason.ts` maps the helper's structured `errorType` field
(`java.util.concurrent.TimeoutException`, by exact equality) and the SIGKILL exit code 137 to
the typed reason `accessibility-timeout` (`ANDROID_CAPTURE_FAILURE_REASONS` in
`@agent-device/contracts/android-snapshot-quality`). The helper-result, session-protocol, and
killed-instrumentation error constructors attach it; every layer above rewraps it rather than
reclassifying. No message shape is consulted anywhere on that path, so rewording helper or
wrapper prose cannot move the reason, and prose that merely reads like a timeout does not become
one — both directions are asserted end to end against the real producer.
`src/snapshot/snapshot-timeout-policy.ts` reads the reason; the human-facing hint is derived
from it rather than decided alongside it.

The published `details.androidSnapshotTimeoutScreenshot` payload is vocabulary in
`@agent-device/contracts/snapshot-timeout-evidence`, a union whose arms encode which claims can
coexist. The annotated arm carries a non-empty ref tuple, so "annotated with zero refs" is not a
state a caller can build, and no arm stores a ref count: a count beside the refs is a second
source of truth the type system cannot hold in step, so it is derived from the refs instead.
The daemon keeps the ordering that genuinely needs it: resolving a bound screenshot runtime,
writing the artifact, annotating it from the stored observation, and emitting the diagnostics.
- **Screenshot-overlay policy.** Which Android nodes earn an overlay ref, and what rectangle an
overlay for one of them covers, live in `src/snapshot/screenshot-overlay/`. The daemon keeps
approved artifact and ref assembly only: ranking, projection to screenshot pixels, drawing, and
PNG IO.

`scripts/layering/snapshot-presentation-boundary.test.ts` enforces the direction for the whole
facet: nothing under `src/snapshot/` may import `src/daemon/`. It carries a positive control,
because a filter that stopped matching would look identical to a boundary being obeyed.

The residual call sites #1983 also named are audited and deliberately left in place.
`src/daemon/direct-ios-selector.ts` carries no presentation policy: `isLocalIosRunnerSession` and
`readSimpleIosSelectorTarget` are session routing (device family, provider ownership, the
stabilization window), while `deriveDirectIosNodeSelector` and `isDirectIosSelectorFallbackError`
are selector derivation and ADR 0011 delegation-on-error. The latter two are pure and
daemon-independent, but their owner would be the selector pipeline governed by R19, not this
facet; moving them under ADR 0004 would widen it to a boundary it does not decide. The
observation and interaction consumers — `selector-capture-runtime.ts`,
`deferred-interaction-outcome.ts`, `snapshot-capture.ts` and
`interaction-touch-android-freshness.ts` — now reach freshness only through the facet or its
session binding.

New consumers must use the facet rather than add another daemon presentation path.

## Regression Notes

Expand Down
6 changes: 0 additions & 6 deletions fallow-baselines/health.json
Original file line number Diff line number Diff line change
Expand Up @@ -112,11 +112,6 @@
"count": 1
}
},
"src/daemon/android-snapshot-freshness.ts": {
"crap_moderate": {
"count": 1
}
},
"src/daemon/client/daemon-client-lifecycle.ts": {
"complexity_high": {
"count": 1
Expand Down Expand Up @@ -575,7 +570,6 @@
"src/daemon/handlers/session.ts:complexity",
"src/replay/script-utils.ts:high impact",
"packages/selectors/src/internal/predicates.ts:high impact",
"src/daemon/android-snapshot-freshness.ts:high impact",
"src/platforms/boot-diagnostics.ts:complexity",
"src/compat/maestro/support.ts:high impact",
"src/daemon/session-routing.ts:high impact",
Expand Down
4 changes: 4 additions & 0 deletions packages/contracts/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -307,6 +307,10 @@
"types": "./src/snapshot-runtime.ts",
"default": "./src/snapshot-runtime.ts"
},
"./snapshot-timeout-evidence": {
"types": "./src/snapshot-timeout-evidence.ts",
"default": "./src/snapshot-timeout-evidence.ts"
},
"./startup-recovery-fence": {
"types": "./src/startup-recovery-fence.ts",
"default": "./src/startup-recovery-fence.ts"
Expand Down
33 changes: 33 additions & 0 deletions packages/contracts/src/android-snapshot-quality.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,3 +33,36 @@ export function isUnreadableCaptureContentError(error: unknown): boolean {
const reason = details?.androidSnapshotHelperFailureReason;
return isAndroidContentRecoveryReason(reason);
}

/**
* Why an Android capture failed at the mechanism level, as a typed reason rather than a message
* shape. `accessibility-timeout` means the hierarchy never arrived — the helper reported a
* structured timeout, or its instrumentation was killed before it could answer.
*
* The producer decides this once, at the platform boundary, and publishes the decision. Readers
* consume the reason instead of re-deriving it from hint text: the hint is human-facing prose
* that may be reworded, and two readers sniffing it will drift apart (#1983).
*/
export const ANDROID_CAPTURE_FAILURE_REASONS = ['accessibility-timeout'] as const;

export type AndroidCaptureFailureReason = (typeof ANDROID_CAPTURE_FAILURE_REASONS)[number];

const ANDROID_CAPTURE_FAILURE_REASON_SET: ReadonlySet<string> = new Set(
ANDROID_CAPTURE_FAILURE_REASONS,
);

export function isAndroidCaptureFailureReason(
value: unknown,
): value is AndroidCaptureFailureReason {
return typeof value === 'string' && ANDROID_CAPTURE_FAILURE_REASON_SET.has(value);
}

/** The typed reason a thrown Android capture failure carries, when its producer named one. */
export function readAndroidCaptureFailureReason(
error: unknown,
): AndroidCaptureFailureReason | undefined {
if (!error || typeof error !== 'object') return undefined;
const details = (error as { details?: Record<string, unknown> }).details;
const reason = details?.androidCaptureFailureReason;
return isAndroidCaptureFailureReason(reason) ? reason : undefined;
}
8 changes: 7 additions & 1 deletion packages/contracts/src/facades/platform.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,17 @@ export type {
AndroidInputOwnershipSource,
} from '../android-input-ownership.ts';
export {
ANDROID_CAPTURE_FAILURE_REASONS,
ANDROID_CONTENT_RECOVERY_REASONS,
isAndroidContentRecoveryReason,
isUnreadableCaptureContentError,
isAndroidCaptureFailureReason,
readAndroidCaptureFailureReason,
} from '../android-snapshot-quality.ts';
export type {
AndroidCaptureFailureReason,
AndroidContentRecoveryReason,
} from '../android-snapshot-quality.ts';
export type { AndroidContentRecoveryReason } from '../android-snapshot-quality.ts';
export {
ANDROID_SYSTEM_CHROME_PACKAGE,
hasAndroidSystemChromeProvenance,
Expand Down
93 changes: 93 additions & 0 deletions packages/contracts/src/snapshot-timeout-evidence.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
import assert from 'node:assert/strict';
import { test } from 'vitest';
import type { ScreenshotOverlayRef } from '@agent-device/kernel/snapshot';
import {
snapshotTimeoutCaptureFailed,
snapshotTimeoutEvidenceOverlayCounts,
snapshotTimeoutEvidenceOverlayFailed,
snapshotTimeoutEvidenceWithOverlayRefs,
snapshotTimeoutEvidenceWithoutOverlaySource,
type SnapshotTimeoutEvidence,
} from './snapshot-timeout-evidence.ts';

function overlayRef(ref: string): ScreenshotOverlayRef {
return {
ref,
label: ref,
rect: { x: 0, y: 0, width: 10, height: 10 },
overlayRect: { x: 0, y: 0, width: 10, height: 10 },
center: { x: 5, y: 5 },
};
}

test('a failed capture publishes no path and no overlay claim', () => {
const evidence = snapshotTimeoutCaptureFailed('adb screencap exited 1');
assert.deepEqual(evidence, { captureFailed: true, error: 'adb screencap exited 1' });
assert.deepEqual(snapshotTimeoutEvidenceOverlayCounts(evidence), {
overlayRefCount: undefined,
overlayRefsAnnotated: undefined,
});
});

test('a capture with no stored observation still discloses that refs were requested', () => {
const evidence = snapshotTimeoutEvidenceWithoutOverlaySource('/tmp/shot.png');
assert.deepEqual(evidence, {
path: '/tmp/shot.png',
overlayRefsRequested: true,
overlayRefsAnnotated: false,
overlayRefSource: 'unavailable',
});
});

test('the union cannot express an annotated capture with no refs', () => {
// The annotated arm carries a non-empty tuple, so this literal is not assignable.
// `@ts-expect-error` fails the build if it ever becomes valid again.
// @ts-expect-error annotated:true requires at least one overlay ref
const contradiction: SnapshotTimeoutEvidence = {
path: '/tmp/shot.png',
overlayRefsRequested: true,
overlayRefSource: 'session-snapshot',
overlayRefsAnnotated: true,
overlayRefs: [],
};
assert.ok(contradiction);
});

test('the union has no stored count that could disagree with the refs', () => {
// The count is derived, so there is no field to set — a mismatched one is not merely wrong,
// it is unwritable. This guards against reintroducing the second source of truth.
const withStoredCount: SnapshotTimeoutEvidence = {
path: '/tmp/shot.png',
overlayRefsRequested: true,
overlayRefSource: 'session-snapshot',
overlayRefsAnnotated: true,
overlayRefs: [overlayRef('e1')],
// @ts-expect-error overlayRefCount is derived, not stored
overlayRefCount: 0,
};
assert.ok(withStoredCount);
});

test('an empty ref list is a capture that was not annotated, not an annotated one', () => {
const none = snapshotTimeoutEvidenceWithOverlayRefs('/tmp/shot.png', []);
const some = snapshotTimeoutEvidenceWithOverlayRefs('/tmp/shot.png', [overlayRef('e1')]);
assert.deepEqual(snapshotTimeoutEvidenceOverlayCounts(none), {
overlayRefCount: 0,
overlayRefsAnnotated: false,
});
assert.deepEqual(snapshotTimeoutEvidenceOverlayCounts(some), {
overlayRefCount: 1,
overlayRefsAnnotated: true,
});
});

test('a failed annotation keeps the screenshot and names why the refs are missing', () => {
const evidence = snapshotTimeoutEvidenceOverlayFailed('/tmp/shot.png', 'png decode failed');
assert.deepEqual(evidence, {
path: '/tmp/shot.png',
overlayRefsRequested: true,
overlayRefsAnnotated: false,
overlayRefSource: 'session-snapshot',
overlayAnnotationError: 'png decode failed',
});
});
121 changes: 121 additions & 0 deletions packages/contracts/src/snapshot-timeout-evidence.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
import type { ScreenshotOverlayRef } from '@agent-device/kernel/snapshot';

/**
* The typed `details.androidSnapshotTimeoutScreenshot` payload published when an accessibility
* snapshot times out and the daemon falls back to a screenshot (#1983).
*
* The shape is a discriminated union rather than a bag of optional fields because each arm is a
* different claim about what evidence exists: no capture at all, a capture with no snapshot to
* annotate from, a capture annotated with overlay refs, or a capture whose annotation failed.
* Callers construct it through the builders below so an assembly site cannot publish a fifth,
* undeclared arm.
*/
type CapturedSnapshotTimeoutEvidenceBase = {
path: string;
overlayRefsRequested: true;
};

/** At least one ref. Annotation is a claim that something was drawn, so it cannot be empty. */
export type NonEmptyScreenshotOverlayRefs = readonly [
ScreenshotOverlayRef,
...ScreenshotOverlayRef[],
];

/**
* No arm stores a ref count. A count beside the refs is a second source of truth that the type
* system cannot keep in step — `{annotated: true, count: 0, refs: [ref]}` would stay assignable —
* so the count is derived from `overlayRefs` by `snapshotTimeoutEvidenceOverlayCounts` instead.
* The arms that carry no refs have nothing to count, which `overlayRefsAnnotated: false` already
* states.
*/
export type SnapshotTimeoutEvidence =
| {
captureFailed: true;
error: string;
}
| (CapturedSnapshotTimeoutEvidenceBase & {
overlayRefSource: 'unavailable';
overlayRefsAnnotated: false;
})
| (CapturedSnapshotTimeoutEvidenceBase & {
overlayRefSource: 'session-snapshot';
overlayRefsAnnotated: true;
overlayRefs: NonEmptyScreenshotOverlayRefs;
})
| (CapturedSnapshotTimeoutEvidenceBase & {
overlayRefSource: 'session-snapshot';
overlayRefsAnnotated: false;
overlayRefs: readonly [];
})
| (CapturedSnapshotTimeoutEvidenceBase & {
overlayRefSource: 'session-snapshot';
overlayRefsAnnotated: false;
overlayAnnotationError: string;
});

/** No screenshot was taken: the evidence path itself failed. */
export function snapshotTimeoutCaptureFailed(error: string): SnapshotTimeoutEvidence {
return { captureFailed: true, error };
}

/** A screenshot exists, but no stored observation was available to derive overlay refs from. */
export function snapshotTimeoutEvidenceWithoutOverlaySource(path: string): SnapshotTimeoutEvidence {
return {
path,
overlayRefsRequested: true,
overlayRefsAnnotated: false,
overlayRefSource: 'unavailable',
};
}

/**
* A screenshot annotated from the stored observation. An empty ref list is not an annotation, and
* the union says so: the annotated arm carries a non-empty tuple, so `annotated: true` with zero
* refs is not a state a caller can build or a reader has to defend against.
*/
export function snapshotTimeoutEvidenceWithOverlayRefs(
path: string,
overlayRefs: readonly ScreenshotOverlayRef[],
): SnapshotTimeoutEvidence {
const base = { path, overlayRefsRequested: true, overlayRefSource: 'session-snapshot' } as const;
return isNonEmptyOverlayRefs(overlayRefs)
? { ...base, overlayRefsAnnotated: true, overlayRefs }
: { ...base, overlayRefsAnnotated: false, overlayRefs: [] };
}

function isNonEmptyOverlayRefs(
refs: readonly ScreenshotOverlayRef[],
): refs is NonEmptyScreenshotOverlayRefs {
return refs.length > 0;
}

/** A screenshot exists and a stored observation existed, but annotating it threw. */
export function snapshotTimeoutEvidenceOverlayFailed(
path: string,
overlayAnnotationError: string,
): SnapshotTimeoutEvidence {
return {
path,
overlayRefsRequested: true,
overlayRefsAnnotated: false,
overlayRefSource: 'session-snapshot',
overlayAnnotationError,
};
}

/**
* The overlay counts a diagnostic may report, derived from the refs the evidence actually holds.
* This is the only place a count exists, so it cannot disagree with the arm it came from.
*/
export function snapshotTimeoutEvidenceOverlayCounts(evidence: SnapshotTimeoutEvidence): {
overlayRefCount: number | undefined;
overlayRefsAnnotated: boolean | undefined;
} {
if ('captureFailed' in evidence) {
return { overlayRefCount: undefined, overlayRefsAnnotated: undefined };
}
return {
overlayRefCount: 'overlayRefs' in evidence ? evidence.overlayRefs.length : 0,
overlayRefsAnnotated: evidence.overlayRefsAnnotated,
};
}
2 changes: 1 addition & 1 deletion scripts/__tests__/test-file-size-ratchet.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ const PINNED_TEST_FILE_LINES: Readonly<Record<string, number>> = Object.freeze({
'src/platforms/apple/core/__tests__/runner-session.test.ts': 2001,
'src/utils/__tests__/daemon-client.test.ts': 1910,
'src/utils/__tests__/output.test.ts': 1861,
'src/platforms/android/__tests__/snapshot.test.ts': 1495,
'src/platforms/android/__tests__/snapshot.test.ts': 1445,
'src/platforms/apple/core/__tests__/runner-client.test.ts': 1615,
'src/__tests__/client.test.ts': 1598,
'test/integration/provider-scenarios/android-lifecycle.test.ts': 1559,
Expand Down
Loading
Loading