Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 30 additions & 4 deletions apps/docs/content/api/install-webgpu.mdx
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
title: installWebGPU
description: Install WebGPU flag constants on a worklet runtime.
description: Install navigator.gpu and the WebGPU flag constants on a worklet runtime.
---

On a worklet runtime, the WebGPU constants are not available by default.
Call `installWebGPU()` once at the top of a worklet to install the needed globals for that runtime.
On a worklet runtime, neither `navigator.gpu` nor the WebGPU flag constants are available by default.
Call `installWebGPU()` once at the top of a worklet to install the needed globals for that runtime: `navigator.gpu` plus `GPUBufferUsage`, `GPUTextureUsage`, `GPUShaderStage`, `GPUColorWrite`, and `GPUMapMode`.

On the main JS thread these are already installed when the native module loads; calling `installWebGPU()` there is a safe no-op.

Expand Down Expand Up @@ -72,4 +72,30 @@ const gpuWork = (device: GPUDevice) => {
};

runOnRuntime(runtime, gpuWork)(device);
```
```

### navigator.gpu on a worklet runtime

After `installWebGPU()`, a worklet can request its own adapter and device instead of receiving them from the main thread. The returned promises settle on the calling runtime.

```tsx twoslash
import { installWebGPU } from "react-native-webgpu";
import { runOnUI } from "react-native-worklets";

runOnUI(() => {
"worklet";
installWebGPU();
globalThis.navigator.gpu.requestAdapter().then((adapter) => {
// …
});
})();
```

## Known limitations

* `globalThis.navigator` instead of `navigator`: inside a worklet, read `navigator` through `globalThis.navigator`. The Worklets babel plugin does not currently treat a bare `navigator` as a known global, so without the prefix it captures the main runtime's `navigator` object by closure instead of reading the one installed on the worklet runtime. This is fixed upstream ([software-mansion/react-native-reanimated#10364](https://github.com/software-mansion/react-native-reanimated/pull/10364)); once you are on a version of `react-native-worklets` that includes it, a bare `navigator` works and the prefix is no longer needed.
* `device.lost` and `uncapturederror` are main-thread only: spontaneous device events are only delivered for devices created on the main JS runtime:
- `device.lost` read on a worklet runtime returns a promise that never settles, unless the device is already lost at that point, in which case it resolves normally.
- `uncapturederror` listeners registered on a device created on a worklet runtime never fire.

If you need to observe device loss or uncaptured errors, create the device on the main JS thread, attach the handlers there, and pass the device into the worklet.
2 changes: 1 addition & 1 deletion apps/docs/content/docs/integrations/worklets.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Follow the Worklets installation guide for the babel plugin and native setup. We

## installWebGPU

Worklet runtimes start without WebGPU flag globals (`GPUBufferUsage`, `GPUTextureUsage`, etc.). Call [`installWebGPU()`](/api/install-webgpu) once at the top of a worklet:
Worklet runtimes start without the WebGPU globals (`navigator.gpu`, `GPUBufferUsage`, `GPUTextureUsage`, etc.). Call [`installWebGPU()`](/api/install-webgpu) once at the top of a worklet to install all of them for that runtime:

```tsx twoslash
import type { RNCanvasContext, CanvasRef } from "react-native-webgpu";
Expand Down
2 changes: 1 addition & 1 deletion apps/docs/next-env.d.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
/// <reference types="next" />
/// <reference types="next/image-types/global" />
import "./.next/types/routes.d.ts";
import "./.next/dev/types/routes.d.ts";

// NOTE: This file should not be edited
// see https://nextjs.org/docs/app/api-reference/config/typescript for more information.
23 changes: 23 additions & 0 deletions apps/example/src/Diagnostics/WorkletRequestAdapter.tsx
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import React, { useState } from "react";
import { Button, StyleSheet, Text, View } from "react-native";
import { installWebGPU } from "react-native-webgpu";
import { runOnUI, scheduleOnRN } from "react-native-worklets";

// Repro for a crash when the first WebGPU call happens on a worklet runtime
Expand Down Expand Up @@ -43,13 +44,35 @@ export const WorkletRequestAdapter = () => {
})();
};

const workletNavigator = () => {
// No WebGPU object crosses explicitly here: installWebGPU() carries the
// GPU instance in its own closure and installs it as navigator.gpu on the
// calling runtime (globalThis. prefix required, see install.ts).
runOnUI(() => {
"worklet";
installWebGPU();
globalThis.navigator.gpu.requestAdapter().then((adapter) => {
scheduleOnRN(
append,
`UI (worklet) runtime: installWebGPU() + navigator.gpu -> ${
adapter ? "GPUAdapter" : "null"
}`,
);
});
})();
};

return (
<View style={styles.container}>
<Button title="requestAdapter on main JS runtime" onPress={mainRuntime} />
<Button
title="requestAdapter on UI worklet runtime"
onPress={workletRuntime}
/>
<Button
title="installWebGPU + navigator.gpu on UI runtime"
onPress={workletNavigator}
/>
{log.map((line, i) => (
<Text key={i} style={styles.log}>
{line}
Expand Down
2 changes: 1 addition & 1 deletion packages/webgpu/src/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import type {

export * from "./main";
export * from "./constants";
export * from "./install";
export { installWebGPU } from "./install";
export type {
NativeVideoFrame,
VideoPlayer,
Expand Down
72 changes: 56 additions & 16 deletions packages/webgpu/src/install.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
/// <reference types="@webgpu/types" />
import {
GPUBufferUsage,
GPUColorWrite,
Expand All @@ -18,37 +19,67 @@ const constants = {
GPUMapMode,
};

// The GPU instance destined for `navigator.gpu` on other runtimes, wrapped in
// a holder object. It cannot be read at module evaluation: this module can be
// evaluated before the native install has populated `RNWebGPU.gpu`, so
// `main/index.tsx` fills the holder right after installing. The holder itself
// is captured into the `installWebGPU` worklet closure; when a worklet is
// serialized (always after startup, so after the holder is filled), the
// Worklets custom serializer boxes the GPU object inside it, and unboxing on
// the target runtime installs the GPU prototype there.
const holder: { gpu?: GPU } = {};

/**
* @internal Called once by `main/index.tsx` after the native install, so
* `installWebGPU()` can put `navigator.gpu` on other runtimes.
*/
export const provideGPUForInstall = (gpu: GPU) => {
holder.gpu = gpu;
};

/**
* Install WebGPU on the runtime that calls it.
*
* The native module installs the WebGPU flag constants (`GPUBufferUsage`,
* `GPUTextureUsage`, `GPUShaderStage`, `GPUColorWrite`, `GPUMapMode`) as globals
* on the main JS runtime, but worklet runtimes (Reanimated UI, dedicated worklet
* runtimes, Vision Camera frame processors) start without them, so referencing
* the bare global inside a worklet yields `undefined`.
* The native module sets up WebGPU on the main JS runtime, but worklet
* runtimes (Reanimated UI, dedicated worklet runtimes, Vision Camera frame
* processors) start without it: `navigator.gpu` and the flag constants
* (`GPUBufferUsage`, `GPUTextureUsage`, `GPUShaderStage`, `GPUColorWrite`,
* `GPUMapMode`) are all `undefined` there.
*
* Call `installWebGPU()` once at the top of a worklet to make those globals
* available there, instead of importing each constant by hand:
* Call `installWebGPU()` once at the top of a worklet to make them available:
*
* ```tsx
* import { installWebGPU } from "react-native-webgpu";
*
* const work = (device: GPUDevice) => {
* const work = () => {
* "worklet";
* installWebGPU();
* device.createBuffer({
* usage: GPUBufferUsage.COPY_DST | GPUBufferUsage.MAP_READ,
* globalThis.navigator.gpu.requestAdapter().then((adapter) => {
* // ...
* });
* };
* ```
*
* The constants are captured into the worklet by closure (the same way a shader
* string is), so they work on every runtime. Calling it on a runtime that
* already has the globals (e.g. the main JS runtime) is a safe no-op.
* Known limitations:
*
* - Read `navigator` through `globalThis.navigator`: the Worklets Babel
* plugin does not treat a bare `navigator` as a known global, so it would
* capture the main runtime's `navigator` object by closure instead of
* reading the one this function installed. Fixed upstream in
* react-native-reanimated#10364; the prefix becomes unnecessary on
* react-native-worklets versions that include it.
* - Spontaneous events are main-runtime only: on a device created on a
* worklet runtime, `device.lost` never settles (unless the device is
* already lost when read) and `uncapturederror` listeners never fire.
* Observe those on a device created on the main JS thread.
*
* This is the explicit entry point for runtime setup; for now it only installs
* the flag constants, but it is the place where other per-runtime WebGPU setup
* (e.g. `navigator.gpu`) can be wired in later.
* Everything is captured into the worklet by closure: the constants like a
* shader string would be, and the GPU object through the Worklets custom
* serializer, which installs the native prototypes on the target runtime when
* it crosses. Promises returned by `requestAdapter`/`requestDevice` settle on
* the calling runtime (each runtime gets its own async pump). Calling it on a
* runtime that already has the globals (e.g. the main JS runtime) is a safe
* no-op.
*/
export const installWebGPU = () => {
"worklet";
Expand All @@ -58,4 +89,13 @@ export const installWebGPU = () => {
g[key] = value;
}
}
const { gpu } = holder;
if (gpu !== undefined) {
const nav = g.navigator as { gpu?: GPU; userAgent?: string } | undefined;
if (nav === undefined) {
g.navigator = { gpu, userAgent: "react-native" };
} else if (nav.gpu === undefined) {
nav.gpu = gpu;
}
}
};
3 changes: 3 additions & 0 deletions packages/webgpu/src/main/index.tsx
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { registerWebGPUForReanimated } from "../external";
import { provideGPUForInstall } from "../install";
import WebGPUModule from "../NativeWebGPUModule";

export * from "../Canvas";
Expand Down Expand Up @@ -36,6 +37,8 @@ if (typeof RNWebGPU !== "undefined") {
}
global.createImageBitmap =
global.createImageBitmap ?? RNWebGPU.createImageBitmap.bind(RNWebGPU);
// Let installWebGPU() put navigator.gpu on other runtimes (see install.ts).
provideGPUForInstall(RNWebGPU.gpu);
} else {
console.warn(
`[react-native-webgpu] install() returned ${_installOk} but RNWebGPU global is not available`,
Expand Down
1 change: 0 additions & 1 deletion packages/webgpu/src/mock.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ import type { NativeCanvas, RNCanvasContext } from "./Canvas";
// `./importDevice.ts`).

export * from "./constants";
export * from "./install";
export * from "./formats";

const NOT_AVAILABLE_UNDER_JEST =
Expand Down
Loading