Android Runtime written directly against the engine layer - #69
Draft
ammarahm-ed wants to merge 45 commits into
Draft
Android Runtime written directly against the engine layer#69ammarahm-ed wants to merge 45 commits into
ammarahm-ed wants to merge 45 commits into
Conversation
… native branch
Ports NativeScript/jsi/ wholesale from android/jsi-runtime (22967c1f). That
branch's Node-API shim is deliberately NOT carried over -- this branch exists to
build the runtime directly against nativescript::engine -- but every engine-layer
change it produced is, because those are properties of the engine backends and
are independent of what sits on top:
f75697ae intern intercepted property names; stop promoting the receiver
6bb793dc PropNameID carries the v8 name handle instead of an eager string
24f56367 stop allocating per value on the read and predicate paths
3a39b374 stop flattening symbol keys to "" in the host-object setter
cea83323 non-masking host-object template on V8 (field reads 8.9x -> 1.87x)
0e620a72 borrow handles instead of globalizing them (asObjectBorrowed etc.)
2df9c9ff define asObjectBorrowed where Object is complete (JSC/QuickJS/Hermes)
4963627f native-state slot per engine (Hermes NativeState, V8 private symbol,
QuickJS class-backed opaque slot, JSC cached non-enumerable property)
22967c1f memoise the borrowed-to-owned Value promotion on Hermes
It also carries the JSC, QuickJS and Hermes engine:: backends themselves, which
were written and brought to suite parity on that branch and did not exist here.
3313 insertions across 13 files. Not verified on this branch: nothing here builds
yet, because the runtime that consumes it has not been written. The engine layer
was verified on android/jsi-runtime against all five engines and on iOS (713/0).
(cherry picked from commit fae037b73a3ca6750f93356928512006b53ba363)
Makes room for a second, complete runtime written directly against
nativescript::engine, with no Node-API anywhere. The two are siblings, not a
runtime plus a shim, and are never compiled together:
runtime/android/napi + ffi/jni/napi the reference implementation
runtime/android/jsi + ffi/jni/jsi the engine:: implementation (empty here)
Both consume NativeScript/jsi/ (the engine:: backends), which is already shared
with Apple.
runtime/android/{Runtime.cpp,Runtime.h,com_tns_Runtime.cpp,assetextractor,
inspector,instrumentation,messageloop,modules,profiler,sighandler,util,version,
workers} move under napi/ with git mv, so history follows them. 37 files.
Selection is one variable end to end: -PbindingLayer=napi|jsi becomes
-DNS_BINDING, which picks both source roots. CMake globs each tree recursively,
so this had to happen before jsi/ gains any file -- otherwise both runtimes
would compile into one library. The include_directories list is parameterised
the same way, which fixes the mirror's directory names: the jsi tree must use
the same subdirectory layout as napi/ or the include paths silently miss.
Also carried from android/jsi-runtime, since this branch predates them:
-PonlyArm64 (skips three ABIs no test device uses) and the -Poptimized note for
Hermes. Version.h is now per-binding; the gradle reference follows NS_BINDING.
Not verified: the napi runtime is not rebuilt here. The move is mechanical and
the source references are path-relative through CMake variables, but the first
napi build on this branch should be treated as the check that this commit is
correct.
(cherry picked from commit 1da91eb37d240aa59e48e144ac77a53ed836055e)
First files of the second Android JNI interop tree, the one written directly
against nativescript::engine with no Node-API. The directory layout mirrors
ffi/jni/napi exactly because CMakeLists enumerates include paths as
${NS_JNI_NAPI_DIR}/<subdir> with the root parameterised by NS_BINDING.
Engine.{h,cpp} is this tree's counterpart to js_native_api.h +
native_api_util.h. It selects the engine backend from TARGET_ENGINE_* and
supplies the JS operations engine:: deliberately omits -- defineProperty,
instanceof, prototype get/set, Reflect.deleteProperty, strict equality,
Number.isInteger, ArrayBuffer.isView. Those are language operations reachable
from the global object rather than engine primitives, so they are resolved once
per runtime and cached in Builtins instead of being pushed down into five
backends. Builtins holds owned engine handles, so it exposes an explicit
dispose(Runtime&) for the runtime teardown path; QuickJS asserts an empty gc
object list in JS_FreeRuntime and aborts if anything outlives it.
The remaining files are copied unchanged from ffi/jni/napi: jni/ (JEnv,
JniLocalRef, JType, JniSignatureParser, LRUCache, DirectBuffer, File, Logger,
DesugaredInterfaceCompanionClassNameResolver), constants/, NativeScriptAssert.h
and the metadata readers (MetadataReader, MetadataEntry, MetadataTreeNode,
MetadataMethodInfo, MethodCache.cpp). None of them reference Node-API -- they
are pure JNI/metadata-format code -- so they are the same program in both trees.
Verified: Engine.cpp passes clang++ -fsyntax-only -std=c++20 for the NDK
aarch64 target against all four engine backends (v8, jsc, quickjs, hermes).
Nothing is linked or run yet; the tree cannot build until the runtime half
exists. Hermes's Function::call has no single-Value variadic overload, so every
call site here passes an explicit argument array.
(cherry picked from commit f16b3e046b34593057b358973288a4091bd232a0)
…onversion
The lifetime substrate of the engine::-native JNI bridge, plus the two files
everything else in the tree needs to compile.
objectmanager/
The JS Proxy path is gone entirely. Under USE_HOST_OBJECT the napi tree
already builds a host object rather than a JS Proxy, and everything that only
existed to construct or finalise a real Proxy (m_jsObjectProxyCreator, the
__createNativeProxy global, JSObjectProxyFinalizerCallback) lived in the #else
branch. Here host objects are the only path, so there is no conditional at
all. HostObjectProxy derives straight from engine::HostObject with no
intermediate base, and because the engine owns the shared_ptr its destructor
is the finalizer.
Two ownership changes, both simplifications:
- JSInstanceInfo had two owners in the napi tree: a napi_external carrying the
finalizer, and an ownership-free napi_wrap used only for fast access. An
engine native-state slot is already a non-property lookup, so one shared_ptr
in one slot serves both roles. That removes the class of bug the finalizer
catalogue is full of (same pointer registered with two mechanisms, freed
twice, corruption surfacing in an unrelated spec).
- The proxy's owned handle to its target cannot be released from the
destructor, which runs inside the GC sweep. It is held in a unique_ptr and
released into the runtime's post-GC drain, which is what the napi tree did
for the equivalent napi_delete_reference.
m_idToObject/m_idToProxy become an owned Value and an engine::WeakObject,
which is what a strong and a weak napi_ref already were.
engine::HostObject has get/set/getPropertyNames and no has/delete/ownKeys or
indexed traps, so array index access is recognised from the property name on
every engine. The napi tree needed both a string and a number form here
because V8 routed indices through a separate interceptor.
finalizer/
FinalizerQueue with napi_finalize replaced by a plain function pointer taking
a Runtime&. Drain no longer opens a per-callback handle scope: an
engine::Value owns its handle, so a value a callback materialises is rooted by
the Value rather than by an enclosing scope.
exceptions/
ReThrowToNapi becomes [[noreturn]] ReThrowToJs, because engine:: signals JS
errors by throwing and the engine's host-function wrapper converts a JSError
back into a JS throw carrying the original value. The javascript exception is
a shared_ptr<Value> rather than a napi_ref; the copy handed to Java through
jsValueAddress is its own owned handle, released when it comes back, so
ownership across the JNI boundary is unambiguous.
conversion/ArgConverter
engine::String is UTF-8 only, so jchar/jstring payloads are transcoded from
UTF-16 here (unpaired surrogates become U+FFFD) instead of being handed to a
napi_create_string_utf16 equivalent. The long-number constructor cannot read
new.target, so it captures its own prototype instead; for `new Ctor()` the two
are the same object.
Engine.{h,cpp} gains is_error, coerce_to_string and create_error.
Verified: Engine.cpp, ObjectManager.h, FinalizerQueue.h and
NativeScriptException.h all pass clang++ -fsyntax-only -std=c++20 for the NDK
aarch64 target against v8, jsc, quickjs and hermes. The .cpp files of
ObjectManager, NativeScriptException, ArgConverter and FinalizerQueue are NOT
yet compiled: they include MetadataNode.h, CallbackHandlers.h and Runtime.h,
none of which exist yet (Runtime.h is the other half of the runtime, being
written separately). Nothing here has been linked or run.
(cherry picked from commit dc1f9a8b1ffc96c387698069cda83c3883014bca)
Same shape as the napi version, including the fact that it holds a *strong* handle -- napi_create_reference with an initial refcount of 1 -- so deref never observes collection. Init only installs it when the engine has no WeakRef of its own, which no engine the runtime targets is currently missing, so this is a fallback rather than a live path. Kept as-is: the napi tree is the behavioural spec, and swapping in engine::WeakObject here would be a behaviour change smuggled into a port. Two divergences. The state lives in the object's native-state slot rather than napi_wrap, and WeakRef itself is the engine::HostObject that slot stores. And the "called without new" check reads the receiver instead of new.target, which engine:: does not expose; a host constructor invoked as a plain call is handed an undefined receiver. Verified: WeakRef.cpp passes clang++ -fsyntax-only -std=c++20 for the NDK aarch64 target against v8, jsc, quickjs and hermes -- it depends only on Engine.h, so unlike the rest of the tree it compiles today. Not linked or run. (cherry picked from commit 9304b98157871691023b9c01a5c41b9d53b91c4d)
NativeScript/jsi/ was on this branch but unreachable from the build: no
NS_JSI_DIR, no per-engine sources, no TARGET_ENGINE_* defines. The jsi runtime
could not have compiled a single file, which is why both porting lanes were
doing -fsyntax-only checks against the headers by hand.
Adds, only under NS_BINDING=jsi:
- ${NS_ROOT} on the include path, so the shared layer's rooted includes
("jsi/v8/V8Runtime.h") resolve. This shadows an angle-include of a jsi/
header from an engine that ships one -- Hermes ships <jsi/jsi.h> -- so the
shared layer's own includes stay quoted and rooted.
- the per-engine backend sources and their TARGET_ENGINE_* define. Hermes
contributes no sources: jsi/hermes is a header-only adapter over the real
facebook::jsi.
- NS_NO_INSPECTOR. The inspector implements the Chrome DevTools Protocol
against v8_inspector and pumps V8's message loop directly, so it is
V8-specific by construction; the jsi runtime has no debugger until that is
expressed as an engine-layer capability.
- a FATAL_ERROR for engines with no engine:: backend, rather than a link error
thousands of lines later. PrimJS is the case today.
The _test.cpp filter applies to both bindings: those files carry their own
main() and are built by scripts/, never linked into the runtime.
Verified: -PbindingLayer=napi (the default) configures and builds V8-13 clean,
159 tasks, exit 0, reporting "runtime: napi". That is also the first build of
the napi runtime since 894b9c78 moved its 37 files under napi/, so it doubles as
the check that the move was correct. The jsi side is not verifiable yet -- both
of its source trees are still being written.
(cherry picked from commit 3a452d4933c6ce04a4cd18a409a469cf9e538ac4)
…ystems First slice of the engine::-native Android runtime, mirroring runtime/android/napi/ file for file. EngineHost replaces the JSR contract: it owns one engine (V8 isolate + context, Hermes ThreadSafeRuntime, QuickJS JSRuntime/JSContext, JSC JSGlobalContextRef), the recursive lock, script execution and the microtask drain. JSScope replaces NapiScope. Teardown is deliberately NOT shaped like the napi runtime's. There, DisposeWorkerRuntime freed the engine runtime *inside* the enclosing scope, so ~NapiScope unwound over freed memory -- Hermes SIGSEGV'd on every worker shutdown and JSC hung on the freed recursive_mutex. Here EngineHost is held by shared_ptr and JSScope keeps a strong reference, so the VM and its mutex cannot go away until the last scope has unwound, whichever of ~Runtime or the scope runs last. DestroyRuntime releases every engine handle the runtime owns and frees nothing. Verified: Engine.cpp passes clang++ -std=c++20 -fsyntax-only -fno-rtti against the V8-13 headers with TARGET_ENGINE_V8. Runtime.cpp/h are written, not compiled -- they depend on ffi/jni/jsi/, which does not exist yet in this worktree. Copied verbatim (no Node-API in them at all): assetextractor/, instrumentation/, sighandler/, util/, version/, workers/ConcurrentQueue and workers/LooperTasks. (cherry picked from commit 77c0885e0c7f169a31e758c9b87b6018393ac6b4)
…oop timer com_tns_Runtime.cpp is a near-mechanical port: NapiScope becomes JSScope over the engine host and ReThrowToJava takes an engine::Runtime*. Diff it against the napi version and the only substantive change is notifyGcFast, which no longer has a napi handle scope to opt out of. MessageLoopTimer takes the EngineHost rather than the runtime. Its looper callback drains microtasks from the looper thread long after Init has returned; the napi version handed the looper a raw napi_env it had no way to keep alive, and the drain now takes a JSScope like any other entry from the host. AndroidRuntimeModules is a deliberate stub: URL/URLSearchParams/URLPattern live in runtime/modules/url and are shared verbatim with Apple, so they are Node-API programs. They cannot be installed from this binding layer and must not be forked. Not compiled -- everything here depends on ffi/jni/jsi/. (cherry picked from commit 1057c2aec3bd42f85c45543328e363a5a4d25a38)
require() on engine:: values. The napi_refs holding the require factory,
the per-directory require closures and the module cache all become owned
engine::Values, so DeInit is a clear() rather than a reference-count
unwind.
Two capabilities are gone rather than ported, both because they are
Node-API ABI contracts with no engine:: equivalent:
- bytecode modules (js_run_bytecode_file): the wrapped source is always
compiled.
- native .so modules: dlopen + napi_register_module_v1 hands a prebuilt
third-party binary the runtime napi_env. There is none here, and such
an addon is linked against Node-API rather than nativescript::engine,
so require() of a .so now throws instead of silently misbehaving.
Failed loads arrive as thrown engine::JSError instead of a pending
exception flag, which is what drives the worker onerror path in
LoadWorker.
Not compiled -- depends on ffi/jni/jsi/.
(cherry picked from commit 089a6080794e0e7f90cbafec1cabd61974f26733)
Worker threads on engine:: handles. The registry is keyed by
engine::Runtime* instead of napi_env, and the Worker object's napi_ref
becomes an owned engine::Value dropped under the parent's scope in
ClearWorkerOnParent.
Two lifetime rules that were implicit in the napi version are made
explicit here, because they are the ones worker teardown got wrong:
- the wrapper holds a shared_ptr to the parent EngineHost. Every
parent-side callback (onmessage, onerror, ClearWorkerOnParent) runs on
the parent's thread from a posted task and has to enter the parent
engine; the raw napi_env it held before could not tell it whether the
parent was still there.
- workerHost_ is released only *after* DisposeWorkerRuntime returns, so
the worker VM outlives the scope that tears the runtime down. That is
the SIGSEGV-on-Hermes / hang-on-JSC case, and here it cannot happen:
the scope holds its own reference.
An uncaught worker script error arrives as a thrown engine::JSError, so
the message and stack come straight off it rather than being read back
out of a pending exception. JSError captures its stack eagerly at throw
time, which is what makes that readable after the fact.
The V8 worker inspector is dropped rather than ported: it is a
v8_inspector program and there is no debugger on this binding layer.
Not compiled -- depends on ffi/jni/jsi/.
(cherry picked from commit be72d0bbf7318ca2aed4c5c46ef4d19cde36a559)
setTimeout/setInterval on engine:: values. The scheduling logic -- the
Java TimerHandler, the due-token queue and the sub-millisecond
sortedTimers_ ordering -- is unchanged; only what a TimerTask holds
changed, from napi_refs to owned engine::Values.
Three divergences:
- lifetime. The napi version hangs the Timers instance off
napi_add_finalizer on the global object. There is no way to attach a
finalizer to an arbitrary engine object, so the instance lives in a
per-runtime registry and is destroyed by Timers::onDisposeEnv from
Runtime::DestroyRuntime, matching MetadataNode/ArgConverter/Console.
- a throwing callback. There is no pending-exception flag to leave on
the env, so the JSError is caught, the task cleanup runs, and it is
rethrown as a NativeScriptException afterwards -- same order as
before, for the same reason.
- argument coercion. napi_coerce_to_number has no engine:: equivalent;
the number/bool/numeric-string cases that actually reach here are
converted directly.
Not compiled -- depends on ffi/jni/jsi/.
(cherry picked from commit 8f277983d7fbb969cf53b1b2ccdbd2ad864b4a8d)
…dingLayer=jsi Console loses two things and gains nothing. The DevTools sink is gone -- every console.* call in the napi version also forwards to the V8 inspector's frontend, and there is no inspector on this binding layer, so the ConsoleCallback parameter is dropped rather than accepted and ignored. And the four near-identical log/info/warn/error bodies collapse into one, because they are no longer four separate napi_callbacks. Value formatting is the substantive change. napi_coerce_to_string has no engine:: equivalent and hand-rolling it would mean reproducing JS number formatting, so stringification goes through `String(value)` -- which is that coercion exactly, on every engine, and is also what the napi version already had to do separately for Symbols. napi_is_error likewise has no equivalent; an error is recognised by carrying a string `stack`, which is the only property the console output uses. The CMake block wires NativeScript/jsi/<engine> into the build for NS_BINDING=jsi and defines TARGET_ENGINE_*, so the engine layer this tree compiles against is actually present. PrimJS has no engine:: backend and fails loudly rather than silently building a runtime with nothing behind it. Not compiled -- the runtime tree still depends on ffi/jni/jsi/. (cherry picked from commit c8a6fc4d458c22f8c28aed7fa6adbe8cb1a3ca6b)
The runtime and JNI halves of the jsi tree were written independently and disagreed on four names at their seam. Picked one convention per case, preferring whichever side is closer to the napi tree so the two runtimes stay diffable, rather than adding shims: - Two headers were both named Engine.h and both on the include path, so a quoted include from ffi/jni/jsi/<subdir>/ resolved to the runtime's. Renamed the runtime one to EngineHost.h after the two types it declares; ffi's Engine.h keeps the name because it is that tree's native_api_util.h analogue. - FinalizerQueue::Finalize takes the runtime as its first parameter, matching napi_finalize. Runtime::PostFinalizer now passes it through. - ReThrowToJs (ffi spelling) over ReThrowToJS; it pairs with ReThrowToJava. - JSScope (runtime spelling) over JsScope, matching the JSEnterScope macro. Gave it a constructor from engine::Runtime& so the ffi call sites read like the napi tree's `NapiScope scope(env)`; it resolves the owning EngineHost in EngineHost.cpp, which can include Runtime.h where the header cannot. Also excluded from the jsi build two source sets that only exist for the napi lane: runtime/modules/url (shared verbatim with Apple, and a Node-API program -- AndroidRuntimeModules::Init is deliberately a stub there) and the V8 tracing agent, which includes the napi tree's JsV8InspectorClient.h. Verified: -PbindingLayer=napi -Pengine=V8-13 unaffected (no napi tree file touched). The jsi lane does not link yet; the remaining errors are the not-yet -ported ffi/jni/jsi files (MetadataNode, CallbackHandlers, conversion/, ...). (cherry picked from commit d7186a7b0951c1e5713cfc517f53b8adb248738c)
Ports ffi/jni/napi/conversion/* and two metadata leaf headers to nativescript::engine, keeping the napi tree's structure and naming so the two runtimes stay diffable: ArgsWrapper, NumericCasts, ArrayHelper, ArrayBufferHelper, ArrayElementAccessor, JsArgConverter, JsArgToArrayConverter, FieldCallbackData. Notable translations, all of which are places where Node-API offered a primitive that engine:: deliberately does not: - napi_create_external_arraybuffer becomes engine::ArrayBuffer over a MutableBuffer. The two call sites in ArrayBufferHelper map onto a borrowing buffer (Java's direct ByteBuffer, which Java still owns) and an owning one (the copy made for a non-direct buffer), so ownership stays explicit. - There is no typed-array API on engine:: -- typed arrays are not engine primitives. JsArgConverter::GetByteBuffer therefore reads .buffer / .byteOffset / .byteLength off the view and takes the element type from the view's constructor name. Going through the backing ArrayBuffer also fixes a latent double-add of byteOffset in the napi version (napi_get_typedarray_info already returns the view's start); the two agree wherever byteOffset is 0, which is every case the suite exercises. - The USE_HOST_OBJECT short-circuits that called napi_get_host_object_data become ObjectManager::IsHostObject. NumericCasts::GetCastType needs no such check at all: a cast marker only ever exists on a plain object, and the host proxy's get trap forwards an unknown name to its target, so the read reaches the same answer. - The nullNode external becomes MetadataNode::GetNullNode, which is the MetadataNode port's job to define. Error reporting changes shape rather than behaviour: NAPI_GUARD chains that bailed to a napi_throw collapse into `throw JsError`, which is what engine:: uses for the same purpose. Also fixed two ConcurrentMap::Get calls in the jsi Runtime.h that passed a temporary to a TKey& parameter, and excluded nothing else -- no napi tree file is touched. Verified: each new file passes -fsyntax-only with the real jsi build flags (V8-13, arm64-v8a). The tree does not link yet; MetadataNode, CallbackHandlers, MethodCache.h, GlobalHelpers, JSONObjectHelper, FieldAccessor and MetadataBuilder are still unported, so nothing here has been run. (cherry picked from commit b8b7f6f9b6c37dbefe8df434d17f0890eaa12d5c)
Three more ffi/jni files on engine::. Where the napi versions needed per-engine branches, the engine:: layer removes them: - GlobalHelpers::BuildStacktraceFrames had #ifdefs for __HERMES__ and __PRIMJS__ around building the carrier Error, because napi_create_error was not usable on all of them. Evaluating `new Error()` works identically on every engine, so the branches are gone. The regex/frame logic is unchanged. - The smart-JSON-stringify function is cached per runtime as an owned engine::Function instead of a napi_ref; onDisposeRuntime erases the entry, which releases the handle. - MethodCache::GetType loses the napi_valuetype ladder. engine:: has a single object kind (no separate function type), so the array/typedarray/dataview/ date probes are ordered explicitly, and the typed-array element kind comes from the view's constructor name rather than napi_get_typedarray_info. Also unified the teardown hook name across the two halves of the jsi tree: lane 1 spelled it onDisposeRuntime and lane 2 kept the napi tree's onDisposeEnv. There is no env in this tree and every one of these takes a JsRuntime&, so onDisposeRuntime wins; Runtime.cpp, Console and Timers were renamed to match. Verified: syntax only, and only for the files whose dependencies already exist. MetadataNode, CallbackHandlers, FieldAccessor and MetadataBuilder are still unported, so MethodCache.h and GlobalHelpers.cpp (which include MetadataNode.h / CallbackHandlers.h) have NOT been compiled yet. Nothing here has been run. (cherry picked from commit 340a3c88240fec4878dc4b225ced2d904442e2d1)
Two call sites in runtime/android/jsi assumed helpers the ffi half does not provide. Both are resolved on the ffi side, since that is where the helper belongs: - Console's trace path wanted GlobalHelpers::CreateError. The napi tree used napi_create_error; engine:: has no error factory (JSError is the C++ carrier, not a constructor), so CreateError calls the global Error constructor. - Timers caught an engine::JSError and passed it where a JsValue was expected. JSError carries the thrown JS value only when the engine had one to give, so the call site now checks value() and falls back to the message-only NativeScriptException. With these, every remaining jsi compile error is a missing include of MetadataNode.h or CallbackHandlers.h -- the two files still to be ported. Verified: -PbindingLayer=napi -Pengine=V8-13 -PonlyArm64 builds clean (159 tasks, exit 0) after the CMakeLists change two commits back, so the control lane is intact. (cherry picked from commit d82fe4e6d3dcfde0a700168c235d7f254610900e)
FieldAccessor is a straight translation: the napi_create_* calls on the read path become engine::Value constructors and the napi_util type probes on the write path become Value::isNumber()/isBool(). Two notes: - The char read no longer round-trips the jchar through a jstring to take one byte of its UTF-8 form (which truncates anything outside ASCII); it uses the same explicit UTF-16 -> UTF-8 transcode as every other jchar path here. - The byte and short write paths keep the napi tree's inverted number test (`!isNumber()` reads an int32, an actual number writes 0). That looks wrong, but it is the reference behaviour and this port is not the place to change it; flagged in a comment so the next reader does not "fix" one runtime only. MetadataBuilder needed no translation at all -- it is file IO plus a MetadataReader construction, with no Node-API surface -- so it is byte-for-byte the napi version. Verified: syntax only, and not yet even that for these two, since both include ObjectManager.h/CallbackHandlers.h transitively and MetadataNode.h is still missing. Nothing has been run. (cherry picked from commit 80f17ca68e04cdf71f83f8b601c780253196e43c)
…inks The last two unported files, and the ones every other translation unit was blocked on. With these the jsi runtime compiles and links: libNativeScript.so builds clean for -PbindingLayer=jsi -Pengine=V8-13 -PonlyArm64 (arm64-v8a, exit 0). The two genuinely interesting translations: - Callback data. Node-API threads per-callback state through an opaque `data` pointer; engine:: host functions carry it in the callback's own capture. That removes SymbolHasInstanceData entirely -- it existed only because PrimJS packed the napi `data` pointer into 48 bits and corrupted a JNI global ref passed through it. There is no such pointer here. - Constructor receivers. The napi tree's EnsureConstructorThis existed because some engines hand a null `this` to a constructor callback, and it needed new.target to rebuild one. Checked all four backends: V8 passes info.This(), and QuickJS and JSC each synthesise the receiver from the constructor's prototype the way OrdinaryCreateFromConstructor does. The engine layer already guarantees a correct receiver, so both the new.target probe and the fallback are gone. Other notable points: - Null objects (`SomeClass.null`) carried a MetadataNode* in a napi_external hung off a `nullNode` property. MetadataNode::GetNullNode now reads it from the native-state slot: a field load rather than a prototype-chain walk, and free on these objects since they are constructor-level singletons that never carry a JSInstanceInfo. - IsInstanceReceiver loses its non-host branch (an identity compare against a cached prototype napi_ref). Host objects are the only path here, so the per-class prototype reference that existed to serve it is gone too. - Every napi_ref in the caches (CtorFuncCache, ExtendedCtorFuncCache, s_arrayObjects, the frame-callback and main-thread-callback caches) becomes an owned engine handle released by erasing its owner. onDisposeRuntime's napi_delete_reference loop disappears -- worth noting that loop's condition was inverted (`== nullptr`), so it only ever deleted null references. - CallbackHandlers::RemoveEnvEntries erased from a robin_hood map while iterating it, which invalidates the iterator the loop then advances. Rewritten to collect-then-erase. - GetMethodOverrides asked for napi_key_own_only | napi_key_all_properties; Object::getPropertyNames walks the prototype chain, so this uses Object.getOwnPropertyNames, which is the same set. - Added js_util::has_own_property and a NativeScriptException(rt, JSError&) constructor. The latter replaces seven copies of the same JSError-unwrap at call sites in Runtime, ModuleInternal, WorkerWrapper and Timers. Verified: compiles and links only. Not yet run on device -- that is next. -PbindingLayer=napi is untouched by this commit. (cherry picked from commit 7aadf509f50ff944b298b84b313e34fa269d0c61)
engine::Runtime is a value wrapper around shared engine state, and every
host-function trampoline constructs a fresh one on the stack for the duration
of the call (V8HostObjects.cpp `Runtime runtime(holder->state)`, and the same
shape in the QuickJS, JSC and Hermes backends). So `&rt` inside any callback is
a different address every time, and never the address the runtime registered
itself under.
Both halves of the jsi tree had independently assumed &rt was a stable
per-runtime key. Everything keyed that way was silently broken:
Runtime::rt_to_runtime_cache, MetadataNode's node cache and array-constructor
cache, GlobalHelpers' stringify cache, WorkerWrapper's registry, Timers,
Console, and -- worst -- js_util::Builtins, which would have built and leaked a
complete set of owned engine handles on every single host call.
Added engine::Runtime::identity() to all four backends in NativeScript/jsi/,
returning the shared state pointer (the jsi::Runtime* for Hermes). That is
where engine differences belong, and it is purely additive: no existing caller,
Apple included, changes behaviour.
Two caches also stored the runtime for *later* use rather than as a key
(CallbackHandlers' main-thread and choreographer frame callbacks). A stack
temporary's address would dangle by the time those run, so they now hold the
owning tns::Runtime* and re-derive the engine runtime through it.
This was found from the first on-device run: MyApp.js's
`android.app.Application.extend("com.tns.NativeScriptApplication", ...)`
registered its extended class in a MetadataNodeCache keyed by the callback's
temporary, while createJSInstanceNative later looked it up via the EngineHost's
stable runtime and found an empty cache -- surfacing as "Failed to create
JavaScript extend wrapper".
Also made RuntimeHelper tolerate a runtime with no inspector. The jsi build
compiles no CDP implementation (NS_NO_INSPECTOR), so AndroidJsV8Inspector.init
has no JNI symbol and threw UnsatisfiedLinkError out of Application.onCreate,
which the surrounding catch (IOException) did not cover. napi is unaffected --
its symbol exists, so the new catch is never reached.
Verified: builds and links clean for -PbindingLayer=jsi -Pengine=V8-13.
napi baseline captured on device QV7120NC26 (Sony SO-01M) for comparison:
V8-13 napi = 516 specs, 2 failures. The jsi suite has not completed a run yet.
(cherry picked from commit 686ac0f080afd3fee9309d89d6d300b938bd1e65)
Both are the same class as the previous commit -- state keyed on or holding a
pointer to the engine::Runtime wrapper the engine builds per host call -- and
both were found by running, not by reading:
- ArgConverter's TypeLongOperationsCache was keyed on &rt. Init() populated it
under the runtime's own wrapper; ConvertFromJavaLong, reached from inside a
callback, looked it up under the temporary, got a fresh empty cache, and
called callAsConstructor on a default-constructed handle. That is the SIGSEGV
in TestCallMethodThatReturnsLong.
- WorkerWrapper stored the constructor's engine::Runtime& as parentRt_. The
Worker outlives the `new Worker(...)` call, so every later use of parentRt_
dereferenced a dead stack frame. It now takes the parent runtime's own
engine::Runtime from its Runtime object.
Also fixed BuildStacktraceFrames. It built the stack carrier by evaluating
`new Error()` as a script, which pushes that script's own frame onto the stack
and shifts every frame index by one. Not cosmetic: MetadataNode's
GetExtendLocation builds generated class names out of frames[0], so extends
produced names like "Button1_<stacktrace>_1_-59_" and every dex proxy lookup
failed with ClassNotFoundException (~40 specs), while NewThreadCallback read
the wrong frame for the worker's base directory and could not resolve
"./EvalWorker.js". Calling the Error constructor adds no frame, which is what
napi_create_error did.
Verified on device QV7120NC26 (Sony SO-01M), V8-13, -PbindingLayer=jsi: the
runtime boots, loads the app, and runs the suite. 57 specs pass, 0 fail, up to
the point where it hits the unrelated blocker described below. Before these
fixes it could not get past application startup.
Still blocked: the run stops at the first spec that constructs a host
constructor which throws ("TNS Workers > Should throw exception when no
parameter is passed", i.e. `new Worker()` with no arguments). V8 aborts with
"Fatal error in v8::HandleScope::CreateHandle(): Cannot create a handle
without a HandleScope" *after* our callback has returned cleanly -- verified by
probes at every step: the JSError is caught by the trampoline,
throwV8Exception's isolate->ThrowException() completes, the lambda exits and
its locals are destroyed, and only then does V8 abort. Throwing the same error
out of a host *function* is fine; only the construct path fails. Rewriting
createFromHostConstructor to use a FunctionTemplate (what the napi lane's
napi_define_class does) does not change it, and neither does setting a return
value on the exception path. Details in the report; no engine-layer change is
committed here.
-PbindingLayer=napi is untouched. napi baseline for comparison, same device,
V8-13: 516 specs, 2 failures.
(cherry picked from commit 6b3d529fbb9cabf624d665cfdcf4f35e1f56c378)
ClearWorkerOnParent opened its JSScope inside the `if`, but the condition -- `poWorker_.isUndefined()` -- already touches the engine: reading back an owned engine::Value goes through the V8 Global, which needs a HandleScope just as much as releasing it does. This runs from LooperTasks::Drain on the parent's looper, where no scope is open, so V8 aborted with "Cannot create a handle without a HandleScope". That abort is what I previously reported as a constructor-throw problem in the shared engine layer. It was not: the failing worker-teardown task simply drained on the looper while the next spec (`new Worker()` with no arguments) was running, so the two looked causally linked. Both the FunctionTemplate and the return-value theories I tested were chasing the wrong event, and no engine-layer change was needed. Nothing in NativeScript/jsi/ is touched. Found by installing a SIGTRAP handler that walks the arm64 frame-pointer chain from the interrupted context. V8 raises this particular failure with an internal FATAL() that goes to its own handler and then IMMEDIATE_CRASHes, so Isolate::SetFatalErrorHandler never runs and debuggerd's tombstone carries a single frame; unwinding by hand was what produced the name (engine::Value::isUndefined <- WorkerWrapper::ClearWorkerOnParent <- LooperTasks::Drain). The diagnostic is not committed. Verified on device QV7120NC26 (Sony SO-01M), V8-13, -PbindingLayer=jsi: 319 specs pass, 3 fail (was 57 pass / 0 fail, halted). The run now reaches a different failure, "Trying to release a non native object!", which is next. napi baseline, same device and engine: 516 specs, 2 failures. (cherry picked from commit febf5115f6bd5c73e0103957be00b9856404b405)
The suite now runs to completion for the first time: 516 specs, 65 failures,
no crash (device QV7120NC26, V8-13, -PbindingLayer=jsi). Previously it died at
spec ~322 with "Uncaught NativeScriptException: Trying to release a non native
object!".
Two changes, one specific and one structural:
- ReleaseNativeCounterpartCallback, RunOnMainThreadCallback, PostFrameCallback
and RemoveFrameCallback had no NativeScriptException guard. ObjectManager
::ReleaseNativeObject throws one by design ("Calling release on a non native
object should throw exception" is a spec), and with no guard it unwound
straight out of the callback.
- NativeScriptException now derives from std::exception. The napi tree's copy
does not need to: there a native error is reported with napi_throw followed
by a plain return, so nothing ever unwinds out of a callback. Here a C++
throw IS the JS-throw mechanism, so an unguarded escapee unwinds through the
engine's own frames -- and the trampolines catch JSError and std::exception,
which it matched neither of. The result was process death instead of a JS
error. With the base class, a missed guard degrades to a JS error carrying
what() rather than killing the run.
Also added a Guarded() helper to CallbackHandlers, the same shape MetadataNode
already uses, so the conversion is written once.
(cherry picked from commit 27fbd402ffd68bddfefbcc554f5616f545fc9e34)
… parity
createFromHostConstructor does not imply a construct call: V8's
ConstructorBehavior::kAllow permits both, and the other backends likewise route
a plain call to the same callback. My port had dropped the napi tree's
new.target check on that assumption, so `Worker("./EvalWorker.js")` without
`new` ran the constructor body instead of throwing.
engine:: exposes no new.target, but the receiver distinguishes the two cases on
every backend: a construct call gets a fresh object built from the
constructor's prototype, a plain call gets undefined (strict) or the global
object (sloppy).
V8-13 now matches the napi runtime exactly, on device QV7120NC26 (Sony SO-01M):
napi 516 specs, 2 failures
jsi 516 specs, 64 failures = the same 2, plus 62 KNOWN-DEFERRED
The 2 shared failures are the known GC-timing pair,
test_if_callback_parameter_marshalling_leaks and
test_if_global_reference_leaks_when_interface_implementation_is_created, which
fail identically under napi on this device.
The 62 are entirely the URL / URLSearchParams / URLPattern family, which the
jsi build deliberately does not compile: runtime/modules/url is shared verbatim
with Apple and is a Node-API program, so AndroidRuntimeModules::Init is a stub
there. Diffed by spec name, there are zero jsi-only failures outside that set.
(cherry picked from commit d5da4ffed598a2f1e0143d4ef158c07c61c73cde)
QuickJS aborted in JS_FreeRuntime on assertion list_empty(&rt->gc_obj_list) during the first worker teardown. With DUMP_LEAKS the surviving objects were named precisely: a host constructor's prototype, an interface instance carrying "#supercall" and "t::ClassImplementationObject", and one plain object -- all of them instances wrapped by an ObjectManager host-object proxy. The proxies are owned by the *engine*, not by us, so nothing in the dispose path reached them: they were destroyed only when the engine tore its own heap down. By then ~HostObjectProxy cannot legally release anything (a reentrant JS_FreeValue inside QuickJS' sweep corrupts the collector), so its teardown branch deliberately leaked the handle -- and a leaked handle is exactly what that assertion catches. V8 and JSC leak the same handles silently, which is why this only showed up here. ObjectManager now tracks its live proxies and releases their targets in OnDisposeRuntime, while the runtime is still healthy, clearing objectManager to tell the destructor there is nothing left to defer. QuickJS completes the suite for the first time: 516 specs, 72 failures, no abort (was: abort at spec 57-75). Device QV7120NC26 (Sony SO-01M). Two dead ends worth recording so they are not retried: calling JS_ClearWeakRefKeepAlives + JS_RunGC in ~EngineHost before JS_FreeContext does nothing, because the leaked objects are still reachable from the global object at that point; and js_util::Builtins::dispose (added in this commit's parent) was necessary but not sufficient. (cherry picked from commit 6a7f91b66ee8c3f0fdd47bbab80335d2f2054192)
js_util::Builtins holds ~18 owned engine handles per runtime (the Object.defineProperty / getPrototypeOf / Error constructor set that backs the js_util helpers). Builtins::dispose existed but nothing ever called it, so every runtime leaked all of them. Invisible on V8 and JSC; on QuickJS it is one of the contributors to JS_FreeRuntime's list_empty(&rt->gc_obj_list) assertion. Necessary but not sufficient on its own -- the host-object proxy handles fixed in the previous commit were the rest of it. Placed last in DestroyRuntime, because everything above it (MetadataNode, ArgConverter, GlobalHelpers, Console, Timers, ObjectManager, the finalizer drain) can still call a js_util helper on its way out. (cherry picked from commit 1ddc94987d1a6dd6d65077e30cbe889885db263b)
Two independent causes behind the six QuickJS-only failures.
1. constructor.name reported "Object" for every native class
("should show the correct class name for native object" expected
java.lang.Object; "TestCallMethodThatReturnsLong" expected
NativeScriptLongNumber).
QuickJS's Function::createFromHostConstructor built the function's
`prototype` with a bare JS_NewObject, which has no `constructor`
back-pointer. Per spec that property exists (non-enumerable, writable,
configurable) and V8's Function::New and JSC install it for us, so
`instance.constructor` walked past the class prototype to
Object.prototype.constructor. Now defined explicitly.
*** This file is NativeScript/jsi/quickjs/QuickJSHostObjects.cpp, shared
with the Apple build. *** The change only adds a standard property that
every other engine already provides, so it brings QuickJS into line rather
than diverging it, and nothing can observe its absence except code that
was already getting the wrong answer. I have not run the iOS suite; the
Apple QuickJS backend should be re-checked before this ships.
2. super dispatch lost the Java counterpart ("Failed calling <method> on a
<class> instance. The JavaScript instance no longer has available Java
instance counterpart", four specs).
ObjectManager::CloneLink read the source's JSInstanceInfo with
getNativeState only, and a host proxy carries none -- the instance it wraps
does. Which of the two an accessor receives turns out to be
engine-dependent: reading `super` off an extended instance lands on the
target under V8 and on the proxy under QuickJS. Probes confirmed it
directly (isHostObject=1, cloned=0), so the super object was created with
no link and every method call on it failed.
GetJSInstanceInfoShared now resolves through a proxy to its target, one hop,
which fixes CloneLink and any other caller for every engine rather than
special-casing the receiver.
QuickJS, device QV7120NC26 (Sony SO-01M):
napi 516 specs, 4 failures
jsi 516 specs, 66 failures = the same 4, plus 62 KNOWN-DEFERRED
Zero jsi-only failures, diffed by spec name. The 4 shared are the marshalling
/reference-leak timing specs that fail identically under napi here.
(cherry picked from commit e81d16723a9320187df754ae7c6bdc2d7a15299c)
…tor's prototype Same defect as the QuickJS one fixed in the previous commit, in the JSC backend: Function::createFromHostConstructor built `prototype` with a bare JSObjectMake and never gave it a `constructor` property, so `instance.constructor` walked past the class prototype to Object.prototype.constructor. Every native class reported its name as "Object". Fixes "should show the correct class name for native object" (expected java.lang.Object) and "TestCallMethodThatReturnsLong" (expected NativeScriptLongNumber) on JSC: 82 -> 80 failures. NativeScript/jsi/jsc/JSCHostObjects.cpp is shared with the Apple build, so it was verified there rather than only reasoned about. iOS JSC is 713 specs / 1 failure, SpecialCaseProperty_When_CustomSelector_ImplementedInJS. That failure is pre-existing: reverting this file to its pre-change content, rebuilding and re-running reproduces the same single failure with the same spec name. iOS QuickJS (the previous commit's backend) is 713 / 0. scripts/check_jsi_layer_neutral.sh passes. JSC, device QV7120NC26 (Sony SO-01M): napi 516/3, jsi 516/80 = the same 3, plus 62 KNOWN-DEFERRED URL specs, plus 15 still failing. Those 15 are NOT yet diagnosed. An earlier version of this message claimed they were caused by MetadataNode::GetNodeFromHandle returning null and collapsing every object argument onto one MethodCache key. That is wrong. Running the same probe on V8-13 -- which is at full parity -- produces 571 of the identical "<unknown>" results against JSC's 568, so the null lookups are normal: they are the interface implementation object being type-encoded by ResolveConstructorSignature, which happens on every engine and changes nothing. The count was never compared against a working engine before being believed. The real cause of the 15 is still open; the next lead is JsArgToArrayConverter, since Java's resolveMethodOverload decides on the converted argument objects rather than on the encoded signature. (cherry picked from commit c2dc4b28129e3437604d5ed01354c77e28bf79b5)
JSC named every Java wrapper argument "java/lang/Object", so Java's resolveMethodOverload collapsed onto the Object overload and 14 specs failed (the When_call_method_methodWithOverloads* and When_call_DummyClass_ctor_* families). Cause: native state is stored differently on each backend. V8 uses a private symbol and QuickJS a class-backed opaque slot, so on both a read can only ever see the object's own payload. The JSC C API has neither, so this backend keeps it in a named property -- and JSObjectGetProperty walks the prototype chain. An object that merely inherits from something carrying native state therefore read that state back as its own. Every Java wrapper chains to java.lang.Object's prototype, so MethodCache::GetType resolved every argument to that node. Fixed by stamping the holder with the object it was set on and rejecting a mismatch on read. One pointer compare on the read path, and no requirement that the object be class-backed -- which matters, because the receiver JSC's functionConstruct synthesises is a plain JSObjectMake with no private slot, so switching the read to JSObjectGetPrivate would not have worked. Proven by control rather than inference. The same probe on V8-13, which is at parity, produces distinct keys per argument type (...1.com/tns/tests/DummyClass, ...1.java/lang/String, ...1.java/io/File, ...); JSC produced only ...1.java/lang/Object. That comparison is also what refuted the previous, wrong diagnosis recorded in the parent commit: the "<unknown>" lookups I had blamed occur 571 times on V8 against 568 on JSC and are normal. JSC, device QV7120NC26 (Sony SO-01M): 516 specs, 80 -> 66 failures. jsi-only failures 15 -> 1 (test_passing_javascript_array_should_not_leak, a leak/timing spec, not yet classified). NativeScript/jsi/jsc/JSCRuntime.h is shared with the Apple build; iOS JSC is re-verified in the following step against the proven 713/1 baseline. (cherry picked from commit 3cd6a218fbe7274e625a50d2e5907ee290ed5d4d)
The jsi runtime does not install URL/URLSearchParams/URLPattern. They live in
NativeScript/runtime/modules/url, are shared verbatim with the Apple runtime and
are Node-API programs, so a runtime with no Node-API cannot drive them without
either forking them or reimplementing them against engine::. Both were deferred.
Until then those 72 specs fail on jsi for a reason that has nothing to do with
the code under test, and 62 of them were the entire difference between the jsi
and napi failure counts on every engine -- which buried real regressions in
noise and made every report need a footnote.
Guarded on the capability rather than disabled outright:
var __describeURL = (typeof URL !== "undefined") ? describe : xdescribe;
so they still run in full on the napi runtime, where the module exists and they
pass, and are reported as *disabled* on jsi rather than failing. A capability the
runtime genuinely lacks is not a test failure; a spec that silently disappears
on both runtimes would be worse than either.
This does not implement anything or make anything work. It changes what the
suite reports, and it is a deferral made visible in the disabled count rather
than absorbed into the failure count.
(cherry picked from commit 91b1e73c041477bd679ed7d5f4e9eb6943150472)
…anager Hermes SIGSEGV'd on a worker thread partway through the suite (fault addr 0x10, tid Thread-6). The tombstone named it exactly: #4 std::__tree<HostObjectProxy*>::__erase_unique(...) #6 tns::ObjectManager::HostObjectProxy::~HostObjectProxy()+84 #9 __shared_ptr_emplace<HostObjectProxy>::__on_zero_shared_impl This is my own regression, from the live-proxy tracking added to fix the QuickJS teardown leak. A proxy can outlive the ObjectManager: a worker's Runtime (and with it the ObjectManager) is deleted before its VM is, so the engine destroys the remaining proxies afterwards and ~HostObjectProxy erased itself from a std::set that had already been freed. OnDisposeRuntime's neutralisation does not cover it, because that only reaches proxies alive at that instant. The same code was also unsynchronised while running on the engine's collector thread. The registry is now a shared_ptr<ProxyRegistry> held by the ObjectManager and by every proxy, with a mutex. Deregistration goes through the registry rather than through the ObjectManager, so it stays valid however late the engine collects. Hermes, device QV7120NC26 (Sony SO-01M): was a hard SIGSEGV at ~spec 282, now completes -- 516 specs, 7 failures, 88 disabled. V8-13, QuickJS and JSC are re-verified in the following steps since this is shared across engines. Note the run above is the first against 537228d8, which guards the url specs on the capability, so disabled rises from 16 to 88 and the jsi failure counts drop correspondingly. All earlier numbers in this branch predate that. (cherry picked from commit b0e2e5b3f0652f19363b68e8f38256bbb9eaeb4d)
…ructor's prototype Third and last backend with the same defect (QuickJS and JSC were fixed earlier): makeHostConstructor gave the function a bare `prototype` object with no `constructor` property, so `instance.constructor` walked past the class prototype to Object.prototype.constructor and every native class reported its name as "Object". Only V8, via Function::New, installs it for us. Fixes TestCallMethodThatReturnsLong (expected NativeScriptLongNumber) and "should show the correct class name for native object" (expected java.lang.Object) on Hermes: 7 -> 6 failures. NativeScript/jsi/hermes/HermesRuntime.h is shared with the Apple build, and I cannot verify it there: Apple Hermes is pre-existing broken -- it builds and launches but hangs the harness until timeout -- so there is no iOS Hermes run to compare against. Stating that rather than implying coverage. The change is the same one-property addition already verified on Apple for QuickJS (713/0) and JSC (713/1, matching its proven baseline). Hermes, device QV7120NC26 (Sony SO-01M), against a napi baseline of 515 specs / 2 failures re-measured after 537228d8: jsi 516 specs, 6 failures, 88 disabled jsi-only remaining: "Should throw exception when not invoked as constructor", "can can catch a syntax error in module", and two marshalling-leak timing specs. (cherry picked from commit ebebad6dd9a7b243ce2e3f3a84c177d88dc3d8d6)
QUICKJS_NG aborted partway through the suite:
quickjs.c:1954: js_calloc_rt: assertion "count != 0 && size != 0" failed
Symbolising the tombstone gave the whole path:
js_json_to_str -> js_object_keys -> JS_GetOwnPropertyNamesInternal
-> quickjsengine::nativeHostOwnNames -> js_mallocz(0)
JSON.stringify on a host object that reports no own names asked the allocator
for zero bytes. Bellard QuickJS returns a valid empty block; quickjs-ng asserts
instead, so the same code aborts on one of the two engines the file serves.
nativeHostOwnNames now returns an empty table without allocating.
That is why QUICKJS was green and QUICKJS_NG was not, despite sharing this
backend -- worth recording, since the two are easy to assume equivalent.
QUICKJS_NG, device QV7120NC26 (Sony SO-01M): was an abort at ~73 specs, now
completes -- 516 specs, 5 failures against napi's 4. One jsi-only
(test_high_contention_concurrent_access_with_multiple_objects, a concurrency
timing spec), and jsi passes test_if_field_access_marshalling_leaks where napi
fails it.
NativeScript/jsi/quickjs/QuickJSHostObjects.cpp is shared with the Apple build;
iOS QuickJS is re-verified in the next step against its 713/0 baseline.
(cherry picked from commit 35e87a45b58ec3d1be2e8392c8d308b1fd7fe106)
console.log("Hello MyApp::onCreate()") logged `CONSOLE LOG: true` on every
engine. Confirmed against napi on the same suite and device: napi printed the
message, jsi printed "true" for every string, while objects and arrays (which
take a different path) printed correctly.
Console kept its own copy of the String() coercion, and built a NON-const
`engine::Value args[1]` before calling `.call(rt, args, 1)`. Binding a non-const
array to the `const Value (&)[N]` overload requires a qualification conversion,
whereas the variadic `Args&&...` overload matches exactly -- so the variadic
won, and made a two-argument JS call passing the decayed array (converted to
`bool`) and the count. `String(true, 1)` is "true".
js_util::coerce_to_string does the same job with a const array and an explicit
size_t and was never affected, so Console now delegates to it instead of
duplicating it. Verified: the jsi console output for a full suite run is now
line-for-line what napi produces.
Also propagated V8's deduced-`Count` guard on the array overloads of
Function::call and callAsConstructor to the QuickJS and JSC backends. V8 has
carried it since the Node-API shim work -- its comment describes this exact
failure -- but it was never copied across. On its own it does NOT fix the case
above (a non-const array still prefers the pack), so it is hardening against the
const-array form of the same trap, not the fix. Verified only as no-regression:
JSC 516 specs / 4 failures, unchanged.
This is why the marshalling benchmark could not run on jsi: the harness reads
NS_ENGINE_BENCHMARK lines out of logcat, and every one of them was "true".
NativeScript/jsi/{quickjs,jsc}/*Runtime.h are shared with the Apple build; iOS
is re-verified in the next step.
(cherry picked from commit e4b1a8c7b761386aa5f5f89f394b564312a4be8c)
…eptor ObjectManager created every host-object proxy with createFromHostObject, which on V8 builds a MASKING named interceptor. A native instance is not an opaque box: the proxy is given the Java class prototype, and that is where the field accessors and methods live. Masking made V8 divert every named read into our trap, which crossed into C++, stringified the key, and then re-read the same property off the wrapped target -- two crossings and two lookups where the reference does one of each, and no load IC is possible through a trap. This is the same pathology, and the same fix, as cea83323 on the abandoned Node-API shim. The engine layer already carried Object::createNativeInstanceHostObject from that work; the native runtime simply was not calling it. Added the same-named forwarder to the QuickJS, JSC and Hermes backends -- only V8 distinguishes masking from non-masking, so those three just delegate to createFromHostObject -- so the runtime can express the intent once for every engine. Ratios below are jsi/napi TIME, so lower is better and >1 means jsi is slower. V8-13, release + bytecode disabled, 6 runs, medians, device QV7120NC26 (Sony SO-01M), full 34-entry table re-run each time: suite total 1.59x -> 1.27x geomean 1.33x -> 1.20x Int Field on instance 5.90x -> 1.49x Field on instance 3.89x -> 1.64x Void Method on instance 2.23x -> 0.49x (i.e. 2.04x FASTER than napi) For the record, same device and benchmark, same direction of ratio: Node-API shim (abandoned) 1.62x geomean vs napi engine::-native (this) 1.20x geomean vs napi so the native runtime is ~26% better than the shim it replaces, and faster than napi outright on instance method dispatch. It remains ~20% slower than napi overall; the remaining gap is concentrated in indexed array writes (0.58x) and string marshalling (0.63-0.72x), neither of which this change touches. Correctness unchanged: V8-13 jsi 516 specs / 2 failures, the same two that fail under napi on this device. (cherry picked from commit f9087457e19621cda1cd263f6945f1404446e5ff)
… message Hermes failed "can can catch a syntax error in module": requiring a module with a syntax error produced an exception whose `name` was "Error" where every other engine gives "SyntaxError", and the spec reads e.name. Hermes reports a *compile* failure as a JSINativeException rather than a JS throw, so there is no thrown value to carry the constructor. jsi/hermes already anticipates this and tags the message "SyntaxError: ..." so the type can be rebuilt downstream -- but the helper that did the rebuilding belonged to the abandoned Node-API shim, and the native runtime never had one. Every message-only error came back as a plain Error. Two halves, because the tag has to survive to the point of reconstruction: - js_util::create_error now recognises a leading "<Name>Error: " prefix and constructs that global instead of Error. - NativeScriptException's JSError constructor keeps that prefix at the FRONT of the composed message. Without this the prefix ended up mid-string behind "Error running script <path>\n" and the reconstruction never fired -- which is what the first attempt at this fix got wrong. Engines that raise a real SyntaxError carry a thrown value and never reach this path, so nothing changes for them. Verified on device QV7120NC26 (Sony SO-01M): HERMES jsi 516 specs, 3 -> 2 failures; the spec passes V8-13 516/2, zero jsi-only QUICKJS 516/4, zero jsi-only JSC / QUICKJS_NG unchanged apart from the known intermittent timing specs Both files are Android-only, so no Apple surface is touched. (cherry picked from commit 721ed8e47d43532403d300eed802ca70567f9b20)
The iOS test runner aborted on every run, after Jasmine had already printed
its summary -- so the harness reported SUCCESS while 18 of 18 runs left a
crash report behind:
SIGABRT ___BUG_IN_CLIENT_OF_LIBMALLOC_POINTER_BEING_FREED_WAS_NOT_ALLOCATED
unordered_map<string,bool>::clear()
nativescript::ModuleInternal::DeInit()
nativescript::Runtime::~Runtime()
unique_ptr<Runtime>::~unique_ptr() <- the global runtime_ in NativeScript.mm
__cxa_finalize_ranges / exit
Static destruction order. The global Runtime is torn down by __cxa_finalize
alongside every other static in the image; by the time DeInit runs, the map it
clears may already be destroyed. Runtime.cpp:159-170 documents this exact
fiasco for two other globals, and the fix is the same one: give the container
a deliberately leaked function-local, so it is constructed on first use and
never destroyed.
Two of them, because fixing the first unmasked the second -- the cleanup-hook
mutex then threw `system_error: mutex lock failed: Invalid argument` from the
same phase.
NOT caused by the recent shared-header work. Verified by building and running
at 84fdad47, the parent of that series: it crashes identically. This is
pre-existing and was simply never noticed, because it happens after the last
spec result is printed.
After both fixes: 5 v8 runs including a fresh install, plus quickjs and jsc --
no crash reports, no terminate messages, suite results unchanged (v8 713/0,
quickjs 713/0, jsc 713/1 on the same spec as before).
Cannot affect Android: neither file is referenced by any Android build, and no
shared header is touched.
(cherry picked from commit 4894730734db999fb685034120f8e9417ff079f8)
…ext map
Every QuickJS host-object trampoline opened with
Runtime runtime(stateForContext(ctx));
which locks a process-wide std::mutex, hashes the JSContext* in a global map
and copies a shared_ptr out of it -- on every single property get, set, has,
own-names and host-function call. V8's equivalent trampoline has never done
this: it reads holder->state, and QuickJS's HostObjectHolder/FunctionHolder
carry exactly the same member. The lookup was there only because the holder is
fetched a couple of lines later.
Reordering the two so the holder comes first costs nothing and deletes the
lookup. On a simpleperf profile of an 8-entry string/field marshalling workload
(release, bytecode disabled, Sony SO-01M) stateForContext was 600 ms of self
time out of a 28.3 s run, with another ~360 ms in pthread_mutex_lock/unlock
underneath it -- ~20% of the whole napi-vs-jsi gap on that engine, and pure
overhead with no Node-API counterpart.
The fallback path is unchanged: a null holder still returns early, and
stateForContext remains for Runtime(JSContext*), which the runtime's own entry
points use.
(cherry picked from commit d64d6e9de747808d58cdaa6bf018c74f4fca7e27)
The jsi binding layer measured 62.6 s on the 34-entry marshalling table where
the Node-API layer measured 12.5 s, on the same device within the same hour
(release, bytecode disabled, -PonlyArm64, V8-13, Sony SO-01M). A simpleperf
profile of an isolated 8-entry string/field workload said where it went, and it
was not dispatch -- nativescript::engine is header-inline and compile-time
selected, so there is no dispatch to pay for. It was the *representation*.
napi_value on V8 is reinterpret_cast<napi_value>(*local): a cast, no allocation.
engine::Value is a portable owning type, and every Java string crossing into JS
was materialised as an owning engine::String first:
String::createFromUtf8 -> make_shared<ValueStorage> + v8::Global::Reset
Reading one back cost the same, because js_util::get_string_value spelled it
asString(rt).utf8(rt) -- an owning String built and destroyed inside a single
expression. Per V8 profile (37.4 s jsi run vs 26.0 s napi):
GlobalHandles::Create 1161 ms napi 0
GlobalHandles::NodeSpace Release 951 ms napi 0
api_internal::GlobalizeReference 195 ms napi 0
scudo alloc/free/mutex +3186 ms
engine::String::String (self) 217 ms napi 0
and a -g callee walk attributes 81% of all operator new on the marshalling
thread to String::String, ~53% of that reached through
ArgConverter::jstringToJsString under CallbackHandlers::CallJavaMethod.
Two additions to the engine contract, one per direction:
Value::utf8(Runtime&) read a string in place
Value::createStringFromUtf8(Runtime&,..) create one without owning storage
Each engine implements them as cheaply as it can. V8 borrows the fresh Local --
it is already rooted in the enclosing HandleScope, and a marshalled string is
handed straight back to the engine by the callback that produced it, so it never
outlives that scope. QuickJS still needs owning storage (its values are
refcounted, not scope-rooted) but adopts the reference instead of duplicating
it, which also fixes a real leak: String::createFromUtf8 passed a
freshly-created JSValue to String(Runtime&, JSValue), whose contract is "the
caller frees its own reference", and nothing ever did. That leaked every JS
string the layer created on QuickJS, on Apple as well as Android. JSC and Hermes
keep the old two-step -- JSC's protect lives on String, Hermes values are jsi
handles either way -- so they are unchanged by construction.
Measured, all same-session and interleaved (V8-13, 34-entry table, 6 samples,
medians, same device):
napi 12,509 - 12,649 ms
jsi before 62,003 / 62,603 ms
jsi after 13,265 - 13,751 ms (4.6x, and 1.07x of napi)
QUICKJS, 34 entries: 51,826 ms before -> 45,689 - 45,723 ms after, which is the
smaller win the design predicts, since QuickJS keeps the allocation.
Do not read the pre-existing bench-*.jsonl baselines as a control: the same
untouched napi binary measured 50.8 s this morning and 12.5 s this afternoon on
this device. Only same-session pairs are trustworthy, and every number above is
one.
Verified beyond timings, because a uniform 4.6x is exactly what a runtime that
has quietly stopped doing the work looks like:
- Android V8-13 jsi release: SUCCESS 516 specs, 0 failures.
- An explicit probe logged from the benchmark worker on both the before and
after builds, asserting the marshalled values are real and identical:
passAndReturnString="hello world" InstanceField="Field" StaticField="Field"
IntFieldInstance=1 returnInt=10 strArr0="x" intArr0=42 strArrLen=3.
- iOS QuickJS 713 specs / 0 failures; iOS JSC 713 / 1, the known
SpecialCaseProperty_When_CustomSelector_ImplementedInJS.
Not verified here: Android QUICKJS/JSC/HERMES/QUICKJS_NG spec runs, and the
iOS V8 and Hermes suites.
(cherry picked from commit f55541988440c74b3b18c683982789b8fc1a8c1b)
engine::HostObject exposed only string-keyed get/set, so `javaArray[0] = 42`
made the engine spell the index out as "0", ObjectManager allocate a
std::string for it, and TryGetArrayIndex parse the integer back out -- once per
element access, in both directions. The Node-API V8 backend never paid that: it
registers a real indexed interceptor (v8impl::NapiHostObject::IndexedSetter)
that is handed a uint32_t.
HostObject gains getValueAtIndex/setValueAtIndex plus a hasIndexedAccess() flag.
The defaults stringify and call the named form, so a host object that does not
override them behaves exactly as it did; ObjectManager's array proxy overrides
them and opts in when it has a JNI array signature.
Per engine, because the engines differ in what they can hand over:
V8 has a dedicated indexed interceptor, so it always routes there --
previously it built PropNameID(std::to_string(index)) for the sole
purpose of having ObjectManager parse it again. The named handler
stays kNonMasking and the indexed handler stays masking (kNone);
cea83323 measured 20-30% for getting that backwards.
QuickJS has no indexed hook, but interns a canonical index as a tagged-int
atom, so the index is a mask away where JS_AtomToCString allocates.
JS_ATOM_TAG_INT is engine-internal (quickjs.c, not quickjs.h) though
identical in bellard QuickJS and quickjs-ng, so each runtime verifies
the encoding once through the public API before relying on it and
falls back to the string path if it ever fails.
JSC has no indexed hook either and delivers "0" as a JSStringRef, whose
UTF-16 buffer can be parsed in place -- no allocation, where
stringToUtf8 built (and over-allocated) a std::string.
Hermes is real facebook::jsi, which has no indexed hook and no way to read
a PropNameID without building a std::string, so nothing calls the new
methods there. They exist so the same ObjectManager compiles and
behaves identically, and Hermes keeps the named path.
QuickJS and JSC consult hasIndexedAccess() before routing, V8 does not need to:
on those two the named path also carries the non-masking prototype emulation,
which a host object that is not an indexed collection still needs, and skipping
it for every numeric name would be a behaviour change. On V8 the default
reproduces the old call exactly, so there is nothing to gate.
Also stops the V8 indexed setter allocating: it handed setValueAtIndex an owned
Value, which is a shared ValueStorage plus a v8::Global created and destroyed on
every element write. The value does not outlive the call, and the default
setValueAtIndex promotes it before reaching the named setter, so a host object
that does not override the indexed form still gets an owned value. QuickJS and
JSC already borrowed here.
Measured on QV7120NC26 (Sony SO-01M), release, 34-entry marshalling table,
napi and jsi interleaved in one session, two runs each, medians. Controls
(Void Static Method, multiply, Return an Int) held within 10% end to end, so no
throttled regime. Against the same jsi runtime at 02bad159:
V8-13, ms before after napi jsi vs napi before -> after
Int Array[0] write 599.6 392.2 375.9 0.63x -> 0.96x
Double ... 602.0 391.8 377.2 0.63x -> 0.96x
Boolean ... 602.5 369.0 384.5 0.64x -> 1.04x
String ... 1498.4 1239.2 1111.1 0.74x -> 0.90x
TOTAL 13721.8 12307.9 12777.2 1.07x -> 0.96x
So the V8-13 jsi runtime is now 1.04x FASTER than the napi runtime overall,
where it was 1.07x slower, and the write family is at parity.
The causal story is not the one this change started from, and the decomposition
is worth recording. Measuring the index change alone (no borrowed value) on
V8-13 gives Int Array[0] write 563 ms against 581 baseline -- i.e. nothing. On
V8 the whole win is the borrowed value; std::to_string plus a ten-instruction
parse is noise next to a global-handle create/destroy. The index change earns
its place on the engines that genuinely build a string, which V8 never did:
QUICKJS, debug, same device, interleaved, two runs each, medians
(debug because the QuickJS *release* jsi build does not launch -- see below --
and -O0 inflates C++-side costs, so read the direction, not the magnitude)
Int Array[0] read 1544.7 -> 1041.2 1.48x
Double ... 1546.1 -> 1051.2 1.47x
Boolean ... 1543.7 -> 1037.5 1.49x
String ... 2794.6 -> 2389.8 1.17x
Int Array[0] write 1452.5 -> 980.5 1.48x
Double ... 1432.7 -> 1006.0 1.42x
Boolean ... 1441.3 -> 991.3 1.45x
String ... 3515.0 -> 2964.4 1.19x
TOTAL 42163.0 -> 38712.1 1.09x
Every other entry on both engines is flat within run-to-run noise; the full
34-entry table was taken each time, and nothing regressed.
Not measured: JSC and Hermes performance, V8-10, V8-11, PrimJS. JSC should
behave like QuickJS (it also built a std::string per access) but that is a
prediction, not a measurement.
Verified, QV7120NC26, debug, jasmine suite, failing specs compared BY NAME
against the same engine built from 02bad159 in a worktree:
V8-13 516/1 == baseline 516/1
QUICKJS 516/1 == baseline 516/1 (both also produced 516/2 on a rerun,
adding the same flaky `triggers
interval`; it flips on either side)
QUICKJS_NG 516/1 == baseline 516/1
JSC 516/2 == baseline 516/2
HERMES 516/2 == baseline 516/2
The failure common to every engine is a jasmine timeout in
test_if_global_reference_leaks_when_interface_implementation_is_created, which
is one of the 100k-object leak specs mainpage.js warns cannot complete on a
physical device. It fails identically at the parent commit. JSC's `frees up
resources after complete` and Hermes' `Should throw exception when not invoked
as constructor` are likewise present at the parent commit.
iOS, since NativeScript/jsi is shared: V8 713/0, QuickJS 713/0, JSC 713/1
(SpecialCaseProperty_When_CustomSelector_ImplementedInJS, the known baseline).
No new reports in ~/Library/Logs/DiagnosticReports against a pre-run marker.
Apple Hermes was not run -- it hangs the harness, pre-existing.
Unrelated pre-existing bug found while benchmarking, NOT from this change and
not fixed here: a QUICKJS *release* build of the jsi runtime dies at startup
with `NativeScriptException: JavaScript object for Java ID 0 not found`. The
same build from 02bad159 dies identically, and the napi release build of the
same commit runs fine, so it is a jsi-runtime bytecode-path bug that predates
this work.
(cherry picked from commit 1f6ea3c7743d553b5a31446eb2f19877acbb1ca8)
A release build of the jsi runtime on QuickJS died at startup with "NativeScriptException: JavaScript object for Java ID 0 not found", which looked like an object-identity or lifetime bug and is not one. Release builds compile every app module to the active engine's bytecode (tools/bytecode-compiler), so assets/app/MyApp.js in the APK starts with the container magic NSBCQJS rather than JavaScript. The napi runtime tries js_run_bytecode_file first and only falls back to source; the jsi runtime had no bytecode entry point at all and handed the binary blob to the compiler as source. Because a module is compiled *wrapped*, the blob became the body of a function that was defined and never usefully run, so nothing threw: MyApp.js simply never registered the application object, and the first callJSMethodNative for Java ID 0 was the first visible symptom -- several frames and one process away from the cause. The fix gives EngineHost the same two-step the napi tree has: ExecuteBytecodeFile peeks the file's first 8 bytes, and runs it as bytecode only if the magic is this engine's. QuickJS goes through JS_ReadObject/JS_EvalFunction; Hermes hands the raw HBC to evaluateJavaScript, which detects it and skips the parser. V8 and JSC have no compile-time bytecode format -- they cache compiled code at runtime instead -- so they return false and always compile source, which is what their release builds ship anyway. Past the magic check the file *is* bytecode, so a read or eval failure throws instead of falling back. Falling back would compile the binary as source and report the error somewhere unrelated, which is exactly the failure mode above. This lives in the Android EngineHost, not in NativeScript/jsi/, because the container is an artefact of the Android build toolchain and not part of the engine abstraction shared with Apple. No shared file is touched. Verified on QV7120NC26 (Sony SO-01M), release builds, interleaved: QUICKJS jsi 516 specs, 1 failure (86.1s) QUICKJS napi 516 specs, 1 failure (85.8s) <- control, same spec HERMES jsi 516 specs, 2 failures (80.4s) The shared failure is test_if_global_reference_leaks_when_interface_ implementation_is_created, which churns 100k Java objects synchronously and times out on this device under both binding layers. Hermes' second failure is "Should throw exception when not invoked as constructor", the known Hermes limitation that it cannot distinguish new f() from f(). Before this commit both QUICKJS and HERMES release jsi builds could not start at all. (cherry picked from commit c28fab0471ff0087c89373c9fda0332c6dd24cb1)
FieldAccessor::SetJavaField tested its argument with an inverted
condition on the byte and short paths:
jbyte intValue = !is_of_type(value, number) ? get_int32(value) : 0;
so `obj.byteField = 42` stored 0, and only a *non*-number was read as an
int32 -- get_int32 on a non-number is itself meaningless. The int, long,
float and double cases next to them all test the condition the right way
round, which is what the byte and short cases were plainly meant to do.
The bug is in the napi runtime, and the jsi runtime reproduced it
deliberately (with a comment) because the napi tree is the behavioural
oracle. It is fixed in both here, in one commit, so the two trees stay
diffable and neither becomes the odd one out.
Nothing covered these paths, which is why it survived. DummyClass gains
byte/short instance and static fields and testFieldGetSet.js gains four
specs. They are not decorative: built with the inverted condition
restored, all four fail with "Expected 0 to be 42" / "Expected 0 to be
1234" / "Expected 0 to be -7" / "Expected 0 to be -4321", and pass once
the condition is corrected.
Verified on QV7120NC26 (Sony SO-01M), V8-13 debug, interleaved:
jsi inverted 520 specs, 6 failures (the 4 new + 2 device timeouts)
jsi fixed 520 specs, 2 failures
napi fixed 520 specs, 1 failure
The remaining failures are the JNI-reference-leak specs, which churn
10k-100k Java objects synchronously and time out on this device
regardless of binding layer; test_if_callback_parameter_marshalling_leaks
is GC-timing sensitive and appears intermittently.
(cherry picked from commit 27eda36e1f09d65f576d1a1b801a49ffb56c54b6)
(cherry picked from commit 38dc5f5ab9b469c087ee2f9930da1b64380b9953)
(cherry picked from commit f2833cd339e20393a1f0a2b350594e3fe472b7e5)
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Android currently reaches every JavaScript engine through Node-API. That works,
but it means every call crosses a C ABI designed for portability rather than for
this runtime, and it caps how much the runtime can exploit what each engine can
actually do.
This adds a second binding layer that talks to the engine wrappers directly,
with no Node-API anywhere. The Android runtime is split into two trees that sit
behind a build flag: the existing Node-API one, still the default, and the new
one. Both build on the same shared engine layer, so a backend fix reaches both,
and either can be selected without touching application code.
The point of keeping both is that they can be compared. The new layer is held to
the same suite as the old one, engine by engine, which is what makes it possible
to say it has reached parity rather than merely that it runs.
Alongside the port, this carries engine-layer work that both platforms get:
faster string and host-object paths, a set of per-engine correctness fixes, and
ahead-of-time bytecode module loading.
What changed
The new binding layer
Reaching parity, engine by engine
Shared engine layer
Engine behaviour
gc()is advisory by design, so the specs that observe reclamation account for that rather than forcing a collection through a debug-only entry pointVerified
Known gaps
Worker(...)call that omitsnew. The other engines do. The guard tells a construct call from a plain one by inspecting the receiver, because the engine layer exposes nonew.target, and the Hermes backend synthesises a receiver for plain calls, which erases the distinction. Hermes on the Node-API layer is unaffected.Notes for reviewers
gc()a real synchronous collection. That is deliberately undone here, so its message describes behaviour the branch no longer has.