Repository navigation
Conversation
…stack
Parsing recurses once per nested object or array, so input such as
`repeat("[", 100_000)` (100 KB, far below typical request-size limits)
threw StackOverflowError — "program state may be corrupted" — from every
entry point: untyped, typed (StructUtils make), lazy materialization,
parse!, isvalidjson and skipping of unknown members. A server that parses
untrusted bodies had no clean way to reject it.
Every parse path enters a container through `applyobject`, `applyarray`
or the tuple `maketuple`, so each now checks a nesting depth carried on the
LazyValue and throws
ArgumentError: JSON nesting depth at byte position 513 exceeds the
`maxdepth` limit of 512; pass a larger `maxdepth` keyword argument to
parse deeper input
before recursing. `maxdepth` is a new lazy/parse keyword (default 512),
passed like `allownan` or `jsonlines`. On master the shallowest path
(a recursive struct) overflowed at ~2,000 levels, so 512 keeps ~4x
headroom while allowing any realistic document.
The depth and limit are Int32 fields placed in existing struct padding,
so LazyValue (80 bytes), LazyOptions (48), LazyObject/LazyArray (72) keep
their sizes and allocations are byte-for-byte unchanged.
Measured against master on 56 read cases (instructions retired per call,
which machine load does not distort): +0.4% to +2% on ordinary documents,
up to +5-6% on inputs made only of empty containers (the check is five
instructions per object/array entered). Cycle counts and wall-clock time
showed no difference outside measurement noise. The same 42 methods
compile at first call; the package image grows 0.3%.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #498 +/- ##
==========================================
+ Coverage 92.57% 92.60% +0.03%
==========================================
Files 7 7
Lines 1927 1935 +8
==========================================
+ Hits 1784 1792 +8
Misses 143 143 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
On 32-bit Julia `DepthBox(1)` holds an Int32 while JSON parses the number as Int64, so comparing the two structs failed on the x86 CI job. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Parsing recurses once per nested object or array. Input such as
repeat("[", 100_000)(100 KB, well under typical request-size limits) throwsStackOverflowError("program state may be corrupted") from every entry point: untypedparse, typedparse(json, T),lazy(...)[],parse!,isvalidjson, and skipping unknown members. A server parsing untrusted bodies has no clean way to reject it.Change
Every parse path enters a container through
applyobject,applyarray, or the tuplemaketuple. Each now checks a nesting depth carried on theLazyValueand, before recursing, throws:maxdepthkeyword onlazy/parse/parse!/isvalidjson(default 512), passed likeallownanorjsonlines.error_context=true, the error arrives as theParseErrorcause, like other errors.Performance
Int32fields in existing struct padding.LazyValue(80 bytes),LazyOptions(48), andLazyObject/LazyArray(72) keep their sizes.benchmarks/cases plus larger realistic and container-only stress inputs).Tests
nesting deeper than maxdepthtestset. It covers 14 entry points/targets, the 512/513 boundary, raising and loweringmaxdepth, anderror_context.maxdepthkeyword under--trim.🤖 Generated with Claude Code