Repository navigation
[API Proposal]: System.Text.Json Add an Utf8JsonReader.TryGetString(). #135938
Description
Background and motivation
Utf8JsonReader doesn't support non-allocating span string token reading. It's pretty painful for the cases when that string token is used as a dictionary key.
GetString()- just returns a string directly, already garbaged.ValueSpan/ValueSequence- returns unescaped bytes, which then need a lot of code to become normal UTF16 chars.CopyString()- you have to know char count initially, otherwise it will allocate a useless exception, since we just need a fail flag.ValueTextEquals()- useless when you want to use that value as dictionary key, since you have to iterate over whole dictionary.
API Proposal
ref struct Utf8JsonReader {
bool TryGetString(Span<char> data, out int charsTotal);
int GetCharsCount();
}TryGetString() works similar to other TryGet*() methods, however return is true when value is consumed as whole, in partial case it is false. Since value is already in mid-counted state, it should just complete the count to the end and write it to charsTotal. Thus TryGetString() returns true if charsTotal <= data.Length. In case of false caller should just realloc the buffer to the charsTotal size on the next call.
GetCharsCount() just counts string length.
Both methods should support both String, PropertyName and Comment tokens.
API Usage
bool TryGetValue<T>(Dictionary<string, T>.AlternateLookup<ReadOnlySpan<char>> dictionary, ref Utf8JsonReader reader, out T value) {
var expectedSize = 16;
return TryExecute(dictionary, ref reader, ref expectedSize, out value)
|| TryExecute(dictionary, ref reader, ref expectedSize, out value);
bool TryExecute(Dictionary<string, T>.AlternateLookup<ReadOnlySpan<char>> dictionary, ref Utf8JsonReader reader, ref int expectedSize, out T value) {
Span<char> temp = stackalloc char[expectedSize];
if (reader.TryGetString(temp, out expectedSize)) {
value = dictionary[temp[..expectedSize]];
return true;
}
Unsafe.SkipInit(out value);
return false;
}
}Alternative Designs
No response
Risks
No response
Activity
dotnet-policy-service commentedon Oct 16, 2026
Tagging subscribers to this area: @dotnet/area-system-collections
See info in area-owners.md if you want to be subscribed.
MihuBot commentedon Oct 16, 2026
I'm a bot. Here is a possible related and/or duplicate issue (I may be wrong):
eiriktsarpalis commentedon Oct 16, 2026
You don't need the exact character count before calling
Utf8JsonReader.CopyString. The raw token's UTF-8 byte length is an upper bound on the decoded UTF-16 character count, even when the token contains escape sequences.For your dictionary lookup, you can use
CopyString(Span<char>)with a small stack buffer and a pooled buffer for larger tokens:This works with a dictionary configured for span-based alternate lookup, such as one using
StringComparer.Ordinal. It handles escaped strings and multi-segment tokens without materializing a temporary string or using exceptions to discover the required capacity. Pooling reuses arrays when available, although a pool miss can allocate.If your consumer accepts UTF-8,
CopyString(Span<byte>)instead copies the unescaped UTF-8 bytes:The byte example uses a small, contiguous input. For arbitrary inputs, use the same length calculation and bounded stack/pool approach as above, with
bytebuffers.Both overloads support
StringandPropertyName, but notCommentorNull, and neither advances the reader.We previously considered
TryCopyStringin #77714 and declined it because the raw token length provides this inexpensive sizing bound. Closing this proposal on the same basis: the existingCopyStringoverloads cover the dictionary-lookup scenario without requiring an exact character count. Thanks for raising the use case.Drafted with GitHub Copilot.
ShadedBlink commentedon Oct 16, 2026
That's mostly okay for ascii strings, but when it comes to non-ascii chars, it becomes x6 times heavier in worst case. Current default json serializer settings serializes all unicode chars as escape sequences unless encoder is manually set, thus for 256 bytes raw input we may cover no more than 42 escaped chars.
Why not add atleast a
int GetCharsCount()then? I see in that proposal that main concern was in excessive iteration over a string. However, forValueIsEscapedyou have to iterate anyway to scan for escaped chars, why not make the same for char count?eiriktsarpalis commentedon Oct 16, 2026
ValueIsEscapeddoesn't scan the token when accessed. It's a flag recorded whileRead()parses the string and validates its escape syntax, without computing the decoded UTF-16 length.A
GetCharCount()method would require an additional pass to interpret the escapes and count the UTF-16 code units, followed byCopyString()processing the same token again to produce the characters. Computing the count duringRead()would instead impose additional work on callers that never need it.The raw-length bound can overestimate the required capacity by a factor of six for entirely
\uXXXX-escaped input, but that doesn't translate into six times the allocation or execution cost. With a reusable or pooled buffer, the extra capacity can be a reasonable trade-off for avoiding an exact-count pass on every lookup.I'd leave this closed on the same basis as #77714. If you have measurements from a representative workload showing that buffer overprovisioning is a significant bottleneck, particularly compared with counting and then copying, that would be useful evidence for reconsidering.