PDFBOX-6251 usecmap cidrange precedence - #515
Open
pcorless wants to merge 2 commits into
Open
Conversation
A CMap that begins with usecmap is meant to override the base CMap, but both the cidchar/cidrange mappings were merged into the same collections as the importing CMap's own. The range list is consulted first-match, so an inherited cidrange that covers a code beat the cidchar the importing CMap declared for it, and the override was silently lost. ETenms-B5-H and UniJIS-UCS2-HW-H are affected among the bundled predefined CMaps -- ETenms-B5-H maps 0x20-0x7e to the proportional Latin cids 1-95, but ETen-B5-H's inherited range won, so 0x41 resolved to cid 13681 (the fullwidth 'A') instead of 34. Together with their vertical variants, 32 of the 100 bundled CMaps carried lost overrides. Inherited mappings now live in their own map and range list, consulted only after the CMap's own map and ranges. Chained usecmap keeps nearest- wins ordering: the map merge is last-wins so the nearer CMap is applied after the deeper one, and the range list is first-wins so the nearer CMap's ranges are added ahead. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The first commit's two tests left the lookup order only partly guarded. Identity-V is the one predefined CMap that declares no cid mappings of its own, so every lookup it answers now goes through the inherited collections, and it is the only reason hasCIDMappings has to consider them at all. It had no test: testIdentity only covers Identity-H. ETenms-B5-V does declare mappings of its own, six cidchars and twelve cidranges for the punctuation whose vertical form differs, so the chain test's javadoc saying it declares none was wrong. Those mappings also cover the case of a CMap's own mappings beating the ranges it inherits from two levels up. The remaining two cases have no predefined CMap that exercises them, so they are built by hand: a cidrange of the CMap's own beating an inherited cidchar, and the fact that what a CMap inherits is copied rather than shared, which matters because predefined CMaps are cached and handed out repeatedly. Also assert the override through the byte[] overload of toCID, which repeats the lookup order of the int one and was only checked for a pass through. Five of the six usecmap tests now fail without the CMap change; the Identity-V one passes either way and is a regression guard for the new path. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PDFBOX-6251: a CMap's own cid mappings lose to the ones it inherits via usecmap
A CMap that starts with
usecmapmay redefine any code it inherits. It currently can't:useCmapmerges the parent'scidchar/cidrangemappings into the same collections as theimporting CMap's own, and since
usecmapis read from the header, the parent's land incodeToCid/codeToCidRangesfirst.toCIDscans the range list first-match, so an inheritedrange that covers a code beats the mapping the CMap declared for it.
ETenms-B5-Hexists only to do this override: it usesETen-B5-H, then remaps 0x20-0x7E to theproportional latin CIDs 1-95 where the parent has the fullwidth forms at 13648+. Its parsed ranges
hold both, inherited first:
Scope: 31 of the 92 bundled CMaps declare mappings of their own on top of a
usecmap, and all31 are affected. Two are horizontal (
ETenms-B5-H,UniJIS-UCS2-HW-H); the other 29 are-Vvariants whose own mappings select the vertical forms, so vertical CJK gets horizontal glyphs.
Symptom: the wrong glyph, and usually the wrong width with it —
/Wis indexed by CID, so awrong CID falls outside the array and the advance drops to
/DW= 1000. Latin text renders asJ a v a S e r v e r.Fix
Inherited mappings move to
inheritedCodeToCid/inheritedCodeToCidRanges, consulted only afterthe CMap's own: own map -> own ranges -> inherited map -> inherited ranges.
A
usecmapchain stays nearest-wins, which needs opposite insertion orders: the maps are keyed, soputAllis last-wins (deepest first, nearer on top); the range list is first-match, so the nearerCMap's ranges go in ahead.
Three things fell out of it:
hasCIDMappingshas to count inherited mappings.Identity-Vdeclares none of its own, soonce the inherited ones move it would report false and
toCIDwould return 0 for every code.codeToCid.putIfAbsent(length, mappings)stored the parent's inner map by reference when theimporting CMap had no mapping of that length yet, so a later
addCIDMappingwrote into thecached predefined parent and corrupted it for everyone after.
range lists would send a CMap's own
cidrangeto CID 0 through to the inherited mappings. Itnow returns -1 for "no range covers this"; public
toCIDstill returns 0.Tests
Seven added to
TestCMapParser,mvn -pl fontbox -am testgreen (214):testUseCmapOwnMappingsWintoCIDoverloadstestUseCmapChainKeepsNearestMappingtestUseCmapOwnMappingsBeatInheritedRangestestUseCmapOnlyInheritedMappingsIdentity-V, everything inheritedtestUseCmapDoesNotShareMappingsWithTheUsedCMaptestUseCmapOwnRangeBeatsInheritedChartestUseCmapOwnMappingToCidZeroIsNotAFallthrough