Skip to content

fix: treat generation-mismatched refs as null and key full-save renumbering by (num, gen) - #107

Open
sreyasvpariyath wants to merge 1 commit into
LibPDF-js:mainfrom
sreyasvpariyath:fix/stale-generation-refs
Open

sreyasvpariyath wants to merge 1 commit into
LibPDF-js:mainfrom
sreyasvpariyath:fix/stale-generation-refs

Conversation

@sreyasvpariyath

Copy link
Copy Markdown

Problem

A reference N G R can point to an object whose xref entry and object header both say N g obj, with g != G. In that case DocumentParser.getObject still returns the live body.

The registry then holds two parsed copies for the same object number, (N,G) and (N,g). writeComplete keys renumberMap by object number only, so both copies are given the same output number.

The result is that the saved file contains two M 0 obj bodies. The xref points at only one of them, so an edit made through the other path is lost on reload. qpdf --check still passes, so the failure is silent.

#92 (dangling refs → null on save) does not cover this case: the stale ref does resolve, just to the wrong object. This PR is complementary to #92 and builds on its "unmapped ref → null" behaviour.

Minimal repro (synthetic)

Object 4 lives at generation 1 (xref 00001, header 4 1 obj). The page reaches it via 4 1 R, while /Dests holds a stale 4 0 R.

import { PDF, PdfName } from "@libpdf/core";

const objs = [
  "1 0 obj\n<< /Type /Catalog /Pages 2 0 R /Dests << /stale [4 0 R] >> >>\nendobj\n",
  "2 0 obj\n<< /Type /Pages /Kids [3 0 R] /Count 1 >>\nendobj\n",
  "3 0 obj\n<< /Type /Page /Parent 2 0 R /MediaBox [0 0 100 100] /Resources << /XObject << /X 4 1 R >> >> /Contents 5 0 R >>\nendobj\n",
  "4 1 obj\n<< /Type /XObject /Subtype /Form /BBox [0 0 100 100] /Length 18 >>\nstream\n0 0 50 50 re f\n   \nendstream\nendobj\n",
  "5 0 obj\n<< /Length 7 >>\nstream\n/X Do\n \nendstream\nendobj\n",
];
let body = "%PDF-1.7\n";
const offs = [];
for (const o of objs) { offs.push(body.length); body += o; }
const gens = [0, 0, 0, 1, 0];
const xref = "xref\n0 6\n0000000000 65535 f \n" +
  offs.map((o, i) => `${String(o).padStart(10, "0")} ${String(gens[i]).padStart(5, "0")} n \n`).join("");
const bytes = new TextEncoder().encode(body + xref + `trailer\n<< /Size 6 /Root 1 0 R >>\nstartxref\n${body.length}\n%%EOF\n`);

const doc = await PDF.load(bytes, { lenient: true });
doc.getCatalog().get("Dests");
const xo = doc.context.resolve(doc.getPage(0).dict.get("Resources").get("XObject").get("X"));
xo.set("Edited", PdfName.of("Yes"));                       // edit through the live 4 1 R path
const out = Buffer.from(await doc.save()).toString("latin1");

const nums = [...out.matchAll(/(?:^|[\r\n])(\d+) (\d+) obj\b/g)].map(m => m[1]);
const dups = [...new Set(nums.filter((n, i) => nums.indexOf(n) !== i))];
const re = await PDF.load(new Uint8Array(Buffer.from(out, "latin1")));
const xo2 = re.context.resolve(re.getPage(0).dict.get("Resources").get("XObject").get("X"));
console.log({ duplicateObjectNumbers: dups, editSurvivesReload: xo2?.get?.("Edited")?.value === "Yes" });

Observed with Node 20.19.2 (main and main + this PR re-checked with Bun 1.4.2 against src/):

build duplicate object numbers edit survives reload
0.4.1 (npm) ["5"] false
0.4.2 (npm, includes #92) ["5"] false
0.5.3 (npm) ["5"] false
main 37e756a ["5"] false
main 37e756a + this PR [] true

Spec

  • ISO 32000-1 §7.3.10: a reference to an undefined object is not an error; it is treated as a reference to the null object.
  • §7.5.4: the xref entry's generation identifies the object, so a reference with a different generation does not refer to it.
  • §7.5.7: objects in object streams have generation 0.

Fix

  • DocumentParser.getObject (src/parser/document-parser.ts):
    • If both the xref entry generation and the parsed header generation differ from the requested generation, push a warning (Stale reference N G R: object N is at generation g; treating as null) and resolve to null. The result is cached.
    • If only the xref generation is wrong and the header matches, the existing lenient behaviour is kept.
    • For compressed (ObjStm) entries, a ref with generation ≠ 0 resolves to null.
  • writeComplete (src/writer/pdf-writer.ts):

Tests

New file src/writer/stale-generation.test.ts (14 tests):

  • Each of these runs with useXRefStream false and true:
    • a stale ref in /Dests;
    • an xref-stream input;
    • a non-zero-generation ref to an ObjStm object.
  • Stale refs in /Outlines, /OpenAction and /AcroForm.
  • A live object at generation 2.
  • The reverse case: a stale higher-generation ref to a live generation-0 object.
  • Negatives:
    • a generation-1 object with no stale refs is kept;
    • a free xref entry is still null;
    • a broken xref generation with a matching header keeps the lenient path.

Results on main @ 37e756a + this PR (Bun 1.4.2, bun install --frozen-lockfile):

  • New tests without the src/ change: 11 of 14 fail. The 3 negatives pass.

  • New tests with the change: 14 of 14 pass.

  • vitest run src/writer src/parser: 12 files, 481 tests, all pass.

  • bun run typecheck: passes.

  • bun run lint: 0 errors. The 4 warnings are in src/signatures/ and are identical on unmodified main. Formatting check passes.

  • bun run test:run (full suite): 2 failed / 138 passed files; 6 failed / 3079 passed tests.

  • The 6 failures are the same tests on unmodified main:

    • lta-finalization.test.ts ×5;
    • ltv/gatherer.test.ts "caches revocation data" ×1.

    They need OCSP/TSA network access, which times out in my environment (ConnectTimeoutError to the TSA host).

Notes

AI assistance

This change was written with AI assistance, as allowed by CONTRIBUTING.md. Condensed prompts used:

  1. "Saving a re-edited PDF loses an edit on reload and the output has duplicate N 0 obj bodies. Find the root cause in @libpdf/core and fix it properly, not with a workaround in my app."
  2. "Build a minimal synthetic PDF that reproduces it, and confirm which published versions and which main commit are affected."
  3. "Check what the PDF spec says about references whose generation does not match the xref entry, and make the parser and the full-save writer follow it."
  4. "Write regression tests for both classic xref and xref-stream output, the outline/open-action/AcroForm paths, object streams, and negative cases that must keep the current lenient behaviour. Show that they fail without the fix."
  5. "Run the full test suite, typecheck and lint on main with the fix, and compare failures against unmodified main."
  6. "Prepare an upstream PR against main, and note conflicts with open PRs."

Review and testing: to be upfront, the code and tests in this PR were written and run by AI coding agents. I am a Java developer and don't read TypeScript fluently, so I have not reviewed every line of the diff myself. I read a plain-language walkthrough of the parser and writer changes and the test results reported above (the new tests fail without the fix and pass with it). The fix has been running in my own app via a patched build. Please review it with that in mind; I'm happy to make changes, and you are very welcome to rework it or treat it as a detailed bug report instead.

Checklist

  • Branch from main named fix/...
  • Tests added for the change (co-located *.test.ts)
  • New tests fail without the fix and pass with it
  • Full test suite run (6 network-dependent failures, same as on unmodified main)
  • bun run typecheck passes
  • bun run lint passes (0 errors)
  • Single focused commit
  • AI-assistance prompts included above
  • I have reviewed and understand all AI-generated code in this PR (not ticked, see "Review and testing" above)

Found while building my side project NixRey.

@vercel

vercel Bot commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

@sreyasvpariyath is attempting to deploy a commit to the mythie's projects Team on Vercel.

A member of the Team first needs to authorize it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant