Skip to content

[RFC]: Extend stdlib's doctesting approach to C examples #13194

Description

@officiallyanee

Description

This RFC proposes a suite of packages under @stdlib/_tools/doctest/c implementing an automated extract → instrument → compile → run → compare pipeline for testing C code examples embedded in src/*.c source files and README.md files.


Proposed Changes

Package layout

@stdlib/_tools/doctest/c/           ← orchestrator (bin/cli)
├── extract/ 
├── instrument/ 
├── compile/ 
├── run/ 
└── compare/

Make targets follow the existing doctest namespace in tools/make/lib/doctest/c/doctest-c.mk.


Sub-package API

@stdlib/_tools/doctest/c/extract

extract( filePath, mode )  Array<block>
// filePath: string  — absolute path to a .c source file or README.md
// mode:     'src' | 'markdown'
// returns an array of self-contained block objects, one per testable example
/* block = {
        file: string,   
        blockIndex: number,                                         ← 0-based, helps segregating example blocks from same file
        includes: Array<{ text: string, line: number }>,            ← #include lines hoisted for this block
        staticHelpers: Array<{ text: string, line: number }>,       ← array of hoisted internal static functions called by this block
        fileScope: Array<{ text: string, line: number }>,           ← pre-function declarations + function definitions for this block (to be hoisted outside main()) (ex, @stdlib/ndarray/base/function-object)
        body: Array<{ text: string, line: number }>,                ← procedural lines for main() for this block
        annotations: Array<{ text: string, line: number }>          ← extracted annotation values (e.g., "~1.571", "true") from // returns comments
      }
*/

Parses @example Doxygen blocks (src mode) or ```c ``` fenced blocks (markdown mode). Each block is independent.
Note: Contiguous lines are grouped into a single text block. The line property indicates the starting line number of that contiguous block

@stdlib/_tools/doctest/c/instrument

instrument( block )  { tmpFile, annotations }
// block:    block object from extract()
// tmpFile:  absolute path to the generated .c file written to os.tmpdir()
// annotations: Array<{ annotation, type, varName, file, line }>

Wraps the block into a compilable .c file: injects necessary stdlib and standard library headers, hoists static helpers and file-scope declarations, wraps procedural code in main(), and emits a type-aware printf per // returns annotation. Injects #line N "original/file.c" directives so all compiler errors reference the original file and line, not the generated tmp file.
For each annotation, type inference runs via a inferType helper that backward-scans the body lines accumulated up to that annotation to find the preceding variable declaration.

@stdlib/_tools/doctest/c/compile

compile( tmpFile )  { ok, binary, stderr }
// tmpFile: path from instrument()
// binary:  path to the compiled executable (in os.tmpdir())
// stderr:  raw compiler output (already line-mapped to original file via #line)

Resolves include paths via manifest.json using @stdlib/utils/library-manifest. Handles supplemental headers referenced inside a snippet but absent from the package's core manifest dependencies.

@stdlib/_tools/doctest/c/run

run( binary, opts )  { ok, stdout, stderr, timedOut }
// binary:  path from compile()
// opts:    { timeout: number }  — default 15000 ms

Executes the compiled binary via spawnSync with a configurable timeout.

@stdlib/_tools/doctest/c/compare

compare( stdout, annotations )  Array<{ pass, got, expected, file, line }>
// stdout:      raw stdout string from run()
// annotations: annotation metadata array from instrument()

Normalizes tokens (inf/-inf±Infinity, nanNaN, trailing f stripped) before comparison. Uses Object.is for -0 detection and delegates ~-prefixed approximate values to @stdlib/_tools/doctest/compare-values and direct comparison for ints and bools.


Block Skip Rules

In src/*.c files, a trailing comment on the function definition opts the @example blocks of that function out:

// Skips @example blocks of that function:
static double foo( double x ) { // stdlib-c-doctest disable
    ...
}

// Skip all remaining @example blocks in the file:
// stdlib-c-doctest disable-file

In README.md files, the existing <!-- run-disable --> comment before a fenced block opts it out. napi packages (paths containing /napi/) are excluded automatically by the orchestrator.


CLI

Invoked by make targets and pre-commit hooks, one file at a time:

node lib/node_modules/@stdlib/_tools/doctest/c/bin/cli.js \
    --mode=src \
    --file=lib/node_modules/@stdlib/math/base/special/sin/src/main.c \
    [--verbose] [--timeout=<ms>]
Flag Description
--mode src or markdown
--file File to process
--verbose Print compiler output and instrumentation trace
--timeout Per-binary execution timeout in ms (default: 15000)

Output Format

Compiler errors are surfaced directly from the compiler's own output, which are already mapped to the original file via #line directives. Annotation mismatches and pipeline errors (compare stage) are formatted by the orchestrator as file:line:col: error: message [rule].


Stretch Goals

  • Mutated scalar annotation: // out => 0.5
  • Flat array annotation: // x => <double>[ 1.0, 2.0, 3.0 ]
  • Complex number annotation: // returns <complex128>[re, im]

Related Issues

Related issues #G96 , #1378.


Questions

  1. Include path resolution: Should supplemental dep resolution remain in the doctest tool, or should a docs task config be added to manifest.json so library-manifest handles it fully?

  2. Static helper handling: Should internal static helpers be hoisted exactly into the tmp file, or promoted to namespaced internal symbols (e.g., stdlib_internal_*) via a restricted header?

  3. Tool structuring/naming: Any feedback on the sub-package naming or placement under @stdlib/_tools/doctest/c/?

  4. Skip rule scope for @example blocks: When // stdlib-c-doctest disable is placed on a function definition, it skips all @example blocks for that function. However, since a function can have multiple example blocks, should there be a way to skip specific blocks (e.g., using an index-based approach)?

Other

No.

Checklist

  • I have read and understood the Code of Conduct.
  • Searched for existing issues and pull requests.
  • The issue name begins with RFC:.

Metadata

Metadata

Assignees

No one assigned

    Labels

    AcceptedRFC feature request which has been accepted.CIssue involves or relates to C.RFCRequest for comments. Feature requests and proposed changes.ToolsIssue or pull request related to project tooling.

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions