Skip to content

Add llms-full.txt, AGENTS.md, context7.json and a machine-readable API reference - #1222

Draft
ru551n wants to merge 9 commits into
VUnit:masterfrom
ru551n:feature/agent-friendly-docs
Draft

ru551n wants to merge 9 commits into
VUnit:masterfrom
ru551n:feature/agent-friendly-docs

Conversation

@ru551n

@ru551n ru551n commented Sep 15, 2026

Copy link
Copy Markdown
Contributor

What this PR does

It makes VUnit's documentation and APIs easy for AI coding assistants to use. The docs build generates these files on the website:

  • llms-full.txt: all documentation pages as plain text in one file, including the VHDL package headers that the pages pull in.
  • api/vunit_lib.json and api/python.json: the VHDL and Python APIs in machine-readable form. The VHDL part covers subprograms with their parameter modes, types and defaults, and aliases, types, constants and doc comments for all 70 packages. It is extracted from syntax trees dumped by vhdl-dump-ast, from the VHDL-LS project. The docs job installs version 0.1.0 with cargo install.

Two files are added to the repository:

  • AGENTS.md: the commands and rules a coding agent needs when contributing: tests, lint, news fragments, the license header and generated files.
  • context7.json: configuration for Context7 (see below).

Why it is beneficial

AI assistants write run scripts and testbenches from the documentation they can find, and today they miss the VHDL API. The API pages include the package headers with literalinclude, so a tool that reads the .rst sources never sees them. The result is guessed subprogram names and outdated usage.

With these files, an agent can look up exact overloads, parameter modes and defaults instead of guessing. Nothing needs manual upkeep: every docs build regenerates the files, and the build fails if the syntax tree format changes unexpectedly.

What Context7 is

Context7 is a documentation service that AI coding assistants query for up-to-date library documentation. It already indexes this repository as /vunit/vunit. Asked about queue_pkg's new_queue, push and pop, it currently returns only com user guide examples and a 2018 blog post, because the API headers are not in the .rst sources.

This PR improves that in two ways. context7.json leaves out the blog and release notes and gives usage rules, and llms-full.txt can be added as a Context7 source so that the API text is indexed.

After merging, a maintainer needs to:

  1. claim /vunit/vunit on context7.com, which adds a key to context7.json;
  2. submit https://vunit.github.io/llms-full.txt as a source.

Testing

The unit tests and lint pass, and tox -e py313-docs passes locally. The generated VHDL API was compared against the output of an earlier GHDL-based version of the generator.

This PR was prepared with substantial use of an AI assistant (Claude Code), under my direction and review.

🤖 Generated with Claude Code

ru551n and others added 3 commits September 15, 2026 12:53
The documentation is also built with the Sphinx text builder into _text/.
Unlike the reStructuredText sources, the text build contains what the
pages include or generate: the VHDL package headers, the command line
reference and the Python API. llms.txt indexes the text pages and
llms-full.txt concatenates all of them except the optional ones (blog,
release notes, ...), both following the llms.txt convention.

AGENTS.md tells coding agents how to work in the repository, and
context7.json tells Context7 which parts of the repository to index and
how VUnit is used.

The docs job gets GHDL, used by the API reference generator added in the
next commit, and VUNIT_DOCS_SKIP_API=1 builds the documentation without
it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Add tools/build_api_reference.py, which compiles vunit_lib and osvvm with
GHDL, runs "ghdl --file-to-xml" per package file, and converts the result
into api/vunit_lib.json (VHDL symbol tables: types, subtypes, constants,
objects, subprograms, aliases, with doc comments and source-text slices for
types/defaults) and api/python.json (the Python API of every autoclass/
autofunction/automodule target under docs/). VUNIT_DOCS_SKIP_API=1 skips
the build with a warning and exit 0.

Add tests/unit/test_api_reference.py, covering the XML-to-dict conversion,
source-text slicing (including GHDL's tab-stop-8 column counting and its
folded-literal position quirk for attribute defaults), doc-comment
extraction, the VHDL package scope filter, the completeness check, Python
API collection, and every converter error in the design's error-handling
table -- all with inline XML/source fixtures, no GHDL or Sphinx needed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ru551n ru551n closed this Sep 15, 2026
@ru551n ru551n reopened this Sep 15, 2026
ru551n and others added 2 commits September 15, 2026 18:00
* Autodoc targets are resolved with pkgutil.resolve_name, and an autodoc
  directive other than autoclass and automodule, of which the
  documentation has none, is reported instead of being supported
  speculatively.
* The one-line doc comment and source line helpers, the per-package
  extraction helper and the ignored kinds set are inlined.
* Protected type methods get a doc comment like every other subprogram,
  which removes a flag.
* llms_txt.py no longer filters hidden paths that the text build never
  produces.
* Repeated unit tests are combined into table-driven tests.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The documentation already covers the pages, so only the machine-readable API reference is generated for AI agents. The text build and tools/llms_txt.py are removed, tools/build_docs.py is back to its original form, and AGENTS.md, context7.json and the news fragment no longer refer to the llms files.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ru551n ru551n changed the title Add llms.txt, AGENTS.md, context7.json and a machine-readable API reference Add AGENTS.md, context7.json and a machine-readable API reference Sep 15, 2026
tools/build_docs.py also runs the Sphinx text builder into a temporary directory and concatenates the pages into llms-full.txt, the front page first, leaving out the blog, release notes and other pages that do not document how to use VUnit. It is the only published form of the documentation that contains the included VHDL package headers as text, and it can be added as a Context7 source. llms.txt and the separate text pages are not generated.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ru551n ru551n changed the title Add AGENTS.md, context7.json and a machine-readable API reference Add llms-full.txt, AGENTS.md, context7.json and a machine-readable API reference Sep 15, 2026
ru551n and others added 3 commits September 15, 2026 19:03
The VHDL symbol tables are extracted from the syntax trees that vhdl-dump-ast 0.1.0 (VHDL-LS project) dumps per file, including comments. This replaces the GHDL compile, --file-to-xml and the line/column based slicing of the source text: types, defaults and doc comments come directly from the tokens, and the docs build needs no compile. The version is pinned, and a different vhdl-dump-ast version fails the build.

Compared with the GHDL based output, the package set and every source-declared subprogram are the same. The declarations VHDL creates implicitly (predefined operators, minimum/maximum and the aliases of implicit operators) are no longer listed, and parameters sharing a type in one declaration all get the type, which GHDL's XML lost for all but the first.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The generator now unwraps the vhdl-dump-ast tree once, numbering the
lines of the tokens as it goes, and handles all declaration kinds in
one function. The output is unchanged.

The tests parse small VHDL snippets with the real vhdl-dump-ast instead
of building syntax trees by hand. They are skipped when vhdl-dump-ast
is not installed, so the docs tox environment, which has it, now runs
them.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The docs workflow runs tox -e py313-docs -- --color, and pytest rejects --color without a value. The arguments are meant for the Sphinx build.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant