feat: link references to symbols documented elsewhere (input tagfiles) - #1267
feat: link references to symbols documented elsewhere (input tagfiles)#1267gennaroprota wants to merge 1 commit into
Conversation
MrDocs can now read external tagfiles and, thus, have a doc comment that references a symbol not extracted by MrDocs itself. A new option, `input-tagfiles`, specifies the external tagfile and maps each one to the base URL its documentation set is published under. A reference that resolves to nothing in the corpus takes its URL from there instead of rendering as plain code. The name is resolved from the scope it is written in outward, as any other reference is, and is never shortened to reach an entry. Every generator takes the URL from one place. Tagfiles are read with a small ad-hoc parser, rather than by LibXml2: the format is a small, machine-generated subset of XML, and linking LibXml2 into mrdocs-core would make it a dependency of every build and of every consumer of the installed package. A path that cannot be read, or a file that is not a tagfile, fails the build. The option MrDocs already had for writing a tagfile (named `tagfile`) becomes `output-tagfile`, so that the pair of options says which direction each of them goes; `tagfile` remains as a deprecated alias. Note that all deprecated options warn now, including `report`, which didn't before.
✨ Highlights
🧾 Changes by Scope
🔝 Top Files
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #1267 +/- ##
========================================
Coverage 83.12% 83.12%
========================================
Files 35 35
Lines 3662 3662
Branches 844 844
========================================
Hits 3044 3044
Misses 410 410
Partials 208 208
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
An automated preview of the documentation is available at https://1267.mrdocs.prtest2.cppalliance.org/index.html If more commits are pushed to the pull request, the docs will rebuild at the same URL. 2026-08-28 17:48:24 UTC |
alandefreitas
left a comment
alandefreitas
left a comment
There was a problem hiding this comment.
This feature is amazing. This is a feature I've wanted for a while, and the layout is right. Resolution happens in the finalizer, which is good, and the reference node has the href.
Is this Doxygen tagfile format good enough? Do we need our own format?
We should load the tagfiles before extraction so that a typo in a tagfile path fails the build before extracting the whole AST, which could take one hour for a large library such as LLVM.
The tagfile to output-tagfile transfer depends on declaration order. This can be solved by doing it once in Config::normalize. Then the special case in normalizeString and the comment can go.
| TIP: A few projects use the tag file in a less obvious way: they read it from their own site-generation scripts to build a navigation tree or a search index from the symbol list, rather than maintaining one by hand. | ||
|
|
||
| NOTE: The inverse direction (consuming tag files emitted by other projects so that your pages link to *their* symbols) is handled by the xref:extensions/antora.adoc#antora-cpp-tagfiles-extension[`antora-cpp-tagfiles-extension`]. Point it at one or more tag files in your Antora playbook, and references like `boost::asio::io_context` resolve to the Boost.Asio site automatically. | ||
| xref:configuration/reference.adoc#input-tagfiles_option[`input-tagfiles`] is the other direction: it reads the tag files of documentation sets you do not build, so a reference to a symbol MrDocs never extracted still becomes a link. Each key is a tag file, each value the base URL that set is published under: |
There was a problem hiding this comment.
Is input-tagfiles that go in the other direction really an "output" option?
| include::example$configuration/input-tagfiles.yml[] | ||
| ---- | ||
|
|
||
| With that in place, `@ref std::vector` in a doc comment links to Cppreference instead of rendering as plain code. A name one of these sets documents resolves the way every other reference does, from the scope it is written in outward, so a name relative to its enclosing scope reaches an external symbol just as it would reach one of yours. What never happens is shortening: `vector` reaches `std::vector` only from inside `std`, because the shorter the name the likelier it is to collide with something unrelated in a foreign set, and a reference that silently points at the wrong page is worse than one that points nowhere. A name no scope accounts for is left as code. |
There was a problem hiding this comment.
cppreference or C++ Reference with link?
| include::example$configuration/input-tagfiles.yml[] | ||
| ---- | ||
|
|
||
| With that in place, `@ref std::vector` in a doc comment links to Cppreference instead of rendering as plain code. A name one of these sets documents resolves the way every other reference does, from the scope it is written in outward, so a name relative to its enclosing scope reaches an external symbol just as it would reach one of yours. What never happens is shortening: `vector` reaches `std::vector` only from inside `std`, because the shorter the name the likelier it is to collide with something unrelated in a foreign set, and a reference that silently points at the wrong page is worse than one that points nowhere. A name no scope accounts for is left as code. |
There was a problem hiding this comment.
This has to be followed by output example, like all other options explained in this section. Without an example in form of narrative, it becomes redundant with the reference section.
| include::example$configuration/input-tagfiles.yml[] | ||
| ---- | ||
|
|
||
| With that in place, `@ref std::vector` in a doc comment links to Cppreference instead of rendering as plain code. A name one of these sets documents resolves the way every other reference does, from the scope it is written in outward, so a name relative to its enclosing scope reaches an external symbol just as it would reach one of yours. What never happens is shortening: `vector` reaches `std::vector` only from inside `std`, because the shorter the name the likelier it is to collide with something unrelated in a foreign set, and a reference that silently points at the wrong page is worse than one that points nowhere. A name no scope accounts for is left as code. |
There was a problem hiding this comment.
What never happens is shortening
These negatives are very AI-ish. I'm sure this didn't cross the reader's mind at this point. There's no reason to negate this or provide any details about edge cases at this point because we haven't even provided the example of the general case yet.
|
|
||
| NOTE: A tag file only says where the pages of that set are, so MrDocs can resolve no more than the set publishes. Cppreference, for one, gives `namespace std` itself no page, so `@ref std::chrono` stays plain text while `@ref std::chrono::duration` links. Either way, it is a reference that resolved to nothing, and xref:configuration/reference.adoc#warn-broken-ref_option[`warn-broken-ref`] reports it as one. | ||
|
|
||
| NOTE: For an Antora site there is also the xref:extensions/antora.adoc#antora-cpp-tagfiles-extension[`antora-cpp-tagfiles-extension`], which adds a `cpp:` macro and bundles the Cppreference tag file. The two are complementary, and reach different text: the extension resolves the names you write in your own pages, `input-tagfiles` resolves the ones inside doc comments, for every generator rather than for Asciidoctor alone. |
There was a problem hiding this comment.
Out of scope (even as a note)
| @param name The name of the symbol to look up. | ||
| @return The URL, or nothing if no tagfile documents the name. | ||
| */ | ||
| std::optional<std::string> |
There was a problem hiding this comment.
Shouldn't this use Expected to match the convention for lookup?
| { | ||
| std::optional<std::string> result; | ||
| std::string_view const scopeQualifier = "::"; | ||
| if (name.starts_with("scopeQualifier")) |
There was a problem hiding this comment.
comparing with a literal string instead of the variable? this works?
| // Only warn once per reference | ||
| !refWarned_.contains({ref.literal, ctx.Name}) && | ||
| // Ignore std:: references | ||
| !ref.literal.starts_with("std::")) |
There was a problem hiding this comment.
This is intentional and makes the comment about warn-broken-ref in the documentation incorrect. The reason we skip this is that users tend to know what they're doing in this case and they typically just want code formatting here. Doxygen libraries come with this a lot where people @ref they're own library and std freely just for the formatting.
| option's properties, and surfaced by | ||
| @ref reportDeprecatedOptions. | ||
| */ | ||
| struct DeprecatedOption |
There was a problem hiding this comment.
This is not our first deprecated option. What was the model we had before, and why was it insufficient?
| @@ -0,0 +1,29 @@ | |||
| <?xml version="1.0" encoding="UTF-8" standalone="yes"?> | |||
There was a problem hiding this comment.
Is this generated by doxygen or us? We need to test both cases to be sure.
|
As a meta-comment, this seems to have a much higher ROI than plugins right now because the design is much clearer. |
3 participants
MrDocs can now read external tagfiles and, thus, have a doc comment that references a symbol not extracted by MrDocs itself. A new option,
input-tagfiles, specifies the external tagfile and maps each one to the base URL its documentation set is published under. A reference that resolves to nothing in the corpus takes its URL from there instead of rendering as plain code. The name is resolved from the scope it is written in outward, as any other reference is, and is never shortened to reach an entry. Every generator takes the URL from one place.Tagfiles are read with a small ad-hoc parser, rather than by LibXml2: the format is a small, machine-generated subset of XML, and linking LibXml2 into mrdocs-core would make it a dependency of every build and of every consumer of the installed package.
A path that cannot be read, or a file that is not a tagfile, fails the build.
The option MrDocs already had for writing a tagfile (named
tagfile) becomesoutput-tagfile, so that the pair of options says which direction each of them goes;tagfileremains as a deprecated alias. Note that all deprecated options warn now, includingreport, which didn't before.Changes
input-tagfilesoption and loading it into the corpus;Corpus::externalUrlbesideCorpus::lookup;hrefonReferenceInlineand the branch that renders it; thetagfile->output-tagfilerename with a deprecated alias; generic, deferred reporting of deprecated options.clang.mrdocs.TagfileIndex,clang.mrdocs.TagfileReader).config/input-tagfilesfixture (xml, adoc, html) covering a fully qualified name, a name relative to its enclosing scope, a namespace page, a member with an anchor, a member on its own page, and names that resolve to nothing; theconfig/tagfilefixture is renamed toconfig/output-tagfileand regenerated for its new file names.ConfigOptions.json, so the generatedconfig schema and the published option reference change; the new
hreffield regenerates the reflected DOM schema (mrdocs.rng,generators/mrdocs.schema.json,dom-schema.adoc).--tagfilenow pass pass--output-tagfile.tagfilestill works but warns. The long-deprecatedreportoption starts warning too. Underwarn-as-error, either fails the build.Testing
The unit suites test the index and the reader, including the inputs it must reject; the golden fixture tests resolution end to end in XML, AsciiDoc and HTML.
Beyond that, the reader was validated against the real Cppreference tagfile, and the possibility to repeat
--input-tagfiles=<path>=<base-url>, needed by the Antora extension, was checked by hand.Documentation
The "External docs" section of configuration/output.adoc documents the new option with an example. commands/inlines.adoc cross-references it from
@ref, and the option reference is generated fromConfigOptions.json. No new page is added.