asciidoc
grade
reliability
overall
AsciiDoc is a text document format parsed by the asciidoctor compiler to generate formatted HTML output. The tool takes a source file containing lightweight markup and renders the structure, tables, and media links into a single static file. It operates as a deterministic, command-line compiler with minimal options and no persistent background dependencies.
Surfaces × lenses
| surface ↓ lens → | Coherence | Economy | Consistency | Verifiability | Safety |
|---|---|---|---|---|---|
| Disclosure | |||||
| Interface | |||||
| Loop | |||||
| Recursion | |||||
| Human |
Click any cell for what was measured and why. A corner dot marks a cell filled by a hard instrument (checker, telemetry); undotted cells are judged by the rater ensemble. 53% of this matrix is hard-measured.
The experience
An agent writes document content directly into a text file ending with the .adoc extension. The command-line compilation to HTML works well because running a single execution of asciidoctor filename.adoc produces the entire document structure with no state side-effects or configuration overhead. The compiler returns concrete exit codes and clear logging messages, which allows the agent to evaluate the rendering state predictably.
The edit-compile-verify loop works adequately. Because the compiler itself has no file-writing or editing features, the agent must run separate text-editing or shell tools to correct syntax mistakes. However, when errors do occur, the compiler outputs the exact issue, resolving the loop cleanly without leaving orphan files or corrupted state.
The markup syntax works well for layout assembly. Structural markers such as double equals for headings, pipe sequences for data tables, and square brackets for hyperlinks map to standard HTML elements. The compiler handles the nested layout levels and cross-references without requiring complex scripting. State management and safety work well because compiling has no destructive system operations, enabling complete rollback of files at any step.
Fit
The tool is a strong fit for agents generating complex text layouts, technical reports, manuals, or standard data sheets. It serves developers who need deterministic, lightweight template compilation with predictable syntax translation.
The tool is a poor fit for applications requiring real-time page updates, interactive user input handling, or dynamic client-side animations. It compiles static files in a batch process and cannot manage live browser interactions.
Notes
This diagram illustrates the agent's iterative verification loop while building and compiling an AsciiDoc report.
Validation
Convergent check. Where a hard instrument and the independent rater panel measure the same cell, they agree on 7 of 16 cross-checked cells (75%). Two independent methods landing on the same number is our accuracy signal — not just the raters agreeing with each other.
| Cell where they disagree | Instrument | Rater | Δ |
|---|---|---|---|
| disclosure.verifiability | -0.40 | +0.80 | 1.20 |
| recursion.coherence | -0.41 | +0.50 | 0.91 |
| disclosure.economy | -0.34 | +0.50 | 0.84 |
| human.safety | -0.10 | +0.70 | 0.80 |
| recursion.verifiability | +0.00 | +0.70 | 0.70 |
| human.verifiability | +0.10 | +0.80 | 0.70 |
| human.coherence | -0.20 | +0.50 | 0.70 |
| interface.coherence | +0.18 | +0.70 | 0.52 |
| recursion.economy | +0.96 | +0.50 | 0.46 |
Ground truth
7 of 7 requirements met · deterministic checker, no model judgment- renders/compiles cleanly — ok
- has a title — —
- nested sections (2+ levels) — —
- a table — yes
- a code block — —
- a link — —
- an image/figure — yes
Reliability (pass^k): one attempt succeeds 100% of the time · three in a row 100% · all runs 100%. A fresh agent resuming from the files alone never broke prior work.
Models & runs
2026-07-07| Model | Harness | Runs | Solved | Turns |
|---|---|---|---|---|
| | 23 | 19/23 | 6 |
Each path is one run through the shared semantic space; nodes are turns (red = an errored turn). Hover a model — here or in the graph — to isolate its runs.
How this was measured
- reference agent
- pi:z-ai/glm-5.2
- trials
- 25 (n=5/tier)
- methodology
- v0.1 3e74a3c6
- cost
- $0.7334
- hard-measured cells
- 53%
- teaching source
- asciidoc (official docs) (official-docs)
Same brief, same reference agent, and the same trial count for every tool in a category — a difference in the numbers is attributable to the tool (and its teaching source), not the setup. Profiles carry the methodology hash; results under different hashes are not compared.