Back to explorer
markup Declarative files best source: official docs

asciidoc

A

grade

100%

reliability

+0.43

overall

AsciiDoc is a text document format parsed by the asciidoctor compiler to generate formatted HTML output. The tool takes a source file containing lightweight markup and renders the structure, tables, and media links into a single static file. It operates as a deterministic, command-line compiler with minimal options and no persistent background dependencies.

Surfaces × lenses

hostile friendly
surface ↓
lens →
Coherence Economy Consistency Verifiability Safety
Disclosure
Interface
Loop
Recursion
Human

Click any cell for what was measured and why. A corner dot marks a cell filled by a hard instrument (checker, telemetry); undotted cells are judged by the rater ensemble. 53% of this matrix is hard-measured.

The experience

An agent writes document content directly into a text file ending with the .adoc extension. The command-line compilation to HTML works well because running a single execution of asciidoctor filename.adoc produces the entire document structure with no state side-effects or configuration overhead. The compiler returns concrete exit codes and clear logging messages, which allows the agent to evaluate the rendering state predictably.

The edit-compile-verify loop works adequately. Because the compiler itself has no file-writing or editing features, the agent must run separate text-editing or shell tools to correct syntax mistakes. However, when errors do occur, the compiler outputs the exact issue, resolving the loop cleanly without leaving orphan files or corrupted state.

The markup syntax works well for layout assembly. Structural markers such as double equals for headings, pipe sequences for data tables, and square brackets for hyperlinks map to standard HTML elements. The compiler handles the nested layout levels and cross-references without requiring complex scripting. State management and safety work well because compiling has no destructive system operations, enabling complete rollback of files at any step.

Fit

The tool is a strong fit for agents generating complex text layouts, technical reports, manuals, or standard data sheets. It serves developers who need deterministic, lightweight template compilation with predictable syntax translation.

The tool is a poor fit for applications requiring real-time page updates, interactive user input handling, or dynamic client-side animations. It compiles static files in a batch process and cannot manage live browser interactions.

Notes

This diagram illustrates the agent's iterative verification loop while building and compiling an AsciiDoc report.

Validation

Convergent check. Where a hard instrument and the independent rater panel measure the same cell, they agree on 7 of 16 cross-checked cells (75%). Two independent methods landing on the same number is our accuracy signal — not just the raters agreeing with each other.

Cell where they disagreeInstrumentRaterΔ
disclosure.verifiability-0.40+0.801.20
recursion.coherence-0.41+0.500.91
disclosure.economy-0.34+0.500.84
human.safety-0.10+0.700.80
recursion.verifiability+0.00+0.700.70
human.verifiability+0.10+0.800.70
human.coherence-0.20+0.500.70
interface.coherence+0.18+0.700.52
recursion.economy+0.96+0.500.46

Ground truth

7 of 7 requirements met · deterministic checker, no model judgment
  • renders/compiles cleanly — ok
  • has a title — —
  • nested sections (2+ levels) — —
  • a table — yes
  • a code block — —
  • a link — —
  • an image/figure — yes

Reliability (pass^k): one attempt succeeds 100% of the time · three in a row 100% · all runs 100%. A fresh agent resuming from the files alone never broke prior work.

Models & runs

2026-07-07
ModelHarnessRunsSolvedTurns
glm-5.2 PI2319/236

Each path is one run through the shared semantic space; nodes are turns (red = an errored turn). Hover a model — here or in the graph — to isolate its runs.

How this was measured

reference agent
pi:z-ai/glm-5.2
trials
25 (n=5/tier)
methodology
v0.1 3e74a3c6
cost
$0.7334
hard-measured cells
53%
teaching source
asciidoc (official docs) (official-docs)

Same brief, same reference agent, and the same trial count for every tool in a category — a difference in the numbers is attributable to the tool (and its teaching source), not the setup. Profiles carry the methodology hash; results under different hashes are not compared.