dokuwiki
grade
reliability
overall
The DokuWiki markup tool allows an agent to write structured documents using DokuWiki syntax and compile them to HTML using Pandoc. The workflow works adequately because the Pandoc compiler consistently transforms the source markup into HTML with no unexpected rendering variations. However, the syntax requires precise compliance with reverse heading counts and custom table indicators, which can result in verbose editing cycles.
Surfaces × lenses
| surface ↓ lens → | Coherence | Economy | Consistency | Verifiability | Safety |
|---|---|---|---|---|---|
| Disclosure | |||||
| Interface | |||||
| Loop | |||||
| Recursion | |||||
| Human |
Click any cell for what was measured and why. A corner dot marks a cell filled by a hard instrument (checker, telemetry); undotted cells are judged by the rater ensemble. 53% of this matrix is hard-measured.
The experience
The local rendering process using Pandoc works well. When executing the command to convert the DokuWiki file to HTML, the compiler executes deterministically and returns immediate feedback. If the syntax is valid, compilation succeeds with no warnings, allowing the agent to verify code block syntax highlighting, link rendering, and image layout through direct inspection of the output file.
The heading syntax behaves adequately but introduces a minor mechanical challenge. Unlike standard Markdown where fewer symbol characters represent larger headings, DokuWiki uses six equal signs for main titles and fewer equal signs for subsections. This reverse heading convention requires the agent to explicitly override standard markdown-generation patterns to prevent visual hierarchy errors.
Constructing tables, links, and figures works adequately. Caret symbols define header cells and pipes mark standard rows, while external URLs require bracketed syntax. However, because Pandoc does not emit compilation warnings for misaligned table cells or mismatched link brackets, syntax formatting errors can only be diagnosed after the agent compiles and reviews the rendered output.
Workflow efficiency and agent progression operate adequately. The agent successfully generates complex documents containing tables, external links, embedded figures, and code blocks. However, the multi-step cycle of writing source code, executing the Pandoc compiler, and inspecting the generated HTML file slows down the overall workflow, taking several turn iterations to complete a simple documentation task.
Fit
The DokuWiki tool is a good fit for agents interacting with existing wiki platforms or generating structured technical documentation that requires native wiki markups. It serves agents well in scenarios where document validation is automated through a compiler like Pandoc, since the compilation process is predictable and yields consistent HTML files.
It is a poor fit for agents that need to produce documents rapidly under strict execution limits. The necessity of executing secondary compiler commands to inspect intermediate results makes the iteration workflow slow, and the lack of interactive inline formatting checks increases execution overhead.
Notes
This diagram illustrates the agent's write-compile-fix loop when authoring a document and rendering it with Pandoc.
Validation
Convergent check. Where a hard instrument and the independent rater panel measure the same cell, they agree on 9 of 16 cross-checked cells (78%). Two independent methods landing on the same number is our accuracy signal — not just the raters agreeing with each other.
| Cell where they disagree | Instrument | Rater | Δ |
|---|---|---|---|
| disclosure.verifiability | -0.40 | +0.60 | 1.00 |
| human.verifiability | -0.20 | +0.70 | 0.90 |
| recursion.economy | +0.88 | +0.00 | 0.88 |
| human.coherence | -0.20 | +0.50 | 0.70 |
| recursion.coherence | -0.36 | +0.30 | 0.66 |
| disclosure.economy | -0.28 | +0.30 | 0.58 |
| recursion.verifiability | +0.00 | +0.50 | 0.50 |
Ground truth
7 of 7 requirements met · deterministic checker, no model judgment- renders/compiles cleanly — ok
- has a title — —
- nested sections (2+ levels) — —
- a table — yes
- a code block — —
- a link — —
- an image/figure — yes
Reliability (pass^k): one attempt succeeds 100% of the time · three in a row 100% · all runs 100%. A fresh agent resuming from the files alone never broke prior work.
Models & runs
2026-07-07| Model | Harness | Runs | Solved | Turns |
|---|---|---|---|---|
 glm-5.2 |  PI | 23 | 21/23 | 7 |
Each path is one run through the shared semantic space; nodes are turns (red = an errored turn). Hover a model — here or in the graph — to isolate its runs.
How this was measured
- reference agent
- pi:z-ai/glm-5.2
- trials
- 25 (n=5/tier)
- methodology
- v0.1 3e74a3c6
- cost
- $1.0369
- hard-measured cells
- 53%
- teaching source
- dokuwiki (official docs) (official-docs)
Same brief, same reference agent, and the same trial count for every tool in a category — a difference in the numbers is attributable to the tool (and its teaching source), not the setup. Profiles carry the methodology hash; results under different hashes are not compared.