Back to explorer
markup Declarative files best source: official docs

textile

A

grade

100%

reliability

+0.50

overall

The textile tool parses plain-text Textile markup and renders it into PDF or HTML files using a Pandoc compiler. Handing this tool to an agent allows it to generate structured, formatted documents locally with standard shell commands. The process is deterministic, delivering consistent and predictable outputs across runs.

Surfaces × lenses

hostile friendly
surface ↓
lens →
Coherence Economy Consistency Verifiability Safety
Disclosure
Interface
Loop
Recursion
Human

Click any cell for what was measured and why. A corner dot marks a cell filled by a hard instrument (checker, telemetry); undotted cells are judged by the rater ensemble. 53% of this matrix is hard-measured.

The experience

Writing Textile markup works well for the agent because the prefix syntax, such as h2. for headings and pipe characters for tables, is direct and simple. The agent can construct complex document hierarchies, inline styling, code blocks, and tables using plain text. This keeps the draft phase straightforward and syntactically uncomplicated.

The tool performs adequately when handling links and embedded images, which use specific tag structures like exclamation marks for image references. For workflows that require embedded figures, the agent executes an external Python script to generate the image, then references its path in the Textile source. Resolving these dependencies requires multi-step coordination, but it works reliably because file paths are resolved locally.

The compiler performs poorly at alerting the agent to formatting mistakes. Pandoc silently compiles incorrect Textile markup as plain text rather than failing or returning explicit error messages. Because the compiler does not raise exceptions for syntax errors, the agent cannot rely on standard error streams to debug format regressions and must manually inspect the rendered XML or HTML outputs to confirm structural integrity.

Fit

This tool is a good fit for agents designed to generate structured files such as scientific reports, technical specifications, or documents with heavy tabular data. It suits workflows where output layout is straightforward and the operating environment allows direct control over local compilation commands.

It is a poor fit for agents that must produce precise visual layouts without external verification capabilities. Because the Pandoc parser does not output compilation errors when encountering invalid markup, an unmonitored agent can easily render broken tables or unparsed tags without detecting the defects.

Notes

This diagram shows the multi-step compile and verification loop used to generate a document with embedded assets.

Validation

Convergent check. Where a hard instrument and the independent rater panel measure the same cell, they agree on 9 of 16 cross-checked cells (78%). Two independent methods landing on the same number is our accuracy signal — not just the raters agreeing with each other.

Cell where they disagreeInstrumentRaterΔ
disclosure.verifiability-0.40+0.701.10
disclosure.economy-0.28+0.801.08
human.verifiability-0.20+0.801.00
recursion.coherence-0.25+0.600.85
human.coherence+0.10+0.700.60
recursion.verifiability+0.17+0.700.53
recursion.economy+1.00+0.500.50

Ground truth

7 of 7 requirements met · deterministic checker, no model judgment
  • renders/compiles cleanly — ok
  • has a title — —
  • nested sections (2+ levels) — —
  • a table — yes
  • a code block — —
  • a link — —
  • an image/figure — yes

Reliability (pass^k): one attempt succeeds 100% of the time · three in a row 100% · all runs 100%. A fresh agent resuming from the files alone never broke prior work.

Models & runs

2026-07-07
ModelHarnessRunsSolvedTurns
glm-5.2 PI2321/236

Each path is one run through the shared semantic space; nodes are turns (red = an errored turn). Hover a model — here or in the graph — to isolate its runs.

How this was measured

reference agent
pi:z-ai/glm-5.2
trials
25 (n=5/tier)
methodology
v0.1 3e74a3c6
cost
$0.8643
hard-measured cells
53%
teaching source
textile (official docs) (official-docs)

Same brief, same reference agent, and the same trial count for every tool in a category — a difference in the numbers is attributable to the tool (and its teaching source), not the setup. Profiles carry the methodology hash; results under different hashes are not compared.