gnuplot
grade
reliability
overall
Gnuplot is a command-line utility that renders data visualizations from declarative script files. When provided to an agent, it serves as a secure, non-destructive execution environment for plotting structured datasets. Standard documentation allows the agent to construct syntactically valid scripts, but the tool is highly sensitive to schema mismatches, making accurate data-to-visual mapping difficult to achieve without multiple iterations.
Surfaces × lenses
| surface ↓ lens → | Coherence | Economy | Consistency | Verifiability | Safety |
|---|---|---|---|---|---|
| Disclosure | |||||
| Interface | |||||
| Loop | |||||
| Recursion | |||||
| Human |
Click any cell for what was measured and why. A corner dot marks a cell filled by a hard instrument (checker, telemetry); undotted cells are judged by the rater ensemble. 53% of this matrix is hard-measured.
The experience
The agent interacts with the tool by writing a declarative script file and invoking the executable through a shell command. While the execution environment is non-destructive and safe to run, aligning input datasets with the script's plotting parameters presents significant operational difficulty. The agent frequently misidentifies column indices during the mapping process—such as pairing population size or GDP variables with the wrong data columns—which leads to malformed plots that must be corrected through successive trial-and-error attempts.
The feedback loop is constrained by the shell interface's lack of native preview capabilities. Because the agent cannot programmatically inspect or query the visual layout properties of the output plot, it must rely on binary exit codes or external validation processes to confirm rendering success. This separation between script execution and visual verification forces the agent to execute a high number of sequential terminal commands, making the task slow and expensive.
The workflow is further degraded by state losses across interrupted sessions. When a new agent instance resumes a session from existing files, it regularly reverts prior progress by overwriting working scripts with default configurations. The tool provides no local metadata or persistence to signal previous execution boundaries, meaning the resuming agent must restart the column alignment process from the beginning.
Fit
Gnuplot is a suitable fit for pipelines where data schemas are static and predictable. In automated workflows with established formats, an agent can rely on the tool's deterministic layout commands and standard color palettes to generate output files without manual alignment loops.
It is a poor fit for dynamic plotting tasks where data schemas vary or are unknown beforehand. When an agent is required to dynamically discover CSV columns and compute variables under strict execution limits, the absence of rich inline parsing feedback makes the alignment process prone to costly, recurring errors.
Notes
The diagram illustrates the iterative execution loop required to resolve column mapping errors in gnuplot scripts.
Validation
Convergent check. Where a hard instrument and the independent rater panel measure the same cell, they agree on 9 of 16 cross-checked cells (78%). Two independent methods landing on the same number is our accuracy signal — not just the raters agreeing with each other.
| Cell where they disagree | Instrument | Rater | Δ |
|---|---|---|---|
| recursion.economy | +0.77 | -0.30 | 1.07 |
| disclosure.verifiability | -0.40 | +0.50 | 0.90 |
| loop.verifiability | +0.60 | -0.20 | 0.80 |
| recursion.coherence | -0.63 | +0.00 | 0.63 |
| interface.verifiability | +0.60 | +0.00 | 0.60 |
| human.safety | -0.10 | +0.50 | 0.60 |
| loop.economy | +0.15 | -0.30 | 0.45 |
Ground truth
1 of 1 requirements met · deterministic checker, no model judgment- figure produced — gapminder.png
Reliability (pass^k): one attempt succeeds 60% of the time · three in a row 10% · all runs 0%. A fresh agent resuming from the files alone sometimes regressed prior work.
Models & runs
2026-07-07| Model | Harness | Runs | Solved | Turns |
|---|---|---|---|---|
 glm-5.2 |  PI | 23 | 16/23 | 6 |
Each path is one run through the shared semantic space; nodes are turns (red = an errored turn). Hover a model — here or in the graph — to isolate its runs.
How this was measured
- reference agent
- pi:z-ai/glm-5.2
- trials
- 25 (n=5/tier)
- methodology
- v0.1 3e74a3c6
- cost
- $0.7248
- hard-measured cells
- 53%
- teaching source
- gnuplot (official docs) (official-docs)
Same brief, same reference agent, and the same trial count for every tool in a category — a difference in the numbers is attributable to the tool (and its teaching source), not the setup. Profiles carry the methodology hash; results under different hashes are not compared.