Back to explorer
infographics-posters CLI & commands best source: official docs

matplotlib

B−

grade

0%

reliability

+0.18

overall

Matplotlib is a programmatic plotting library that enables agents to generate charts, diagrams, and posters by executing Python code. The tool requires writing layout, styling, and structural commands into a script and running it to export files which then require external validation. Because execution runs headlessly, the agent cannot directly view the visual output during the design process.

Surfaces × lenses

hostile friendly
surface ↓
lens →
Coherence Economy Consistency Verifiability Safety
Disclosure
Interface
Loop
Recursion
Human

Click any cell for what was measured and why. A corner dot marks a cell filled by a hard instrument (checker, telemetry); undotted cells are judged by the rater ensemble. 53% of this matrix is hard-measured.

The experience

The agent designs standard visualizations using Matplotlib's object-oriented interface. Standard methods like subplots, bar charts, and scatter plots work adequately because their parameters correspond directly to data series and match predictable API patterns. Creating advanced multi-axis layouts using methods like twinx and applying styling options like hatch patterns also works adequately, as these properties can be configured explicitly in Python code.

However, constructing multi-element compositions like data cards and custom legends works poorly. The agent must position titles, borders, and text blocks by specifying explicit coordinate systems and bounding boxes. Because the agent lacks direct visual feedback, calculating precise spatial offsets is highly iterative and error-prone, where minor alterations frequently cause overlapping text or clipped borders without throwing structural errors.

State persistence across multiple sequential executions works poorly. Repeated attempts to run or modify code in an ongoing session frequently fail due to execution environment unreliability. Conversely, resuming development from a clean state works well, since a fresh agent resuming from saved code files can modify the layout without breaking previously established structures.

Fit

Matplotlib is a good fit for agents executing automated data visualization tasks that require standard charts and structured layouts defined through explicit code parameters. It is effective when the layout is simple or when exact coordinate alignment is not critical to the final output.

The tool is a poor fit for tasks that demand complex page designs, custom templates, or precise canvas layouts. When an agent must design visually dense media containing multi-axis charts, multi-column metric boxes, and precise text callouts, the lack of real-time visual feedback makes the process inefficient.

Notes

This diagram illustrates the agent's iterative write-execute-verify workflow when generating artifacts with Matplotlib.

Validation

Convergent check. Where a hard instrument and the independent rater panel measure the same cell, they agree on 9 of 16 cross-checked cells (69%). Two independent methods landing on the same number is our accuracy signal — not just the raters agreeing with each other.

Cell where they disagreeInstrumentRaterΔ
recursion.coherence-0.85+0.701.55
interface.coherence-0.70+0.801.50
disclosure.verifiability-0.40+0.601.00
human.verifiability-0.20+0.801.00
loop.determinism-0.30+0.600.90
human.coherence-0.20+0.600.80
human.safety-0.10+0.400.50

Ground truth

1 of 1 requirements met · deterministic checker, no model judgment
  • poster produced — gapminder_infographic.pdf

Reliability (pass^k): one attempt succeeds 40% of the time · three in a row 0% · all runs 0%. A fresh agent resuming from the files alone never broke prior work.

Models & runs

2026-07-07
ModelHarnessRunsSolvedTurns
glm-5.2 PI2316/236

Each path is one run through the shared semantic space; nodes are turns (red = an errored turn). Hover a model — here or in the graph — to isolate its runs.

How this was measured

reference agent
pi:z-ai/glm-5.2
trials
25 (n=5/tier)
methodology
v0.1 3e74a3c6
cost
$0.8619
hard-measured cells
53%
teaching source
matplotlib (official docs) (official-docs)

Same brief, same reference agent, and the same trial count for every tool in a category — a difference in the numbers is attributable to the tool (and its teaching source), not the setup. Profiles carry the methodology hash; results under different hashes are not compared.