seaborn
grade
reliability
overall
Seaborn is a Python statistical data visualization library that allows agents to programmatically generate styled charts and plots. Agents interact with the tool by writing scripts that use Seaborn's high-level API to load datasets, map variables to visual properties, and export images. Although the library reliably outputs files when code execution succeeds, standard text-based agents cannot verify the visual style, alignment, or legibility of the generated plots.
Surfaces × lenses
| surface ↓ lens → | Coherence | Economy | Consistency | Verifiability | Safety |
|---|---|---|---|---|---|
| Disclosure | |||||
| Interface | |||||
| Loop | |||||
| Recursion | |||||
| Human |
Click any cell for what was measured and why. A corner dot marks a cell filled by a hard instrument (checker, telemetry); undotted cells are judged by the rater ensemble. 53% of this matrix is hard-measured.
The experience
Seaborn's high-level API works well because it maps dataset columns to plot aesthetics with minimal code. Functions such as lineplot and scatterplot automatically handle legend generation, color-coding, and markers via properties like hue and size. Setting options like style and palette configures a legible layout without requiring the agent to write verbose Matplotlib configuration routines. This concise syntax prevents the agent from introducing coding errors during plot configuration.
The interface for data input works adequately but is highly dependent on pandas parsing mechanics. In practice, when source files contain metadata comments or custom structural lines above tabular records, the agent must debug data-loading issues before invoking Seaborn. Once the data frame is cleaned and loaded with proper string headers, Seaborn parses the input variables and assigns data groups to axes without further agent intervention.
Visual verifiability and error feedback work poorly. Standard agent environments lack multimodal feedback, which prevents the agent from reviewing the final SVG or PNG files for styling defects. If annotations overlap, or if axis labels are clipped due to tight margins, the agent cannot detect these visual errors. It must rely solely on the zero-exit-status of the Python subprocess, meaning visual regressions often pass unnoticed.
Script execution and reproducibility work well. Seaborn scripts are plain text Python files that can be run repeatedly to regenerate identical image outputs. This deterministic behavior allows an agent to safely overwrite existing figures and iterate on script parameters without corrupting prior work. Fresh agents can also resume the workflow and maintain continuity using only the saved script and clean data files.
Fit
Seaborn is a good fit for autonomous agents that need to quickly generate standard visual reports from structured, pre-cleaned tabular data. Its predictable, single-statement chart creation makes it highly reliable for tasks where visual layouts do not require real-time human or programmatic adjustment.
It is a poor fit for workflows that demand precise aesthetic refinement, heavy customized layouts, or dynamic visual validation. Because text-based agents cannot visually inspect the image files, tasks requiring exact marker placement, collision avoidance for text annotations, or custom canvas dimensions will frequently result in uncorrected visual bugs.
Notes
This diagram illustrates the agent's offline execution loop where script generation and data parsing are validated entirely by subprocess exit status, bypassing actual image verification.
Validation
Convergent check. Where a hard instrument and the independent rater panel measure the same cell, they agree on 6 of 16 cross-checked cells (70%). Two independent methods landing on the same number is our accuracy signal — not just the raters agreeing with each other.
| Cell where they disagree | Instrument | Rater | Δ |
|---|---|---|---|
| recursion.coherence | -0.71 | +0.60 | 1.31 |
| interface.coherence | -0.42 | +0.80 | 1.22 |
| human.verifiability | -0.20 | +0.90 | 1.10 |
| disclosure.verifiability | -0.40 | +0.50 | 0.90 |
| recursion.verifiability | +0.00 | +0.70 | 0.70 |
| human.coherence | -0.20 | +0.50 | 0.70 |
| human.safety | -0.10 | +0.50 | 0.60 |
| interface.prior_alignment | +0.40 | +0.90 | 0.50 |
| disclosure.economy | +0.24 | +0.70 | 0.46 |
| loop.determinism | +0.24 | +0.70 | 0.46 |
Ground truth
2 of 2 requirements met · deterministic checker, no model judgment- figure produced — figure.png
- chart source runs clean — exit 0
Reliability (pass^k): one attempt succeeds 100% of the time · three in a row 100% · all runs 100%. A fresh agent resuming from the files alone never broke prior work.
Models & runs
2026-07-07| Model | Harness | Runs | Solved | Turns |
|---|---|---|---|---|
| | 23 | 20/23 | 8 |
Each path is one run through the shared semantic space; nodes are turns (red = an errored turn). Hover a model — here or in the graph — to isolate its runs.
How this was measured
- reference agent
- pi:z-ai/glm-5.2
- trials
- 25 (n=5/tier)
- methodology
- v0.1 3e74a3c6
- cost
- $0.8943
- hard-measured cells
- 53%
- teaching source
- seaborn (official docs) (official-docs)
Same brief, same reference agent, and the same trial count for every tool in a category — a difference in the numbers is attributable to the tool (and its teaching source), not the setup. Profiles carry the methodology hash; results under different hashes are not compared.