pygal
grade
reliability
overall
Pygal is a Python charting library that outputs plots in native SVG format. Handing this tool to an agent allows it to construct customizable data visualizations using direct object configurations. However, execution unreliability and interactive tooltip defaults present significant programmatic and operational friction, requiring the agent to spend substantial resources to achieve a simple static plot.
Surfaces × lenses
| surface ↓ lens → | Coherence | Economy | Consistency | Verifiability | Safety |
|---|---|---|---|---|---|
| Disclosure | |||||
| Interface | |||||
| Loop | |||||
| Recursion | |||||
| Human |
Click any cell for what was measured and why. A corner dot marks a cell filled by a hard instrument (checker, telemetry); undotted cells are judged by the rater ensemble. 53% of this matrix is hard-measured.
The experience
The initial chart configuration works well. The agent instantiates charts like pygal.XY or pygal.Line, populating attributes like titles, styles, and axis labels using standard Python dictionary syntax. These API declarations are predictable, allowing the agent to align its code with the documentation on the first try.
Annotating specific data points works adequately but introduces workflow friction. Pygal encodes user labels into SVG <desc> tags, rendering them as interactive hover-state tooltips rather than visible text. For static image output, the agent must realize that these annotations are invisible, prompting a pivot to configure static value printing directly on-chart. This transition requires verbose sub-dictionaries inside the data payload.
Execution stability within the runtime environment works poorly. Repeated attempts to compile and render the charts frequently fail, indicating high unreliability from one run to another. This instability forces the agent to spend a large amount of tokens and make many successive execution turns to produce a simple, single-frame plot.
State preservation across agent handoffs works well. When a new agent resumes a task relying solely on existing project files and generated SVG code, it consistently maintains progression without breaking prior configurations. This makes the tool robust to state interruptions.
Fit
This tool is a good fit for agents that need to output clean, lightweight vector graphics without installing browser-based rendering stacks. The predictable API and standard Python-based object model make it easy for an agent to draft and adjust basic chart structures.
This tool is a poor fit for automated pipelines requiring high run-to-run reliability and low execution latency. The intermittent execution failures and the complexity of rendering static annotations on raw SVGs make the tool expensive in terms of token usage and overall processing time.
Notes
The diagram illustrates the agent's iterative sequence to generate the chart and resolve hidden label issues.
Validation
Convergent check. Where a hard instrument and the independent rater panel measure the same cell, they agree on 11 of 16 cross-checked cells (80%). Two independent methods landing on the same number is our accuracy signal — not just the raters agreeing with each other.
| Cell where they disagree | Instrument | Rater | Δ |
|---|---|---|---|
| recursion.determinism | +0.98 | +0.00 | 0.98 |
| disclosure.verifiability | -0.53 | +0.30 | 0.83 |
| recursion.economy | +0.12 | -0.50 | 0.62 |
| human.safety | -0.10 | +0.50 | 0.60 |
| loop.economy | -0.10 | -0.50 | 0.40 |
Ground truth
4 of 4 requirements met · deterministic checker, no model judgment- figure produced — figure.svg
- chart source runs clean — exit 0
- title/axis/legend labels (3+ text) — 20 text nodes
- 2+ series colours — 2 colours
Reliability (pass^k): one attempt succeeds 60% of the time · three in a row 10% · all runs 0%. A fresh agent resuming from the files alone never broke prior work.
Models & runs
2026-07-07| Model | Harness | Runs | Solved | Turns |
|---|---|---|---|---|
 glm-5.2 |  PI | 23 | 14/23 | 7 |
Each path is one run through the shared semantic space; nodes are turns (red = an errored turn). Hover a model — here or in the graph — to isolate its runs.
How this was measured
- reference agent
- pi:z-ai/glm-5.2
- trials
- 25 (n=5/tier)
- methodology
- v0.1 3e74a3c6
- cost
- $1.0289
- hard-measured cells
- 53%
- teaching source
- pygal (official docs) (official-docs)
Same brief, same reference agent, and the same trial count for every tool in a category — a difference in the numbers is attributable to the tool (and its teaching source), not the setup. Profiles carry the methodology hash; results under different hashes are not compared.