bokeh
grade
reliability
overall
Bokeh is a Python data visualization library that produces interactive, browser-based plots exported as self-contained HTML files. The tool utilizes a structured Python API that allows an agent to define figures, lines, scatter markers, labels, and legends in script files. For an AI agent, Bokeh provides a deterministic framework to output visual assets that render cleanly in standard web browsers.
Surfaces × lenses
| surface ↓ lens → | Coherence | Economy | Consistency | Verifiability | Safety |
|---|---|---|---|---|---|
| Disclosure | |||||
| Interface | |||||
| Loop | |||||
| Recursion | |||||
| Human |
Click any cell for what was measured and why. A corner dot marks a cell filled by a hard instrument (checker, telemetry); undotted cells are judged by the rater ensemble. 53% of this matrix is hard-measured.
The experience
The agent interacts with Bokeh by writing Python scripts to load CSV datasets, instantiate figure layouts, and apply glyph methods such as line and scatter. Saving outputs to HTML via the output_file and save functions is highly reliable and operates without external dependencies. However, exporting to SVG requires a browser webdriver, and when this webdriver is absent, the export fails silently without returning clear error messages, creating a significant troubleshooting hurdle.
Raw CSV files with irregular preambles or parsing anomalies also introduce friction for the agent. While the official documentation provides clear and structured plotting examples, it does not address common data preparation or parsing challenges in real-world files. As a result, the agent spends substantial developmental effort preparing input data before it can successfully invoke the plotting library.
Consequently, the initial development feedback loop requires multiple execution attempts to resolve formatting and layout requirements. The process often takes several iteration steps and a large amount of token exchanges to compile, making the workflow visually successful but computationally expensive to execute.
Despite these hurdles, the final artifact production is fully reproducible. The tool behaves deterministically, ensuring that consecutive attempts produce identical visual charts and that subsequent agent sessions can safely resume editing without destroying existing layout files.
Fit
Bokeh is a good fit for agents that need to programmatically construct interactive, complex visually-oriented dashboards or plots in HTML. It is best used when visual results can be delivered directly inside a browser file rather than as static image formats.
It is a poor fit for environments that demand static image exports such as SVG without webdriver configurations, or for resource-constrained workflows that require rapid, low-cost plotting with minimal token consumption.
Notes
This diagram shows the execution path and failure points during the agent's visualization loop when producing HTML versus SVG outputs.
Validation
Convergent check. Where a hard instrument and the independent rater panel measure the same cell, they agree on 7 of 16 cross-checked cells (77%). Two independent methods landing on the same number is our accuracy signal — not just the raters agreeing with each other.
| Cell where they disagree | Instrument | Rater | Δ |
|---|---|---|---|
| interface.verifiability | +1.00 | -0.10 | 1.10 |
| recursion.determinism | +1.00 | +0.10 | 0.90 |
| recursion.economy | +0.52 | -0.30 | 0.82 |
| interface.coherence | -0.61 | +0.20 | 0.81 |
| recursion.coherence | -0.80 | -0.10 | 0.70 |
| disclosure.verifiability | -0.40 | +0.30 | 0.70 |
| loop.economy | +0.25 | -0.30 | 0.55 |
| loop.determinism | +0.70 | +0.20 | 0.50 |
| loop.safety | +0.70 | +0.20 | 0.50 |
Ground truth
1 of 2 requirements met · deterministic checker, no model judgment- figure produced — none
- chart source runs clean — exit 0
Reliability (pass^k): one attempt succeeds 100% of the time · three in a row 100% · all runs 100%. A fresh agent resuming from the files alone never broke prior work.
Models & runs
2026-07-07| Model | Harness | Runs | Solved | Turns |
|---|---|---|---|---|
 glm-5.2 |  PI | 23 | 19/23 | 8 |
Each path is one run through the shared semantic space; nodes are turns (red = an errored turn). Hover a model — here or in the graph — to isolate its runs.
How this was measured
- reference agent
- pi:z-ai/glm-5.2
- trials
- 25 (n=5/tier)
- methodology
- v0.1 3e74a3c6
- cost
- $1.0401
- hard-measured cells
- 53%
- teaching source
- bokeh (official docs) (official-docs)
Same brief, same reference agent, and the same trial count for every tool in a category — a difference in the numbers is attributable to the tool (and its teaching source), not the setup. Profiles carry the methodology hash; results under different hashes are not compared.