folium
grade (soft)
reliability
overall
The folium library lets an agent generate interactive HTML maps using a programmatic Python interface. The agent builds map visualizations by instantiating map objects, adding layers or coordinate markers, and exporting the final state to a local HTML file. This local-only, object-based workflow allows the agent to construct geography-based representations without external state mutations or security risks.
Surfaces × lenses
| surface ↓ lens → | Coherence | Economy | Consistency | Verifiability | Safety |
|---|---|---|---|---|---|
| Disclosure | |||||
| Interface | |||||
| Loop | |||||
| Recursion | |||||
| Human |
Click any cell for what was measured and why. A corner dot marks a cell filled by a hard instrument (checker, telemetry); undotted cells are judged by the rater ensemble. 47% of this matrix is hard-measured.
The experience
An agent interacts with folium by writing Python scripts that instantiate spatial visualization layers. Standard API components such as markers, circle markers, and line layers align closely with common programmatic patterns, rendering predictable outputs when given coordinates. The final map objects are saved directly as HTML, which allows the agent to verify the output easily by checking the generated file content.
Friction occurs when the raw input data lacks direct geo-coordinate columns, as observed with the NYC Taxi dataset. The agent had to spend extra development cycles parsing custom file formats—including a non-standard header prefix in the data—and mapping distances to coordinate landmarks. Furthermore, the tool itself can be run-to-run unreliable, which causes repeated automated execution attempts to fail even when the underlying python code is syntactically correct.
Because coordinate generation and format parsing require substantial custom logic, completing a single map figure incurs a large amount of token cost. However, state accumulation within the tool is clean, as the cumulative map is simply an in-memory object constructed by adding separate geographic layers. The tool remains safe to run because file generation is strictly local, removing risks of remote state destruction or credential leaks.
Fit
This tool is a good fit for agents tasked with generating clean, interactive spatial visualisations from structured tables that already contain explicit latitude and longitude coordinates. Its standard Python interface is ideal for agents that can easily leverage prior training on geo-spatial plotting libraries.
It is not a good fit for workflows requiring high reliability across repeated runs, or when the agent must handle raw datasets that require extensive geocoding and preprocessing. In those scenarios, the high token cost associated with writing custom data loaders and the tool's runtime instability outweigh the simplicity of the map export.
Notes
The diagram below illustrates the programmatic loop where the agent parses the raw data headers, maps distances to coordinates, translates them into map layers, and exports the map.
Validation
Convergent check. Where a hard instrument and the independent rater panel measure the same cell, they agree on 9 of 14 cross-checked cells (82%). Two independent methods landing on the same number is our accuracy signal — not just the raters agreeing with each other.
| Cell where they disagree | Instrument | Rater | Δ |
|---|---|---|---|
| recursion.economy | +0.85 | -0.10 | 0.95 |
| disclosure.verifiability | -0.37 | +0.50 | 0.87 |
| human.safety | -0.10 | +0.50 | 0.60 |
| human.coherence | -0.20 | +0.40 | 0.60 |
| loop.economy | +0.23 | -0.20 | 0.43 |
Models & runs
2026-07-07| Model | Harness | Runs | Solved | Turns |
|---|---|---|---|---|
| | 23 | 15/23 | 6 |
Each path is one run through the shared semantic space; nodes are turns (red = an errored turn). Hover a model — here or in the graph — to isolate its runs.
How this was measured
- reference agent
- pi:z-ai/glm-5.2
- trials
- 25 (n=5/tier)
- methodology
- v0.1 3e74a3c6
- cost
- $1.1315
- hard-measured cells
- 47%
- teaching source
- folium (official docs) (official-docs)
Same brief, same reference agent, and the same trial count for every tool in a category — a difference in the numbers is attributable to the tool (and its teaching source), not the setup. Profiles carry the methodology hash; results under different hashes are not compared.