pillow
grade
reliability
overall
Pillow is a Python library that enables agents to programmatically build, manipulate, and render raster images. It provides low-level drawing commands for shapes and text, along with basic utility filters and resizing operations. Handing an agent this library for complex layout tasks requires writing heavy coordinate-level math and integrating independent visualization libraries because Pillow does not provide automated layout or charting capabilities.
Surfaces × lenses
| surface ↓ lens → | Coherence | Economy | Consistency | Verifiability | Safety |
|---|---|---|---|---|---|
| Disclosure | |||||
| Interface | |||||
| Loop | |||||
| Recursion | |||||
| Human |
Click any cell for what was measured and why. A corner dot marks a cell filled by a hard instrument (checker, telemetry); undotted cells are judged by the rater ensemble. 53% of this matrix is hard-measured.
The experience
Drawing shapes and compositing raster layers works adequately. The agent uses ImageDraw.Draw to overlay rectangles, lines, and text onto a background image canvas, and uses Image.open and Image.paste to combine separate graphic assets such as charts. This direct manipulation is fast and predictable, allowing the agent to assemble a multi-layer layout without significant runtime overhead.
Managing typographic layouts and text wrapping works poorly because Pillow has no formatting engine. The agent must write custom wrapping functions in Python that calculate text dimensions character-by-character and line-by-line using font.getbbox or ancestral methods. These manual calculations often result in fragmented iteration loops and output clipping when text lengths or font sizes vary.
Font handling and cross-platform consistency work poorly due to font loading mechanics. To load TrueType fonts, the agent specifies file paths using ImageFont.truetype, which frequently fail with an OSError if the host environment does not possess the specific font file. The agent relies on fallback structures that default to ImageFont.load_default, which alters rendering sizes and corrupts layout geometry, making the rendering behavior unstable across environment reinits.
Generating vector formats like SVG or PDF works poorly. Because Pillow is natively designed for rasterization, the agent cannot directly export scalable graphics. The agent must instead bypass Pillow entirely, writing manual XML code to build SVGs and embedding raster graphics inside them as base64-encoded strings, resulting in code churn and file format mismatches.
Fit
Pillow is a good fit for agents performing basic raster image transformations, resizing, cropping, watermarking, and minor annotation tasks on existing images. The clean API and fast rendering are well suited for simple, single-frame automation in pipelines that have predictable font assets and do not require complex text layout.
The library is a poor fit for agents tasked with generating complex layouts, infographics, charts, or vector collateral. Without robust automated spacing, text flow controls, and native chart generation, the agent must write long layout scripts that are vulnerable to font changes and coordinate recalculations.
Notes
The diagram shows the agent loop when a font-loading failure triggers a fallback and corrupts visual alignment.
Validation
Convergent check. Where a hard instrument and the independent rater panel measure the same cell, they agree on 13 of 16 cross-checked cells (87%). Two independent methods landing on the same number is our accuracy signal — not just the raters agreeing with each other.
| Cell where they disagree | Instrument | Rater | Δ |
|---|---|---|---|
| recursion.determinism | +1.00 | +0.00 | 1.00 |
| disclosure.verifiability | -0.40 | +0.30 | 0.70 |
| recursion.economy | +0.64 | +0.00 | 0.64 |
Ground truth
5 of 5 requirements met · deterministic checker, no model judgment- poster produced — gapminder_poster.svg
- portrait orientation — 900×1350
- at least ~800×1200 — 900×1350
- 3+ callouts match the real data — 5 data values found in text
- has a headline — —
Reliability (pass^k): one attempt succeeds 40% of the time · three in a row 0% · all runs 0%. A fresh agent resuming from the files alone never broke prior work.
Models & runs
2026-07-07| Model | Harness | Runs | Solved | Turns |
|---|---|---|---|---|
 glm-5.2 |  PI | 25 | 17/25 | 9 |
Each path is one run through the shared semantic space; nodes are turns (red = an errored turn). Hover a model — here or in the graph — to isolate its runs.
How this was measured
- reference agent
- pi:z-ai/glm-5.2
- trials
- 25 (n=5/tier)
- methodology
- v0.1 3e74a3c6
- cost
- $0.8753
- hard-measured cells
- 53%
- teaching source
- pillow (official docs) (official-docs)
Same brief, same reference agent, and the same trial count for every tool in a category — a difference in the numbers is attributable to the tool (and its teaching source), not the setup. Profiles carry the methodology hash; results under different hashes are not compared.