imagemagick
grade
reliability
overall
ImageMagick is a command-line image processing suite that agents invoke directly through shell execution. It processes images by applying sequential operations—such as resizing, cropping, and annotation—in a predictable left-to-right order. While the tool allows highly token-efficient programmatic edits, agents struggle to verify if the output is visually correct because the interface does not provide immediate render feedback.
Surfaces × lenses
| surface ↓ lens → | Coherence | Economy | Consistency | Verifiability | Safety |
|---|---|---|---|---|---|
| Disclosure | |||||
| Interface | |||||
| Loop | |||||
| Recursion | |||||
| Human |
Click any cell for what was measured and why. A corner dot marks a cell filled by a hard instrument (checker, telemetry); undotted cells are judged by the rater ensemble. 53% of this matrix is hard-measured.
The experience
Command-line execution of ImageMagick works well because the agent uses native bash commands, which prevents the conversational inflation associated with JSON payloads or complex API wrappers. Because the tool relies on standard CLI parameters such as -resize, -gravity, and -annotate, the agent easily leverages its pre-trained knowledge to compose correct commands. The predictable left-to-right processing model ensures that tasks like scaling, cropping, and text compositing execute in the exact order specified.
Output verification works poorly because raw shell execution does not return visual feedback. The agent is forced to execute secondary inspection commands, such as checking file sizes with ls -la to ensure compliance with a size budget, rather than verifying the composition itself. Consequently, the agent must run multiple iterative loops to refine the image layout, leading to a high turn count before finishing the task.
Safety and persistence work adequately when performing non-destructive modifications. Programmatic safety is preserved because the agent specifies distinct output filenames, which avoids overwriting original source assets. Additionally, if the agent fails or must resume from a saved session, it can continue processing the existing image files immediately without corrupting previous modifications or losing progress.
Run-to-run reliability works poorly because execution can fail unpredictably across successive runs without clear error feedback to guide the agent. Escaping special shell characters, such as the greater-than symbol in resize constraints, also introduces minor variations in how the shell parses commands. When these failures occur, the lack of an integrated visual debugger makes troubleshooting highly inefficient for the agent.
Fit
This tool is a good fit for agents performing programmatic, high-volume image operations where the requirements are governed by strict numeric constraints such as dimensions and file size budgets. Because the sequential CLI syntax is deterministic, the agent can execute standard crops, scaling, and compression adjustments efficiently.
It is a bad fit for tasks requiring precise aesthetic refinement, subjective visual quality checks, or complex spatial alignment. Without an integrated rendering check or an automated visual feed, the agent cannot verify if the composite text or adjusted tones are legible and visually balanced.
Notes
This diagram shows the iterative loop where the agent edits images, checks file properties, and refines parameters.
Validation
Convergent check. Where a hard instrument and the independent rater panel measure the same cell, they agree on 9 of 16 cross-checked cells (79%). Two independent methods landing on the same number is our accuracy signal — not just the raters agreeing with each other.
| Cell where they disagree | Instrument | Rater | Δ |
|---|---|---|---|
| disclosure.verifiability | -0.40 | +0.60 | 1.00 |
| recursion.economy | +0.80 | +0.00 | 0.80 |
| recursion.determinism | +1.00 | +0.30 | 0.70 |
| disclosure.economy | +0.04 | +0.70 | 0.66 |
| human.coherence | -0.20 | +0.40 | 0.60 |
| human.verifiability | +0.10 | +0.60 | 0.50 |
| loop.determinism | -0.04 | +0.40 | 0.44 |
Ground truth
4 of 4 requirements met · deterministic checker, no model judgment- WebP output produced — 50KB
- JPEG fallback produced — 72KB
- cropped to ~1.91:1 frame — 1200×630 (1.90:1)
- each output under 200KB — 50KB, 72KB
Reliability (pass^k): one attempt succeeds 80% of the time · three in a row 40% · all runs 0%. A fresh agent resuming from the files alone never broke prior work.
Models & runs
2026-07-07| Model | Harness | Runs | Solved | Turns |
|---|---|---|---|---|
 glm-5.2 |  PI | 23 | 20/23 | 7 |
Each path is one run through the shared semantic space; nodes are turns (red = an errored turn). Hover a model — here or in the graph — to isolate its runs.
How this was measured
- reference agent
- pi:z-ai/glm-5.2
- trials
- 25 (n=5/tier)
- methodology
- v0.1 3e74a3c6
- cost
- $0.7901
- hard-measured cells
- 53%
- teaching source
- imagemagick (official docs) (official-docs)
Same brief, same reference agent, and the same trial count for every tool in a category — a difference in the numbers is attributable to the tool (and its teaching source), not the setup. Profiles carry the methodology hash; results under different hashes are not compared.