a-frame
grade
reliability
overall
A-Frame enables an AI agent to build interactive 3D visualizations inside a single, self-contained HTML file using declarative tags. The agent structures the layout, animations, interactions, and custom javascript logic within standard HTML tags without requiring external compiled assets. Crucially, the resulting scene executes inside a browser sandbox, though the tool exhibits high instability during successive automated validation runs.
Surfaces × lenses
| surface ↓ lens → | Coherence | Economy | Consistency | Verifiability | Safety |
|---|---|---|---|---|---|
| Disclosure | |||||
| Interface | |||||
| Loop | |||||
| Recursion | |||||
| Human |
Click any cell for what was measured and why. A corner dot marks a cell filled by a hard instrument (checker, telemetry); undotted cells are judged by the rater ensemble. 53% of this matrix is hard-measured.
Teaching sources compared
Teaching source barely matters for a-frame: sources land within a grade of each other (docs B−, skill B−).
| Source | Grade | Renders | Disclosure | Interface | Loop | Recursion | Human |
|---|---|---|---|---|---|---|---|
| official docs ★ | B− | ✓ | +0.28 | +0.54 | +0.25 | +0.29 | +0.15 |
| skill | B− | ✓ | +0.36 | +0.26 | +0.14 | +0.37 | -0.07 |
Same harness, same reference agent, same briefs, same trial count — the only variable is how the agent was taught the tool. ★ = best source.
The experience
The declarative markup syntax in A-Frame works well for agent code-generation. The agent structures complex assemblies by mapping positions, rotations, colors, and animations directly as HTML attributes on primitive entities like <a-box> or <a-cylinder>. Because the framework uses a highly readable HTML element model rather than imperative drawing APIs, the agent is able to compose a detailed 3D environment in a single pass without generating massive code files.
Attaching custom interactive behaviors works well through local script integration. The agent registers new behavioral scripts using standard AFRAME.registerComponent definitions, creating reactive nodes that handle cursor click events or execute per-frame logic in tick hooks. Because the runtime runs entirely in the standard browser sandbox, the implementation is safe from system side-effects and is easily reversed by editing the single source file.
However, runtime validation and error correction work poorly. The agent has no native pathway to inspect the executed 3D space, meaning it cannot verify position coordinates, event listeners, or animation trajectories without external rendering helpers. If a custom component fails silently or elements clip behind the camera, the console errors do not propagate to the agent's context, leading to high run-to-run instability during successive automated tests.
Fit
A-Frame is a good fit for agents tasked with producing lightweight, interactive 3D visualizations, educational simulations, or web-based UI scenes where sandboxed execution and quick multi-object assembly are key. It is not a good fit for projects requiring high-fidelity deterministic consistency across automated parallel environments, or complex interactive projects where the agent must perform closed-loop visual validation and debugging before final delivery.
Notes
A workflow diagram illustrating the single-file development loop and verification gap:
Validation
Convergent check. Where a hard instrument and the independent rater panel measure the same cell, they agree on 9 of 16 cross-checked cells (77%). Two independent methods landing on the same number is our accuracy signal — not just the raters agreeing with each other.
| Cell where they disagree | Instrument | Rater | Δ |
|---|---|---|---|
| disclosure.verifiability | -0.40 | +0.70 | 1.10 |
| human.verifiability | -0.20 | +0.80 | 1.00 |
| recursion.coherence | -0.25 | +0.60 | 0.85 |
| human.safety | -0.10 | +0.70 | 0.80 |
| loop.determinism | -0.20 | +0.50 | 0.70 |
| human.coherence | -0.20 | +0.50 | 0.70 |
| recursion.determinism | +1.00 | +0.50 | 0.50 |
Ground truth
6 of 6 requirements met · deterministic checker, no model judgment- html scene produced — raft-consensus.html
- real 3D runtime — —
- scene root — —
- 3+ entities — 28 entities
- a light — —
- animation or interaction — —
Reliability (pass^k): one attempt succeeds 60% of the time · three in a row 10% · all runs 0%. A fresh agent resuming from the files alone never broke prior work.
Models & runs
2026-07-07| Model | Harness | Runs | Solved | Turns |
|---|---|---|---|---|
 glm-5.2 |  PI | 25 | 15/25 | 6 |
Each path is one run through the shared semantic space; nodes are turns (red = an errored turn). Hover a model — here or in the graph — to isolate its runs.
How this was measured
- reference agent
- pi:z-ai/glm-5.2
- trials
- 25 (n=5/tier)
- methodology
- v0.1 3e74a3c6
- cost
- $0.5
- hard-measured cells
- 53%
- teaching source
- a-frame (official docs) (official-docs)
Same brief, same reference agent, and the same trial count for every tool in a category — a difference in the numbers is attributable to the tool (and its teaching source), not the setup. Profiles carry the methodology hash; results under different hashes are not compared.