Benchmarks
What an app costs to build with AI
Claude built the same apps in each stack: small apps from scratch, full-stack apps, and changes to existing projects of 11, 42 and 102 components. Every app was run and used in a simulated browser. The cost is the real API bill: the spec in the prompt, every retry and the thinking tokens.Claude Haiku 4.5−36%cost per working app vs React + TypeScript
ArtScript
React + TS
Svelte 5
Vue 3
SolidJS
Next.js
Claude Sonnet 5.5−48%cost per working app vs React + TypeScript
ArtScript
React + TS
Svelte 5
Vue 3
SolidJS
Next.js
Claude Opus 5.5−45%cost per working app vs React + TypeScript
ArtScript
React + TS
Svelte 5
Vue 3
SolidJS
Next.js
Read the numbers honestlyTwo or three runs per task is an early signal, not a definitive benchmark. With the smallest model, Haiku, ArtScript solved fewer of the new-app tasks than React and Next.js (39/48, against 40/48 and 44/48). The raw data of every run is in the repository, and anyone can reproduce it with npm run eval.
Where it doesn't save: imperative codeOne task where almost all the code is a physics loop on a canvas (gravity, walls, elastic collisions, drawing): there ArtScript is JavaScript with another syntax for statements, and the question is only whether it costs the same. Measured apart from the tables above, two runs per cell, USD per solved task.
Four runs per cell (two full rounds), with the full spec and no project rules: here ArtScript cost more than React with every model. The breakdown showed why: the code was still a quarter shorter, but the model deliberated 650–1,600 extra tokens writing a less familiar syntax, and retried more. Then, measured apart (ArtScript only, two runs per model): with the project rule that puts the physics in a .ts file imported with use, and the core spec with a whole app as the example, every model wrote sim.ts + app.art at the first attempt with React-level thinking: Sonnet $0.0227 (React $0.0209), Opus $0.0618 (React $0.0634), Haiku $0.0084 (React $0.0110). New projects now carry that rule and the core spec. Then the core spec was measured on all 50 tasks (ArtScript cells, two runs): Sonnet $0.0092 against $0.0088 with the full spec, Haiku $0.0088 against $0.0088, Opus $0.0239 against $0.0220: the same results, no cheaper, so the full spec stays what new projects read first.
| Model | ArtScript | React + TS | Svelte 5 | Vue 3 | SolidJS | vs React |
|---|---|---|---|---|---|---|
| Claude Haiku 4.5 | $0.0201 (4/4) | $0.0110 (4/4) | $0.0411 (3/4) | $0.0231 (4/4) | $0.0275 (3/4) | +83% |
| Claude Sonnet 5.5 | $0.0287 (4/4) | $0.0209 (4/4) | $0.0203 (4/4) | $0.0207 (4/4) | $0.0207 (4/4) | +37% |
| Claude Opus 5.5 | $0.0928 (4/4) | $0.0634 (4/4) | $0.0536 (4/4) | $0.0584 (4/4) | $0.0580 (4/4) | +47% |
Against Next.js, with and without a backendPer working result, ArtScript cost 50% less than Next.js 16 with Sonnet, 45% less with Opus and 29% less with Haiku. Next.js costs about the same as React with Sonnet and Opus, also on the full-stack tasks: writing route handlers wasn't cheaper for the model than a hand-written Node server.With Haiku, Next.js solved more tasks than ArtScript: 96/100 against 90/100, and 16/18 against 13/18 on the full-stack ones. It costs more per result, but it fails less with the smallest model. For real work, use Sonnet or Opus: both stacks solved every task with them.
How it was measured: every Next.js answer had to pass a real next build, a real next start served its route handlers, and the pages went through the same checks as every other stack, in a simulated browser with a small stand-in for Next's client router. Its apps ship about 147 KB of JavaScript (brotli, all of the build's client chunks), against about 8 KB.Keep in mind: Next.js and ArtScript's Sonnet cell are from 2026-11-04; ArtScript with Opus and Haiku and React are from 2026-10-02, when ArtScript's spec was shorter (with Sonnet that made it about 5% more expensive). Two runs per task. The Next.js apps use route handlers and client components, with data in memory; Server Actions, and Server Components reading data directly, weren't measured.
| Model | USD per working result | ArtScript | Next.js | Difference | React |
|---|---|---|---|---|---|
| Sonnet 5.5 | All 50 tasks | $0.0093 (100/100) | $0.0186 (100/100) | −50% | $0.0178 (100/100) |
| Sonnet 5.5 | Without a backend (41) | $0.0092 (82/82) | $0.0170 (82/82) | −46% | $0.0162 (82/82) |
| Sonnet 5.5 | Full-stack (9) | $0.0095 (18/18) | $0.0261 (18/18) | −64% | $0.0250 (18/18) |
| Opus 5.5 | All 50 tasks | $0.0220 (100/100) | $0.0403 (100/100) | −45% | $0.0397 (99/99) |
| Opus 5.5 | Without a backend (41) | $0.0210 (82/82) | $0.0349 (82/82) | −40% | $0.0338 (82/82) |
| Opus 5.5 | Full-stack (9) | $0.0265 (18/18) | $0.0652 (18/18) | −59% | $0.0683 (17/17) |
| Haiku 4.5 | All 50 tasks | $0.0088 (90/100) | $0.0123 (96/100) | −29% | $0.0137 (88/100) |
| Haiku 4.5 | Without a backend (41) | $0.0068 (77/82) | $0.0083 (80/82) | −18% | $0.0099 (73/82) |
| Haiku 4.5 | Full-stack (9) | $0.0207 (13/18) | $0.0326 (16/18) | −37% | $0.0322 (15/18) |
Motion-heavy design: measured tooA landing with a hero that rises in, a gradient that drifts behind it, three cards that reveal on scroll one after another, a button that grows on hover and a number that counts up. In ArtScript those are flags on any element (reveal, animate=, stagger, hover=); in the other stacks the model writes the CSS, an IntersectionObserver and a requestAnimationFrame loop. The code ArtScript writes is 40–55% shorter and Sonnet and Opus solve it at the first attempt; measured per fresh conversation it still costs more (the spec written to the cache each time, plus 900–1,200 tokens of thinking over props the model hasn't seen), and in a session with the spec cached the same runs cost 3–9% less than React. The table and the full reading are on the benchmarks section of the README.
Runtime speedIn the official js-framework-benchmark results (Chrome 154, October 2026), ArtScript scores 1.32 (1.00 is the fastest keyed implementation): between Svelte (1.24) and Vue (1.33), 1.14× Solid and well ahead of React (1.91), with 7.5 KB compressed. It ties the fastest at clearing rows and is slowest relative to Solid at creating 10,000 rows (1.20×).
How it was measured
- The 2026-10-02 measurement (the numbers above): 50 tasks, three models, five stacks, two runs per task. Before it, a pilot ran ArtScript alone once per model (USD 2.34, files
2026-10-02T16-1*); what the models tripped on was fixed in the compiler (accepting what they write, clearer errors, aforstatement) and the measurement started from that version. Haiku then solved 76/100 ArtScript runs, against 86–89 in the other stacks; two more rounds of the same kind of fixes followed, and its ArtScript cells were run again each time (80/100, then 90/100, the one reported). React, Svelte, Vue and Solid ran once: their toolchains didn't change. Sonnet's and Opus's ArtScript cells ran once, between those rounds. Every answer of every run is in the raw data, and--replayre-checks them with the current compiler. - Each task is the same functional request for every stack: small apps created from scratch (some full-stack), small modifications, and modifications to larger generated projects (11, 42 and 102 components). Claude gets the task, returns files, and the harness validates them: ArtScript with its compiler, React and SolidJS with strict
tsc, Svelte and Vue with their compilers. Errors are fed back, up to 3 attempts. - In modification tasks each stack may use its cheapest edit format: ArtScript an
art patch, React and Svelte search/replace edit blocks (like a coding agent's Edit tool). Full files are also accepted. Runs before 2026-10-01 had no edit formats: every stack returned full files. - Cost is computed from the real
usagethe API returns: the ArtScript spec in the system prompt, retries and thinking tokens (billed as output) all count. - ArtScript's system prompt includes its spec (1.2K–3.2K tokens depending on the run date; 3.2K in the 2026-10-02 measurement), which is served from the prompt cache after the first request; the "without prompt cache" column prices those tokens at the full input rate.
- Since 2026-10-01 every app is also run and used like a person would: it's mounted in a simulated browser (happy-dom) and a stack-agnostic check clicks, types and reads the screen (e.g. adds and completes todos, reloads the page to check data persisted on the server). A failed check is fed back to Claude like a compiler error. Earlier runs only checked that code compiled and typechecked.
- Next.js (2026-11-04, the three models): every answer has to pass a real
next build, a realnext startserves its route handlers, and the pages go through the same checks in the simulated browser, with a small stand-in for Next's client router (benchmarks/eval/next-stack.ts). The apps use route handlers and client components with data in memory; Server Actions, and Server Components reading data directly, aren't measured. Its App JS counts all of the build's client chunks. - Full-stack tasks: the other stacks also write their own
server.ts(Nodehttp, no dependencies, data in memory); ArtScript usesapi, which also persists to SQLite. Svelte is validated without TypeScript type checking of.sveltefiles, which favors it. - Since 2026-10-01 18:47, ArtScript modification tasks get docs/SPEC-EDIT.md (~800 tokens) instead of the full spec; creation tasks keep the full spec. Earlier runs sent the full spec everywhere.
- Each run uses the ArtScript spec as of its date (in English since 2026-10-02).
- A model refusal (Opus, on the photo-upload task: twice in React, once in Vue) is not a result: it is left out, and re-run when the run is resumed.
- Task prompts (and the feedback given to the model) are in Spanish; they are the fixed dataset these numbers were measured on.
- App JS: each working app bundled with esbuild (minified, production mode) and compressed with brotli: the JavaScript a browser downloads. ArtScript's includes its runtime; React's includes React DOM; Svelte's includes its client runtime.
- Two runs per task is a signal, not a definitive benchmark: the smaller the model, the more its results move between runs (Haiku's ArtScript cells went 76, 80, 90 out of 100 as the compiler improved, and part of that is noise). Reproduce it with
npm run eval;npm run eval -- --dry-runchecks the harness and every reference app offline, and--replay <results.json>re-checks stored answers with the current compiler.