Model comparison
Inkling vs o1
Inkling is the stronger model overall, scoring 44.1 to 40.9 on the Noometry Index.
Last verified . 28 shared benchmarks.
Summary
- They share 28 benchmarks with published results for both. Inkling scores higher in 6 categories and o1 in 3 categories; 8 gaps are clear of the uncertainty.
- The widest gap is in knowledge, where Inkling leads 55.1 to 41.5.
- The biggest single-benchmark swing is ARC-AGI-1: 79.5% for Inkling and 30.7% for o1.
- Inkling is cheaper at $1.87 / $4.68 per million input/output tokens, against $15 / $60 for o1.
- o1 accepts more context: 200K tokens versus 66K.
- Inkling has downloadable open weights; the other is API-only.
Side by side
| Inkling | o1 | |
|---|---|---|
| Provider | Thinking Machines Lab | OpenAI |
| Noometry Index | 44.1 | 40.9 |
| Released | 2026-07-15 | 2024-09-12 |
| Weights | Open | Proprietary |
| Context window | 66K | 200K |
| Max output | 66K | 100K |
| Input $ / M tokens | $1.87 | $15 |
| Output $ / M tokens | $4.68 | $60 |
| Results tracked | 41 | 52 |
Sponsored placements are available on pages like this one. Advertise on Noometry
Category by category
Coding o1 leads
Inkling: 34.5 (#234), o1: 46.1 (#70)
| Benchmark | Inkling | o1 |
|---|---|---|
| WeirdML | 32.3% | 47.6% |
| LMArena Coding | 1464 | 1367 |
| FrontierCode | 14% | — |
| Aider Polyglot | — | 61.7% |
| LMArena WebDev | 1413 | — |
| FrontierSWE | 4.1% | — |
| SciCode | 47% | — |
| LiveBench Coding | — | 69.7% |
| CadEval | — | 56% |
| ALE-Bench | 946 | — |
| HumanEval+ | — | 89% |
| MBPP+ | — | 80.2% |
Agentic & Tool Use Inkling leads
Inkling: 29.6 (#85), o1: 24.6 (#117)
| Benchmark | Inkling | o1 |
|---|---|---|
| APEX-Agents | 33.8% | — |
| τ²-bench Banking | 25% | — |
| Cybench | — | 10% |
| METR Time Horizons | — | 51.1% |
Reasoning Inkling leads
Inkling: 40.4 (#56), o1: 27.9 (#111)
| Benchmark | Inkling | o1 |
|---|---|---|
| SimpleBench | 50% | 41.7% |
| ARC-AGI-1 | 79.5% | 30.7% |
| Chess Puzzles | 21% | 15% |
| LMArena Hard Prompts | 1451 | 1371 |
| DTBench | 87.5% | 74.7% |
| LMCA | 37.6% | 22.3% |
| Epoch Capabilities Index | 148.54 | 141.91 |
| ARC-AGI-2 | 36.5% | — |
| CritPt | 5.4% | — |
| EnigmaEval | — | 5.7% |
| LiveBench Reasoning | — | 91.6% |
| LiveBench Data Analysis | — | 65.5% |
| LiveBench | — | 75.7% |
Math o1 leads
Inkling: 31.3 (#225), o1: 36.1 (#175)
| Benchmark | Inkling | o1 |
|---|---|---|
| FrontierMath (Tiers 1-3) | 33.3% | 14.7% |
| OTIS Mock AIME 2024-2025 | 88.9% | 73.3% |
| LMArena Math | 1479 | 1388 |
| FrontierMath Tier 4 | 4.9% | — |
| ProofBench | 0% | — |
| LiveBench Math | — | 80.3% |
| MATH Level 5 | — | 94.7% |
| FrontierMath (Feb 2025 set) | — | 9.3% |
Knowledge Inkling leads
Inkling: 55.1 (#49), o1: 41.5 (#110)
| Benchmark | Inkling | o1 |
|---|---|---|
| GPQA Diamond | 88.3% | 76.8% |
| SimpleQA Verified | 40.3% | 41.1% |
| LMArena Expert | 1465 | 1361 |
| Humanity's Last Exam | — | 8% |
| Confabulations | — | 11.7% |
Multimodal Not comparable
Inkling: —, o1: 34.2 (#93)
| Benchmark | Inkling | o1 |
|---|---|---|
| LMArena Vision | — | 1168 |
| GeoBench | — | 80% |
| VPCT | — | 37% |
| SpatialViz-Bench | — | 41.4% |
Multilingual Inkling leads
Inkling: 54.0 (#52), o1: 48.6 (#142)
| Benchmark | Inkling | o1 |
|---|---|---|
| LMArena Non-English | 1434 | 1358 |
| LMArena Chinese | 1490 | 1394 |
| LMArena French | 1458 | 1344 |
| LMArena German | 1446 | 1337 |
| LMArena Japanese | 1429 | 1346 |
| LMArena Korean | 1404 | 1396 |
| LMArena Russian | 1429 | 1356 |
| LMArena Spanish | 1448 | 1345 |
Instruction Following Too close to call
Inkling: 75.1 (#71), o1: 74.8 (#86)
| Benchmark | Inkling | o1 |
|---|---|---|
| LMArena Instruction Following | 1426 | 1367 |
| LiveBench Instruction Following | — | 81.5% |
Long Context o1 leads
Inkling: 43.8 (#86), o1: 50.3 (#9)
| Benchmark | Inkling | o1 |
|---|---|---|
| LMArena Longer Query | 1434 | 1378 |
| Fiction.LiveBench | — | 83.3% |
Writing & Preference Inkling leads
Inkling: 65.2 (#51), o1: 55.6 (#144)
| Benchmark | Inkling | o1 |
|---|---|---|
| LMArena Text | 1441 | 1366 |
| LMArena Creative Writing | 1387 | 1348 |
| LMArena Multi-Turn | 1436 | 1369 |
| Short-Story Creative Writing | — | 70.2% |
| EQ-Bench Creative Writing | 1611 | — |
| EQ-Bench 4 | 1226 | — |
| LiveBench Language | — | 65.4% |
Frequently asked questions
Is Inkling better than o1?
Inkling is the stronger model overall, scoring 44.1 to 40.9 on the Noometry Index.
Which is cheaper, Inkling or o1?
Inkling is cheaper. It lists at $1.87 per million input tokens and $4.68 per million output tokens; o1 lists at $15 and $60.
Is Inkling or o1 better for coding?
o1 scores higher on coding benchmarks: 46.1 versus 34.5 in the Noometry coding category.
Which has the bigger context window?
o1 does, with 200K tokens against 66K.
How many benchmarks do Inkling and o1 share?
28 benchmarks have published results for both models. Inkling has 41 scored results on Noometry and o1 has 52.