AI Has an IQ of 151. What Have We Actually Measured?

The cover is an AI illustration of the transition from a completed test to a new problem. This article analyzes public measurements and our experience; it is not a new IQ experiment. At the end you will find eight images: original screenshots, data charts and explanatory diagrams.

The number 151 looks convincing. When it appears beside the name of an artificial intelligence, it is easy to imagine a machine genius: excellent at mathematics, able to understand complex situations and probably capable of solving the problems that defeat the rest of us.

After our experience with four AIs on a virtual island, however, I am interested in another question: will this genius notice that it is standing in a colleague's way?

On our island, the agents needed to build steps, coordinate and actually climb to the top. This apparently simple task exposed difficulties with coordination, memory and checking their own actions. Our changes to the environment also helped in the end. The experiment did not measure IQ. It did remind me how many different abilities we hide behind the single word “intelligence.” I told the full story in I Let Four AIs Loose on an Island. The Hardest Part Was Staying Out of Their Way.

So I looked into what lies behind AI IQ leaderboards.

One question, four different tests

TrackingAI, created by Maxim Lott, compares models on two sets of tasks: the public Mensa Norway test and an “Offline Test” whose questions are not released. Each set can be presented as images or text.

The distinction matters. Imagine looking for a rule in a grid of differently rotated shapes. With an image, you must first correctly identify what is where. A text description has already done some of that work for you. At the same time, it may introduce ambiguity or overwhelm you with detail.

Changing how a task is presented also changes its demands. A difference between the image and text versions may help reveal a perception problem, but it does not perfectly separate “vision” from “logic.”

When I checked the public records on October 25, 2026, the following selection came from near the top of the rankings:

Our calculation from TrackingAI's public records, using its published conversions and at most the seven most recent runs for each entry. These are ongoing records under the site's labels, not our own simultaneous experiment with fixed model versions. The data was checked on October 25; individual runs come from different days.

Even this small table shows why the last column matters. Fable has a slightly higher text mean than Opus, but only two runs support it. Declaring a winner in intelligence on a difference of two tenths of a point would be bold.

TrackingAI also averages up to seven recent tests, not the last seven days. Both tests have ceilings: 151 for the public test and 136 for the nonpublic one. As models approach those ceilings, the tests become less effective at distinguishing between them. See TrackingAI's methodology.

How an IQ of 151 is produced

In the published Mensa Norway conversion, 35 correct answers out of 35 correspond to a score of 151. One answer is worth approximately three points. The number therefore comes from converting performance on particular tasks. Mensa Norway's online test itself states a range of 85–145 and notes that it does not replace a professional intelligence assessment. See the author's conversion formula and Mensa Norway.

Solving all 35 problems is a notable achievement. Transferring that result to the human IQ scale, however, requires more than the same number of correct answers. Conditions, presentation and the population used to establish the scale all matter.

I must apply the same standard to my mini IQ test from April. A single puzzle can illustrate how a model handles ambiguity and whether it questions its own hypothesis. But the “IQ impression” I described then remains an impression. One task does not establish an actual IQ number.

Then there is the question almost everyone asks: what if the AI already knows the answers?

That is a real possibility with a public test. Its tasks, analyses or similar examples may have appeared in training data. Nonpublic questions reduce this risk, but do not eliminate familiarity with general procedures and similar patterns.

A private test does not automatically acquire the quality of a large psychometric study, either. In his original description, Lott reports 16 created questions, of which 14 were ultimately scored, and calibration based on 35 usable responses from volunteers. That is a useful exploratory experiment whose precision has limits. See the original Offline Test methodology.

Pretraining and reasoning are not mutually exclusive. Humans learn from examples, too. The important question is whether a learned principle can then be applied in a new situation.

The study On Memorization of Large Language Models in Logical Reasoning examined this relationship using generated logic puzzles. After fine-tuning, models could memorize particular problems and be surprisingly fragile when those problems changed. At the same time, they also improved on new tasks. Memorization and the acquisition of transferable procedures can therefore happen together.

For me, that suggests a simple requirement: do not stop testing after the correct answer. Change the names, shuffle the answer choices and create a new example of the same rule. Then see how much of the original success survives.

A model also needs working conditions

An even bigger surprise comes when a model receives memory and the ability to act.

In September 2026, the ARC Prize organizers published GPT-6 Astra results on ARC-AGI-3. In this environment, an agent explores unfamiliar games, discovers their rules and plans its actions. At the same High reasoning setting, it scored 54.8% in the standard system and 99.9% with a “Provider Adapter.”

The difference was in how it operated: the standard system had the model carry forward its own visible notes, while the adapter preserved internal reasoning state between calls and managed long context. This is a benchmark score that also accounts for action efficiency, not a success rate for arbitrary work. Details are in the official ARC Prize analysis.

When we evaluate AI, we often measure a model together with the system that enables it to work. What it can retain, the information it receives and how it checks results all matter.

Our island offered a smaller, practical illustration. Regular chat reading, checking repeated errors and changes to the steps helped. But we changed conditions throughout the experiment and altered several things at once. We therefore cannot isolate each change's contribution or attribute the entire success to the models alone.

Such restraint is often missing from leaderboards. A team led by the Oxford Internet Institute examined 445 language-model benchmarks. Only 16% of the reviewed papers used statistical methods to compare performance. Around half of the benchmarks also failed to define clearly enough the abstract abilities they intended to measure. See Oxford's study summary.

A number with a decimal point can look more precise than the quality of the experiment warrants.

How we assess model capabilities in Hyperprostor

That is why, in Hyperprostor, we work with models from many providers and assess their capabilities using dozens of benchmarks and criteria. We want to know which work each model suits, how reliably it handles that work and how much time and money it costs.

One reference point is the Artificial Analysis Intelligence Index, labeled AA Index v4.3 in our overview. It is a composite score from ten evaluations covering areas such as programming, scientific reasoning, document work and agentic tasks. Its components include SciCode, Humanity's Last Exam and Terminal-Bench. Artificial Analysis conducts these evaluations; we use its results to compare models. See the index's composition and methodology. Comparisons also need to account for the methodology version and model configuration.

Even this broader index does not cover everything we need to know. We therefore look at several different aspects:

• Writing quality: style, clarity, creativity and adherence to the request. Elo-style rankings based on pairwise output comparisons can also help.
• Programming: writing and fixing code, creating an HTML page or a React application. A useful result must also work and meet the request.
• Mathematics and science: correct calculations, logical procedures and solutions to specialist problems.
• Agentic work: tool use, multistep planning, long-context handling and task completion.
• Speed and latency: how long we wait for a response to begin and how quickly it then appears.
• Cost: input and output token prices and how usage translates into the cost of actual work.

With Elo, it matters who judges the outputs. Arena, for example, uses human preferences, while EQ-Bench's Creative Writing v3 evaluates texts using another language model. The score expresses relative performance within a particular comparison. A high writing Elo does not, by itself, guarantee factual accuracy. See Arena Creative Writing and EQ-Bench Creative Writing.

The gallery below includes our price–performance overview. Its horizontal axis shows a comparison price per million tokens, its vertical axis AA Index v4.3, and point size indicates speed. The green line marks the Pareto frontier: the boundary of the most favorable combinations of price and score in the displayed selection. It is a snapshot, not a permanent ranking of models.

This kind of chart supports a more practical question: How much extra am I paying for a better result, and do I need that difference for this task?

For quickly sorting thousands of short messages, cost and speed at adequate accuracy may be decisive. Complex programming may justify a more expensive model that saves several rounds of fixes. When writing an article, we may value a feel for language, good structure and the ability to maintain the author's voice.

Even a cheap million tokens can produce an expensive result if the model uses too many or the work needs repeated corrections. Tracking cost per successfully completed task therefore makes sense, too.

Together, these perspectives give us a map of model strengths and weaknesses in Hyperprostor. An IQ test is one piece of that map. Combined with other results and practical experience, it helps us choose the right assistant for a particular job.

What I would test next

How would I test an AI I actually intend to use?

I would give it previously unreleased tasks, prepare several variants with the same logic and test both text and images. I would let it respond to a correction and check whether it repeated the same mistake next time. Runs would be repeated with recorded settings and comparable budgets. Alongside success, I would track cost, time and the amount of human help.

I would also evaluate the model on its own separately from an agent equipped with memory and tools. This is a proposal for another experiment; we conducted no new measurement of this kind for this article.

The result would be a capability profile that helps decide which work to assign to whom. A model excellent at visual puzzles need not be best at prolonged investigation across documents. An agent with good notes may finish a task that a more capable model without suitable memory repeatedly loses sight of.

High AI scores deserve attention. They show progress on particular problems. But I am most interested in what happens after the familiar task ends: new conditions arrive, the first error occurs, information is missing or a colleague is standing in the wrong place.

That is where we see how well the system can cope.

Next time you see “AI has an IQ of 151,” ask: What problem will that help it solve — and how much assistance will it need?

Image gallery: from a test to a useful model

Eight views of what the results mean. Actual screenshots are unchanged and retain their original Czech interfaces. The charts are our own presentations of the cited data; the diagrams explain methodology and do not claim to show additional measurements.

1. Hyperprostor: price, performance and speed in one view. An original screenshot of our interface. The green Pareto frontier helps identify favorable price–score combinations. The chart uses a blended price with a 3:1 ratio. Point size adds information about speed. The values reflect the time of the screenshot.

2. Four testing situations. Our diagram based on TrackingAI's methodology. Text and visual presentations make different demands on a model. “Offline” refers to an unreleased question set, not a model running without an internet connection.

3. Scores need run counts. Our calculation from the public records, checked on October 25, 2026. Each panel shows a separate test variant; n is the number of runs. The chart does not show confidence intervals, and small differences do not prove superiority. These are selected results under the site's ongoing model labels.

4. Where 151 comes from. A mathematical plot of the author's conversion formula, not a curve of increasing intelligence. Values are rounded. Mensa Norway's online test itself states a range of 85–145.

5. Model performance also depends on context management. A comparison from the ARC Prize report of September 3, 2026. Both values use High and the same test set. The difference is 45.1 percentage points. The score also accounts for action efficiency; it is neither a universal work success rate nor proof of AGI.

6. A model has a capability profile. Our overview of the criteria we care about in Hyperprostor. This is an explanatory diagram without scores for particular models. Public benchmarks and experience from actual use complement each other.

7. What does not fit in an IQ table. An original screenshot from our October 24 experiment. Kimi's climb was verified through the character's movement. We had reinforced the steps and added an automated test, so the success belongs both to the models and to our environmental changes. This image is not evidence of their IQ.

8. What the next experiment could look like. Our methodological proposal. Each stage should keep conditions comparable and record the assistance the system received. This is a proposal, not the results of an experiment already performed.

Sources appear beside individual claims and images. A machine-readable record of image and data provenance is also available, along with the translation record. Source check: October 25, 2026.