By Mehdi HassanHow should an AI company show benchmarks on its website?
An AI company should show benchmarks with enough context that a skeptical reader could check them: which benchmark, which version, when it was run, with what settings and against which exact competitor model. A bare "state of the art" badge does the opposite. It invites doubt and, in some cases, regulatory attention.
This guide covers what to publish, how to compare fairly, and how to avoid the overclaiming that regulators now call AI-washing. It's for teams adding a benchmark or evals page to their site, or putting performance claims on the homepage. For how this fits the rest of an AI site, see our hub on AI startup website design.
What is AI-washing, and why does it matter for benchmark pages?
AI-washing is overstating what AI does in a product, or claiming AI where there's little or none. It matters for benchmark pages because performance numbers are the most specific, most quotable claims on an AI site, which makes them the easiest to challenge.
US regulators have acted on this. The FTC has brought cases over deceptive AI claims, including its $1M order against accessiBe, finalized in April 2025, over claims about what its automated tool could do. The SEC has charged investment advisers for misleading statements about their use of AI. Neither agency needs a special AI law: ordinary rules on deceptive advertising apply. Check specific claims with counsel.
The same discipline applies in other claim-heavy categories. Our guides on defensible climate claims and wellness and health claims cover how regulators treat unsupported marketing claims there.
What should an AI benchmark page include?
An AI benchmark page should include the results, the method behind them and the limits, in that order. Readers skim the results, experts check the method, and buyers who'll rely on the product read the limits.
- A plain summary: two or three sentences on what you tested and what you found, with the date.
- Results table: each benchmark by name and version, your score, competitor scores and the model versions compared.
- Settings: prompting method, number of attempts, temperature or equivalent, and any tools or retrieval used.
- Your own task evals: results on tests built from your users' real work, with how the test set was built.
- Where you do worse: tasks or benchmarks where you lose or tie, stated as plainly as the wins.
- Method details: test set size, scoring method, who graded and how ties were handled.
- Change log: when each result was run and what changed since.
What to put in the method section
A useful method section answers the questions a technical reviewer would ask before trusting a number.
What exactly was tested, and which version of the product or model? What data was used, and could any of it have been in training data? How many examples, and how were they chosen? How was output scored: exact match, a rubric, human graders or a model grading another model? If humans graded, were they blind to which system produced each output? Were all systems run with the same prompts and settings, or each with its best-known settings?
If any answer is uncomfortable, say so. A disclosed weakness costs you less trust than one a reader discovers.
Should you use public AI benchmarks or your own evals?
Use both, and say which is which. Public benchmarks show general capability and let readers compare across vendors. Your own evals, built from the tasks your buyers actually do, are more relevant to a purchase decision but harder for outsiders to verify.
Eval type | Strength | Weakness | How to present it |
|---|---|---|---|
Public benchmark | Comparable across vendors, widely known | May not reflect your users' tasks; can be saturated or leaked into training data | Name, version, settings and date |
Your own task eval | Matches real buyer work | Hard to verify; you built the test | Describe how the test set was built and graded |
Customer-run eval | Strongest proof for similar buyers | Needs permission; narrow scope | Attribute with consent, state the task |
Head-to-head human preference | Shows output quality people notice | Depends on graders and prompts | Show grader setup and sample outputs |
For most products sold to business buyers, the task eval and real examples persuade more than a leaderboard. A buyer choosing a contract-review tool cares about clause detection on contracts like theirs, not a general reasoning score.
How do you compare your AI against competitors fairly?
Compare fairly by using the same tasks, prompts and settings for every system, naming the exact competitor version and date, and saying so if you couldn't test their latest release. Comparisons that pit your newest model against a competitor's old one are the fastest way to lose credibility with technical buyers.
Fix the test before you run it
Choose tasks and scoring in advance, so results don't shape the test.
Use the same conditions
Same prompts, attempts and tool access for every system, or document each difference.
Name versions and dates
Competitor model names change often; a result without a version is meaningless within months.
Show the whole table
Publish every benchmark you ran, not only the ones you won.
Re-run or retire
When either side ships a new model, re-run the comparison or mark the old one as outdated.
Where should benchmarks appear on an AI website?
Benchmarks should live on a dedicated page, with only the strongest, best-supported result summarised on the homepage and a link to the method. A homepage badge with no link to evidence is the pattern most likely to read as AI-washing.
On product and use case pages, real output examples usually do more than scores. Show a before-and-after, then link to the benchmark page for readers who want numbers. Our AI landing page design guide covers how to place proof near the hero, and our AI company website examples show sites that lead with output rather than leaderboards. Data and training questions belong on the AI trust page, not mixed into performance claims.
Write the benchmark page so each section opens with a direct answer. AI answer engines quote pages that state findings plainly, and a well-structured evals page is often what gets cited when buyers ask a chatbot how vendors compare. See answer-first page structure.
How often should an AI benchmark page be updated?
Update an AI benchmark page every time you or a compared competitor ships a model that would change the result, and review it at least quarterly. AI results go stale faster than any other content on your site, and an outdated win is a false claim.
Put a visible "Last run" date on every table. Keep old results in a dated archive rather than deleting them, so readers can see the trend and you have a record of what you claimed and when. Store benchmark names, versions and scores as CMS fields so updates don't require a developer.
