How we test
A review is only as good as its method. Here is ours, in full, so you can hold us to it.
The testing process
- We pay. We buy the plan a real user would buy, with our own money, using the same signup flow.
- We use it like you would. We run a fixed set of realistic tasks: starting a new project, editing existing code, refactoring, debugging, and working inside a large existing codebase.
- We score it. Every coding assistant is scored on the same rubric below, 100 points total, so tools are directly comparable.
- We write the verdict first. Scores and verdicts are finalized before any commercial discussion with a vendor.
- We show our work. Each report lists what was hands-on tested and what comes from public documentation, so you can tell the difference.
The rubric
Every coding assistant report uses these six criteria and weights, for a total of 100 points.
| Criterion | Weight | What it measures |
|---|---|---|
| Code quality | 25 pts | Correctness, idiom, and maintainability of the code the tool produces. |
| Agentic coding | 20 pts | How well it plans and executes multi-step tasks: reading, editing, running, iterating. |
| Codebase understanding | 15 pts | Context of large or existing codebases, indexing, memory of conventions. |
| Speed and reliability | 10 pts | Latency, uptime, and whether usage limits interrupt real work. |
| Value for money | 15 pts | What the price buys relative to competitors, including how billing actually behaves. |
| Privacy and data controls | 15 pts | Training opt-outs, zero-retention options, SSO, audit controls. |
Honest-method policy
What "tested" means here. When we say we tested something, we ran it ourselves and saw the result. When we did not run it, we label the claim "based on public documentation and pricing as of October 1, 2026" and treat it as the vendor's word, not ours.
We never invent benchmark numbers, performance figures, or quotes. If a number appears in a report, it comes from a run we did, a page we checked on the stated date, or a clearly labeled third-party source.
Pricing is checked on the stated date and cited with its source. Vendors change pricing often, especially usage-based plans, so treat any pricing section as a snapshot and verify before buying.
Affiliate disclosure policy
Affiliate disclosure
The Lab Report is supported by affiliate commissions. If you buy a tool after clicking a link on this site, we may earn a commission at no extra cost to you. Rankings and scores are never for sale: vendors cannot pay for better placement or better scores, and our verdicts are written before any commercial discussion. Commission income never influences what we test or what we recommend.
Until affiliate programs are approved, tool links point to official homepages only. Disclosures appear on every report page, on the homepage, and in our machine-readable data (JSON-LD, llms.txt, agent manifest, reports.json).
Corrections policy
If we get something wrong, we fix it and say so. Corrections appear at the top of the affected report with the date and what changed. Readers can report errors by contacting us through the information in our agent manifest, which identifies this publication. We re-check pricing on every report at least quarterly.