Direct testing must be specific and reproducible
“Tested” is not enough information. A reader needs to know the product version, task, environment, period, methodology and important conditions. A short test can show an observed experience; it cannot prove every claim a manufacturer makes or predict every reader’s use.
When direct testing is possible, a comparison should state what was tested, the product and version, relevant configuration, task, environment, period and observed result. It should distinguish that observation from vendor claims, disclose material access or samples, and state what was not tested. No page should imply hands-on testing where none occurred.
A useful testing note
- Exact product, version, plan, configuration and region where relevant.
- Test period and date, including whether it was a short trial or longer use.
- Task, setting, devices, connection and data conditions that affected the result.
- Who performed the test and the expertise relevant to the stated observation.
- Observed result, not an unsupported claim about all users or all circumstances.
- Material limitations, untested features, accessibility needs or workload not represented.
- Whether access, a sample, loan or payment was supplied and any relevant condition.
- What evidence was used for claims outside the direct test.
Testing should answer a decision risk
Useful tests are designed around the issue that could change a reader’s choice: a critical workflow, an accessibility control, an integration, reliability in a stated environment or a support interaction. They do not exist to produce a dramatic verdict or to simulate certainty beyond the available time and conditions.
For device decisions, the NCSC notes that support duration matters because it determines whether users receive updates and fixes. Review its device-selection guidance alongside any model-specific documentation.
Testing FAQs
Does testing one unit prove product reliability?
No. A single unit and setting can provide an observation, not a statistical reliability conclusion. The comparison should describe the test conditions and avoid claims that reach beyond them.
Can a manufacturer demonstration count as testing?
It can show how the company presents the product, but it is controlled by the presenter. Treat it as a demonstration unless the comparison independently performs and documents a relevant task.
What if testing is not possible?
The page should be clear that it uses documented evidence or another method. It can still explain a decision, but must not use language or imagery that suggests direct hands-on experience.
Read comparisons with the right context
Use these guides to understand what a comparison can and cannot tell you. A useful result makes the inclusion, evidence, uncertainty, regional limits and commercial context visible before a reader acts.
