Keyword research review: the two-source scoring rule

What to take away

Fix the brief and the evidence deadline first

Start by writing down the decision the review must support. A team choosing a tool for one England campaign has a different brief from one standardising a group-wide content programme. The brief fixes the criteria, the candidate count and the evidence deadline.

The method fits any tool or agency and names no product, because scores come from the evidence you collect. Keep the scope narrow enough to finish. Three to five candidates is workable; ten is a research project nobody reads. If your list is longer, use the keyword research selection checklist to cut it before scoring starts.

The two-source rule

Each criterion needs two independent sources before it scores above 2, on whichever scale your rubric uses. A source is independent when the seller does not control it. Vendor benchmarks are marketing until checked, so a seller's own page counts as a claim rather than as evidence.

Published comparisons and public procurement standards are sources you can cite. A trial dataset you control is a source too. Score only what documentation or that dataset can verify.

Take coverage of the England market. The vendor's page is a claim, so the criterion stays at 2. A published comparison and a trial run on your own England search terms are two independent sources, and the score may rise. Both go into the write-up, named and dated.

The do-not-assess list

Write down what you will not score, so a persuasive demo cannot pull the weighting off course. These things may influence buying but are not evidence of fit:

Four things remain to assess: data provenance, coverage of the England market you sell into, contract terms and reporting. Weighting set late drifts towards the loudest pitch, so fix the weights before you see any vendor. Keep the list in the review file and point to it when a stakeholder asks why a well-liked feature earned no points.

Contract and reporting evidence

Contract terms are scored from the document itself: seat limits, export rights, notice periods and price escalation. A sales assurance that the terms are flexible scores nothing until it is in writing.

Treat missing evidence as a finding. If a vendor cannot show notice periods or export rights on paper, score the criterion at 2 or below and note what you asked for and when. The contract checks for research suppliers show what to look for before signing.

Reporting is scored by trial: can you extract the evidence stakeholders need without manual work? Ask where search volumes, click estimates and difficulty scores come from, using the guide to tracing data lineage before you trust a report.

Public bodies publish useful models for supplier management. The Public Contracts Regulations 2015 and the government's contract management professional standards describe performance monitoring and supplier management expectations that translate well to agency retainers. Government guidance for new in-role contract managers says the role includes ensuring suppliers conform to contractual terms, service level agreements and key performance indicators.

Government Functional Standard GovS 008: Commercial adds that supplier performance monitored should show whether intended outcomes and benefits are realised. Those ideas translate well to an agency retainer.

Record limits and sign-off

Every review has limits. Name them: no hands-on testing of the full platform, no access to the vendor's raw index, no long-run accuracy check. Stating limits protects the review when a tool underperforms on a criterion nobody tested.

One named owner signs the scoring sheet, and that person is accountable for the content plan, not the one most enthusiastic about the tool. Their signature covers the scores, the claims marked as claims and the limits.

The method should be reusable. Run the same rubric next year and the comparison becomes a trend rather than a fresh argument. The wider guide to reviews and comparison methods helps when you set criteria for the first time.

Common questions

What if only the vendor supports a criterion?

Score it no higher than 2 and mark it as a claim in the write-up. Where a criterion rests only on a vendor's word, the reader needs to see that the evidence came from the seller.

Who should own the review?

The person accountable for the content plan, not the person most enthusiastic about the tool. They sign the rubric and the scores, and they own the list of limits.

How often should the rubric change?

Review the weights annually, before you look at candidates. Changing weights after seeing scores turns the method into a justification exercise.

What counts as a limit of the review?

Anything you could not test: hands-on use of the full platform, access to the vendor's raw index or a long-run accuracy check. Naming each one protects the review when a tool underperforms on a criterion nobody tested, and it tells the reader how far the scores can be trusted.