Before you plan content, the keyword research rules and ethics to follow

What to take away

Keyword research rarely fails because the tool was wrong. It fails because a team exported a list, pasted it into a brief, and never asked whether the underlying data was lawfully obtained or how the resulting content would be described. That is the mistake to avoid first.

This guide sets out the rules and ethical expectations that apply to keyword research in England, with notes where the position differs in Scotland, Wales or Northern Ireland. It is general guidance only. Individual cases need a qualified adviser, particularly where personal data, regulated products or contracts are involved.

Where the rules come from

Keyword research sits at the intersection of data protection, consumer protection, advertising regulation and contract law. No single statute is called "keyword research law". Instead, several regimes apply depending on what you collect, how you collect it and what you do with the output.

In England and the rest of the UK, the Data Protection Act 2018 supplements the UK GDPR and sets out how personal data must be handled. That matters when your research involves customer queries, support tickets, CRM exports or survey responses rather than anonymous search volume.

The ICO regulates data protection across the UK. Its guidance on choosing your lawful basis for direct marketing explains that you must identify a lawful basis before you process personal data, and that consent, legitimate interests and contractual necessity have different consequences.

Advertising content is regulated by the Advertising Standards Authority through the CAP Code. Equality duties come from the Equality Act 2010 guidance published by GOV.UK, which applies across England, Scotland and Wales, with some differences in Northern Ireland.

Crawling rules sit with the platform. Google's documentation on how it interprets the robots.txt specification explains that robots.txt is a standard for controlling automated access, and that directives are advisory rather than a security mechanism.

Personal data in keyword research

Search volume figures are aggregated and normally anonymous. The risk appears when you enrich them with personal data: a list of customer searches tied to email addresses, a spreadsheet of competitor ad copy with named contacts, or a CRM export showing which phrases led to a purchase.

Under the UK GDPR, personal data means information relating to an identified or identifiable person. If a keyword list can be linked back to an individual, it is personal data, and you need a lawful basis.

The ICO's direct marketing guidance is clear that legitimate interests is not a free pass. You must balance your interests against the individual's rights, and you must tell people what you are doing in a privacy notice. For cold outreach based on research, consent or soft opt-in rules may apply.

A common failing is to assume that because a tool aggregated the data, the output is anonymous. Aggregation is not anonymisation. If re-identification is reasonably possible, treat the dataset as personal.

Our keyword research UK regulations in England article walks through the specific statutory tests, and is worth reading before you build any list that includes identifiable people.

Disclosure and transparency

Disclosure has two sides. The first is telling people what you do with their data. The second is telling readers how your content was produced, particularly where affiliate links, sponsored placements or AI assistance are involved.

If your keyword research feeds a page that earns commission, the CAP Code requires that commercial intent is identifiable. A reader should not have to guess that a recommendation is paid for. The same principle applies to sponsored content and to links that pass value.

Transparency also covers research methods. If you publish a "best tools" list and your ranking is based on affiliate payouts rather than testing, saying so is both an ethical and a regulatory expectation. The ASA has repeatedly ruled against ads that were not obviously identifiable as such.

For a fuller treatment of what a public policy should contain, see our guide to a keyword research disclosure policy in England. It covers privacy notices, affiliate labels and internal sign-off.

Advertising rules and claims

The CAP Code applies to ads in paid and some organic contexts. Keyword research often generates the claims that later appear in ads, so the two activities are linked.

If your research shows that a phrase like "cheapest" or "fastest" has high volume, that does not make the claim safe to use. Comparative claims must be verifiable and not misleading. Superlatives need evidence you can produce.

Regulated sectors add another layer. Financial promotions must be approved by an authorised person unless an exemption applies, and health claims face separate restrictions. Keyword lists can surface demand for claims you cannot lawfully make.

Our keyword research advertising rules in England article sets out how the CAP Code interacts with search-led content, including the treatment of price claims and testimonials.

Equality law also matters. The Equality Act 2010 prohibits discrimination in the provision of services and in advertising. Targeting keywords that exclude protected groups, or writing copy that stereotypes them, can breach those duties even if the search volume looks attractive.

Supplier contracts and data ownership

Most keyword research relies on third-party tools. That creates a contractual relationship, and the terms determine what you may do with the output.

A supplier's privacy policy is part of the picture. For example, Ahrefs' privacy policy describes what data the company collects and how it is used, which is relevant if your research inputs include client information. Reading it before you upload anything is basic due diligence.

Contracts should also address audit rights and breach notification. If the supplier suffers an incident, you may have your own notification duties to clients and regulators.

Our keyword research commercial contracts in England article covers the clauses that matter, including data processing agreements and liability caps. It is aimed at buyers rather than lawyers, but it flags where legal review is essential.

Crawling, scraping and robots.txt

Automated collection is a distinct activity from using a research tool. If you write a script to crawl a site, robots.txt is the first thing to check.

Google's documentation is explicit that robots.txt is not a security mechanism. It is a convention that well-behaved crawlers follow. Ignoring it may not be illegal in itself, but it can breach contract terms, and it may breach computer misuse law if access is unauthorised.

The practical rule is simple: respect robots.txt, rate-limit your requests, and identify your crawler. If a site's terms prohibit scraping, do not assume that a public page makes it acceptable.

There is also an ethical dimension. Scraping a competitor's site to lift their keyword strategy may be lawful in narrow circumstances, but it is rarely a good look, and it can expose you to breach of confidence claims if the data was not public.

A worked example

Consider a small ecommerce team in England selling garden furniture. They want to build a content plan for spring 2027 and decide to enrich their keyword list with customer data.

They export 12,000 support tickets containing names, email addresses and search phrases. They pay a contractor £600 to clean the data, and a tool subscription costs £150 a month for six months, so £900. Total research spend: £1,500.

If they process the ticket data without a lawful basis, the ICO can issue a fine. Under the UK GDPR, penalties can reach £17.5 million or 4 per cent of annual worldwide turnover, whichever is higher. That figure comes from the ICO's penalty framework, and it is a maximum, not a typical outcome.

The cheaper route is to anonymise the tickets before analysis, or to rely on legitimate interests with a documented balancing test and a privacy notice that mentions research. The savings are not just financial: the team avoids a complaint, a subject access request and a reputational hit.

Our keyword research tools and supplier guide for 2027 covers how to choose suppliers that support this kind of compliance work.

A pre-project compliance checklist

Before you commission research or brief a writer, run through these checks.

This is not legal advice. It is a practical filter that helps you spot the questions a qualified adviser will ask.

Start with a data map. List every input to your research, mark which are personal, and write down the lawful basis for each. Keep a record of those decisions, because accountability is a legal requirement rather than a nicety. Reviewing supplier contracts and public disclosures now is easier than fixing them after a complaint.

Common questions

Who enforces keyword research rules in England?

The ICO enforces data protection law across the UK, including England. The ASA enforces the CAP Code for advertising. The CMA can act on consumer protection breaches. Equality duties are enforced through the courts and tribunals, and the Equality and Human Rights Commission has powers in England, Scotland and Wales.

Does keyword research count as processing personal data?

It can. If your list can be linked to an identifiable person, it is personal data. Aggregated search volume is usually anonymous, but CRM exports, ticket data and named contact lists are not. The test is identifiability, not the format of the file.

Can I scrape competitor sites for keyword ideas?

Only if you respect robots.txt, the site's terms and computer misuse law. Scraping public data is not automatically lawful, and lifting non-public data can breach confidence. Most teams are better served by licensed tools with clear terms.

Do I need consent to use customer queries for research?

Not always. Consent is one lawful basis, but legitimate interests or contractual necessity may apply depending on the purpose. You must still be transparent, and you must honour individual rights such as objection and erasure.