Updated guide for the settled keyword discovery default: start with one DataForSEO discovery endpoint, turn SERP info on, keep clickstream off, then add extra endpoints only when coverage or enrichment gaps justify them.
Identify 15-30 target landing pages, then generate a 5-tier keyword matrix of roughly 100-300 seed keywords. Review the landing pages and seed set before spending API credits.
Primary outputs
landing-pages.mdseed-keywords.csvUse Keyword Suggestions first for most seed expansion work. Before running, check the client's brand context for their target market — confirm the location with them if unsure. Confirm expected returned rows and estimated cost before the billable run.
Default settings
Australia (location code 2036)include_serp_info=trueinclude_clickstream_data=falseStore the full raw API response, then check whether the first pass produced enough relevant keyword coverage. Add Keyword Ideas, Related Keywords or Keyword Overview only when there is a clear gap.
Deduplicate by normalised keyword and keep the record with the richest metric payload. A complete working row should include search volume, keyword difficulty and search intent.
Patch rules
Use a Haiku agent to semantically remove keywords that do not belong. Do not use pattern matching — evaluate the meaning and intent of each keyword. Read the brand TOV / business context guideline before applying geographic or provider rules.
14 removal categories
Use a Haiku agent to assign each remaining keyword to its best landing page based on topical relevance. If a keyword could fit more than one page, assign it to the most specific match. Use the Services Overview page as a catch-all for genuinely general terms.
Assignment examples
Within each landing page, cluster keywords into logical topical subcategories. Each page should have between 3 and 15 subcategories depending on keyword volume. Use Title Case for subcategory headers and lowercase for all keywords.
Naming rules
Air Freight Rates, Container Tracking, FCL ShippingJoin the organised keyword list back to the DataForSEO filtered output on the keyword field to attach all metrics. Calculate the opportunity score and apply priority tiers, then sort and export.
Columns in keyword-mapping-final.csv
page_name — human-readable page labelpage_url — target landing page URLsubcategory — topical group within the pagekeyword — approved keyword (lowercase)search_volume — monthly searches from keyword_info.search_volumecompetition — paid competition score 0–1 from keyword_info.competitioncompetition_level — LOW / MEDIUM / HIGH from keyword_info.competition_levelcpc — cost-per-click USD from keyword_info.cpckeyword_difficulty — organic difficulty 0–100 from keyword_properties.keyword_difficultymain_intent — informational / commercial / transactional / navigational from search_intent_info.main_intentserp_types — comma-separated SERP features from serp_info.serp_item_typesopportunity_score — search_volume × (1 − competition)priority — High / Medium / Low (see priority tiers below)Sort order
implementation_priority (from priority matrix)search_volume descending within each subcategory| Category | Remove if… | Keep if… |
|---|---|---|
| Job / career | keyword describes finding employment (jobs, salary, hiring, career) | keyword describes hiring a service provider (hire a freight forwarder) |
| Educational | keyword requests a document, course or certification (pdf, manual, training) | keyword describes what the service does in a commercial context |
| Geographic | location is outside the brand's actual service/product area, shipping region, or explicitly targeted market | location is part of the brand's serviced or targeted regions; generic international terms kept only when supported by brand context |
| Consumer (B2B clients) | keyword is clearly personal / retail (clothing, shoes, personal electronics) | keyword relates to bulk or commercial supply |
| Retail / unserved provider | keyword names a retail chain, clinic, pharmacy, or local provider where the brand does not sell or deliver its offering (e.g. yellow fever vaccine near me walgreens) | brand explicitly serves that provider or sells through that channel |
| Competitor brand | keyword uses a competitor name as a branded term (FedEx tracking, maersk container tracking) | generic service term that happens to share a competitor concept (container tracking) |
| Definition / meaning | keyword is a pure lookup (what is X, X meaning) | informational keyword that has clear commercial service intent |
| Business directory / source noise | keyword is a scraped label, navigation heading, contact detail, or opening-hours snippet (e.g. Threat Removal Phrase Variants, closes 5pm, fees apply) | never — source noise is always removed |
| Date / numeric / OCR junk | keyword has a year prefix (2023–2029), local postcode, numeric scraper ID, or OCR fragment (e.g. 0940 server upgrades, orthodontist 5034) | never — numeric/date junk is always removed |
| Incomplete fragment | keyword is a stub, reversed phrase, duplicate-word sequence, or cut-off scrape fragment (e.g. virus removal and, removal virus, managed it managed it) | never — fragments without complete, natural query shape are always removed |
Expected reduction: 60–80% for B2B service clients. The Magellan Logistics example went from 7,741 to 1,949 keywords (75% reduction). With the enhanced 14-category filter, expect 10–20% more removal of junk that previously slipped through.
# For each keyword after filtering: 1. Identify the topic What service or product does this keyword describe? 2. Match to the most specific landing page air freight rates → Air Freight Services page customs clearance nz → Customs Brokerage page logistics solutions → Services Overview (catch-all) 3. Assign a subcategory within that page "air freight rates", "air freight cost", "air cargo pricing" → subcategory: Air Freight Rates 4. Format rules Subcategory headers: Title Case Keywords: lowercase Blank line between subcategory groups
Keyword distribution guidelines
Join on the normalised keyword field. DataForSEO may return slightly different casing or punctuation — normalise both sides to lowercase with stripped punctuation before joining.
| CSV column | DataForSEO field path | Notes |
|---|---|---|
search_volume | keyword_info.search_volume | Standard Google monthly volume |
competition | keyword_info.competition | Paid competition 0–1 |
competition_level | keyword_info.competition_level | LOW / MEDIUM / HIGH |
cpc | keyword_info.cpc | USD, may be null for low-volume terms |
keyword_difficulty | keyword_properties.keyword_difficulty | 0–100 organic difficulty |
main_intent | search_intent_info.main_intent | informational / commercial / transactional / navigational |
serp_types | serp_info.serp_item_types | List joined with , e.g. organic, ai_overview, related_searches |
avg_backlinks_info is available in the raw DataForSEO response and can be null — handle safely. It is useful internally for difficulty context but is not included in the client CSV.
# keyword-mapping-final.csv (header + 3 example rows) page_name,page_url,subcategory,keyword,search_volume,competition,competition_level,cpc,keyword_difficulty,main_intent,serp_types,opportunity_score,priority Sea Freight Services,https://example.com/sea-freight/,Container Shipping,sea freight rates,2400,0.14,LOW,3.20,38,commercial,"organic, related_searches",2064,High Sea Freight Services,https://example.com/sea-freight/,Container Shipping,sea freight tracking,1000,0.09,LOW,1.10,22,informational,"organic, related_searches",910,Medium Air Freight Services,https://example.com/air-freight/,General,air freight cost per kg,1300,0.21,LOW,4.50,44,commercial,"organic, ai_overview",1027,High
The decision table is now an escalation guide. Operators should not run every discovery endpoint by default; they should run one good first pass, inspect the results, then add the smallest next step needed.
| Situation | API name | Endpoint | Docs | Why | Do not do this first |
|---|---|---|---|---|---|
| Normal seed keyword expansion | Google Keyword Suggestions Live |
POST /v3/dataforseo_labs/google/keyword_suggestions/live |
Docs | Best simple first pass for phrase variants, long-tail modifiers and usable keyword metrics. | Running Suggestions, Ideas and Related together before checking coverage. |
| First pass lacks broader topical coverage | Google Keyword Ideas Live |
POST /v3/dataforseo_labs/google/keyword_ideas/live |
Docs | Adds broader category and product/service opportunities after the tighter pass. | Using it before the seed-focused pass when scope needs to stay tight. |
| Need SERP-adjacent clusters | Google Related Keywords Live |
POST /v3/dataforseo_labs/google/related_keywords/live |
Docs | Adds Google related-search style topics when clustering or adjacent-search coverage is needed. | Using it as the default first step for all research. |
| Refreshing an approved known keyword list | Google Keyword Overview Live |
POST /v3/dataforseo_labs/google/keyword_overview/live |
Docs | Refreshes volume, CPC, difficulty and intent for known terms without rediscovery. | Discovery endpoints unless the brief asks for new opportunities. |
| Only search intent is missing | Google Search Intent Live |
POST /v3/dataforseo_labs/google/search_intent/live |
Docs | Cheapest patch endpoint for intent-only enrichment. | Keyword Overview for all rows. |
| Only keyword difficulty is missing | Google Bulk Keyword Difficulty Live |
POST /v3/dataforseo_labs/google/bulk_keyword_difficulty/live |
Docs | Direct patch endpoint for difficulty-only gaps. | Keyword Overview for all rows. |
Run with SERP info on and clickstream off. This returns the useful SEO metric set without an observed cost increase in the controlled Keyword Suggestions test.
Patch only after deduplication. Search Intent, Bulk Keyword Difficulty and Keyword Overview are additional costs, so avoid enriching discarded duplicates.
Keep clickstream off unless explicitly approved. The controlled 10-item test increased from $0.011 to $0.021.
| Endpoint or flow | Best use | Approximate cost per 1,000 |
|---|---|---|
| Keyword Suggestions | Phrase and long-tail expansion | ~$0.110 |
| Keyword Ideas | Broad topical/category expansion | ~$0.110 |
| Related Keywords | Google related-search expansion | ~$0.110 |
| Search Intent | Intent-only patching | ~$0.101 |
| Bulk Keyword Difficulty | Difficulty-only patching | ~$0.110 |
| Keyword Overview | Full enrichment for an existing list | ~$0.120 |
| Expansion + Keyword Overview | Discovery plus full fallback enrichment | ~$0.230 |
| Suggestions + Ideas + Related | Only after first-pass coverage review justifies all three | ~$0.330 if all three return 1,000 items each |
Actual cost depends on task count, returned item count and patch endpoints used. The simple default keeps the first run to one endpoint, then reports whether any extra endpoint was justified by coverage gaps.
| Checkpoint | Review before proceeding | Key decision |
|---|---|---|
| After Phase 1 | Landing pages, seed keywords and the simple first-pass setup. | Approve one-endpoint API spend. |
| After Phase 2 | Keyword counts, volume distribution, first-pass coverage, any justified extra endpoints and cost. | Approve filtering. |
| After Phase 3 | Final keyword research with keywords sorted and mapped to landing pages. | Confirm completion. |
keyword-mapping-final.xlsx) with two tabs.search_volume, competition, competition_level, cpc, keyword_difficulty, main_intent, serp_types, opportunity_score, priority.The operator-facing default is deliberately simple: run one discovery endpoint first, with SERP info on and clickstream off. Start with Keyword Suggestions for most seed expansion work, then decide whether another endpoint is actually needed.
include_serp_info=true by default — the controlled test did not increase cost.include_clickstream_data=false by default — the controlled test almost doubled cost.# 3-phase workflow summary Phase 1 Site Analysis & Seed Keywords ↓ checkpoint: review pages and seeds Phase 2 DataForSEO Research ↓ checkpoint: review first-pass keywords, coverage and cost Phase 3 Filtering, Organisation & Landing Page Mapping ↓ complete: keyword research sorted and mapped to landing pages