Tools

See where AI answers stop naming your brand

I built these tools to measure what happens between a buyer's question and an AI answer. They keep brand mentions, cited sources and gaps at each buying stage on separate lines.

One answer cited Skill Trade. Two did not.

In a three-question OpenAI API sample, Skill Trade appeared in the answer asking for a tool. It was absent from the two broader answers.

Observed 10 September 2026 through OpenRouter, requesting openai/gpt-5.2:online with native search. One fresh request per question, written before seeing the answers. The returned model was openai/gpt-5.2.

Problem: no mention or citation: “How can a B2B marketing team tell whether AI answers name its brand?”
Category: no mention or citation: “What kinds of tools can a B2B marketing team use to track citations in AI answers?”
Decision: named and cited: “Which free, open-source tools can a B2B marketing team use to track whether AI answers cite its website?”

Read the cited answer excerpt

The answer listed SkillTrade citation_meter.py and described it this way:

“Runs a fixed question set across assistants, extracts cited URLs/domains, and trends results”

The answer linked to this Tools page. The response predates this page revision; it cannot show that the revision improved visibility. The complete generated answer and its citation annotation are in the download below.

See the missing coverage

The same three questions were attempted on four models through OpenRouter: 12 attempts, three scorable answers. Perplexity and Gemini requests returned HTTP 404 on the selected native-search route. Anthropic returned answers with citation positions of 0–0, which the parser could not map reliably. Those nine attempts are unavailable, not zero citations. All 12 attempts remain in the downloadable record.

The first two OpenAI answers contained no URL annotations at all. This records visible absence; it does not establish which pages were searched. One question per stage cannot establish a general funnel pattern, a market-wide citation rate or a change over time. These are API observations, not tests of the consumer apps.

The gap suggests a test

These answers suggest a question worth investigating: could a practical guide to evaluating citation evidence make Skill Trade useful earlier in the buying process? First, expand the fixed, unbranded question set and collect three baseline runs. Then publish one guide answering those broader questions and repeat the same measurement weekly for three weeks.

Readout: compare mentions and owned-site citations by stage under the same model and search settings, with every failed attempt visible. Stop rule: if the baseline is not repeatable, or three valid follow-up runs add no problem- or category-stage citations, stop expanding that content approach and revisit the hypothesis. This is a proposed experiment; no uplift has been measured.

Download the report · Inspect answers and citation evidence (JSON). The export keeps complete generated answers and URL annotations; retrieved source excerpts and provider request metadata are omitted. Full raw responses remain private.

Run your own question set

Download the citation meter and the example config into the same private working folder. You need Python 3.10 or later; the script uses the standard library.

Replace the example domain, competitor and brand aliases. Write questions for three stages: understanding the problem, choosing a category and deciding between vendors. Give each question a stable ID and stage. Each question starts a fresh API request.

python3 citation_meter.py --config example.json --dry-run
python3 citation_meter.py --config example.json
python3 citation_meter.py --config example.json --trend

The first command previews the setup without an API call. For a real run, supply a provider key through its environment variable, or use OPENROUTER_API_KEY. Native keys take priority. Keep keys, real configs and raw responses outside the published site.

Read the setup and reporting guide for key names, model settings and saved-report commands. The APIs provide controlled measurements; they do not reproduce each provider's consumer app.

Inspect what each number counts

Brand mention: The answer names a configured brand alias or domain. This can occur with or without a citation.
Owned-site citation: The answer cites the brand's domain or a subdomain. A page appearing only in search results does not count.
Third-party citation candidate: A cited passage names the brand and links to another domain. Read that source before calling it coverage, support or a recommendation.
Uncited mention: The answer names the brand but contains no citations at all. The tool cannot tell whether training or retrieval produced the mention.

These outcomes can overlap. The report shows counts by problem, category and decision stage, with completed and planned observations beside them.

How the source evidence is read

OpenAI: URL citations attached to the answer text.
Anthropic: web citations attached to text blocks, separate from search-result blocks.
Gemini: grounding sources linked to answer segments. Redirect links retain both their original and resolved URLs.
Perplexity: numbered answer references mapped to the returned citation list.
OpenRouter: standardized answer annotations, with native search explicitly requested.

Every attempted question retains its status, raw response when returned, answer text, cited URLs, provider and model. A citation shows an attached source; it does not establish that the source supports every claim in the answer.

Keep the comparison honest

A failed request is unavailable. A run with missing answers does not get a complete-run citation rate. Changes to the questions, brand aliases, models, provider or parser restart the comparison. Older results from the original URL-harvesting parser remain labeled as unverified.

To compare category wording, use the framing example. Both versions use the same question IDs, stages and buying criteria.

python3 citation_meter.py --config framing-example.json --all-arms --repeats 3 --seed 0

The meter repeats questions and interleaves the two versions. It compares observations that finished in both, shows missing coverage and reports the range across repeats. A difference suggests a hypothesis to test; it does not prove that wording caused the change. Naming a brand in a question also prompts its visibility.

Choose the audience before the questions

sparktoro.py uses SparkToro's audience data to help choose assistants and topics. It returns assistant usage and prompt topics, not citations or verbatim buyer questions.

python3 sparktoro.py --describe "small teams evaluating project management software" --slug sample
python3 sparktoro.py --slug sample --report
python3 sparktoro.py --slug sample --sync-config example.json

Generated questions are drafts. Rewrite and approve them before measurement; existing questions stay intact. Dated audience data can supply weights for a citation index. People may use several assistants, so that index does not measure unique reach.

Questions people ask

How do I check if AI answers cite my website?

Add your domain, brand aliases and buyer questions to the example config, then run citation_meter.py. It records mentions and answer citations by buying stage. Use --dry-run first to preview the setup without API calls.

Do I need all four API keys?

No. Use a provider's own key or OPENROUTER_API_KEY. The optional engines list selects providers. OpenRouter requests native search; support varies by model, and unusable responses stay unavailable. A provider's own key takes priority.

What does a run cost?

The scripts are free to download. Providers charge for model use and search; more questions and repeats increase usage. SparkToro requires an account and credits. Check your provider's current pricing and set a spending limit before running.

I publish these tools because a team should be able to inspect the measurement behind a decision. Building them sits alongside my work setting strategy, coaching teams and leading demand generation. See the complete hiring case.

Talk about a full-time role