Check the figures in an AI's procurement analysis against the source data

Why an AI's figures need checking

More people researching government procurement now give the data to an AI assistant such as ChatGPT or Claude and ask, for example, which ministry buys the most. The assistant answers in readable prose full of figures, and those figures do not always match the data. A miscopied number, a miscalculated one and one that was never in the data all arrive with the same confidence.

Hashike is a private bilingual (Japanese and English) service that organizes public procurement records: US federal contract opportunities performed in Japan, and award results from Japanese central-government ministries. It is not affiliated with any government. Every Hashike figure states what was counted, over what period and from which source, which is what lets you use it as the reference when you check an AI's answer.

Before an AI answer goes into an internal report or a bid review, someone has to compare its figures with the source data. Below are a procedure for doing that the same way every time, and prompts you can copy as they are.

• ·AI assistant: a tool such as ChatGPT or Claude that answers written questions in writing. The system underneath is called a language model.
• ·Population: the set of records a figure counted, such as one ministry's award results over the last 60 months. The same number means something different when the population changes.

How a figure in an AI answer goes wrong

A figure in an AI answer was either copied from the data, calculated by the AI from the data (a total or a share, say), or found nowhere in the data. Checking means sorting each figure into one of those, then confirming that copied figures match the original and calculated figures are calculated correctly. A figure of the last kind has no source.

A matching number is still wrong if its population or period changed on the way. Hashike's supplier_profile (one Japanese supplier's award history) and buyer_profile (one Japanese ministry's purchasing) report counts and values over a rolling 60-month window. If the AI calls one of those an all-time or an annual total, the number is right and the sentence around it disagrees with the data.

Where the reference figures come from

The reference is the data the AI received. Data leaves Hashike by three routes: a Markdown export that saves a page as a plain-text file, the public API that a program calls to fetch data, and MCP (Model Context Protocol), which lets an AI assistant query Hashike directly. Hashike's service registry defines 9 MCP tools, 11 public API endpoints and 3 download formats. Hashike has no CSV or Excel export. The checking procedure is the same whichever route the data took.

If the AI fetched its figures from Hashike over MCP, you can fetch the same figures again without the AI, because each public API endpoint corresponds to one MCP tool and returns the same aggregates. The request below re-fetches one company's award history by its 法人番号, the number Japan's National Tax Agency assigns to each company.

curl https://hashike.io/api/v1/companies/{corporateNumber} \
  -H "Authorization: Bearer <token>"

The checking procedure

1. Save the data you give the AI. Export the Markdown from a notice page while signed in, or keep the API response in a file. If your assistant is connected to Hashike over MCP, note the tool it called and the arguments it passed.
2. Ask for each figure's origin along with the analysis. The first prompt below has the AI end its answer with a table giving each figure's unit, population, period and location in the data.
3. Compare each figure in that table with the saved data, character for character. A rounded value, a value in different units or a converted currency is not a match.
4. Check how the population and period are described. Look for a 60-month figure presented as an all-time or annual total, and for a Japanese fiscal year confused with a US federal fiscal year.
5. Check that yen and dollars never meet in one calculation. Japanese ministry award values are in yen and US federal contract values in dollars, from different sources and fiscal calendars, so they cannot be added or compared.
6. Remove any figure that is not in the data and any calculation that does not check out. Use only what remains.

Using only the Hashike data below, answer this question: <your question>.

Rules:
- Copy every figure exactly as it appears in the data. Do not round, convert currencies or change units.
- Never add or rank a JPY amount together with a USD amount.
- If you calculate a figure yourself, label it "computed" and show the calculation.
- If the data does not contain a figure you need, say so instead of estimating.

End your answer with a table with these columns: figure, unit or currency, what was counted (population), period, where in the data it appears (field name or heading), copied or computed.

Data:
<paste the Hashike Markdown export or API response here>

Check again in a new conversation

An AI asked to check an answer in the same conversation that produced it can lean on what it already wrote. Start a new conversation and give it only the source data and the answer. Then check its verdicts against the data yourself.

You are checking an analysis against its source data. Do not judge whether the analysis is persuasive; only check its figures.

For every number in the ANSWER, report one line with:
- the number as written in the answer
- verdict: MATCHES (identical value, unit and currency in the DATA), DIFFERS (a value exists in the DATA for the same thing but is different), COMPUTED (derived from DATA values; show the calculation and whether it is correct), or NOT IN DATA
- whether the answer describes the population and period the same way the DATA does
- whether the number mixes JPY and USD amounts

Quote the line of the DATA you relied on for each verdict.

DATA:
<paste the Hashike Markdown export or API response>

ANSWER:
<paste the AI answer>

This check confirms only that an answer's figures agree with Hashike's data. Hashike organizes public records and does not assess whether any opportunity is worth bidding on.

Allowance

API and MCP access is metered against a plan allowance, and every response states what is left. Repeating a call to check an answer uses the same allowance.