An ungrounded model answers from memory. A grounded one answers from the page in front of it.
How it works
Google’s Vertex AI docs define grounding as “the ability to connect model output to verifiable sources of information.” The source can be a web search, your own documents, a map service or a database. Whatever the source, the pattern has three parts:
- Fetch evidence for the question: retrieved passages, a whole file, an image of an invoice, rows from SQL.
- Put it in the prompt with an instruction to answer only from it.
- Make the answer checkable with quotes, citation markers or links back to the source.
Retrieval-augmented generation is the best-known method. Lewis and colleagues paired a generator with a “dense vector index of Wikipedia”, reached by a neural retriever, and reported answers that were more specific and more factual than a model working from its parameters alone. Building that index uses embeddings and often a vector database.
A minimal grounded prompt:
Answer using only the policy below. Quote the sentence that supports your answer.
If the policy does not answer the question, reply "Not in the policy."
<policy>Refunds are available within 14 days of purchase.</policy>
Question: Can I get a refund after three weeks?
A grounded reply says no and quotes the 14-day sentence. An ungrounded model might guess a 30-day policy that sounds normal.
For long inputs, Anthropic suggests a quote-first step: for documents over 20,000 tokens, have the model extract word-for-word quotes, then answer from those quotes only.
Why it matters
- Fewer made-up facts. Google lists reduced hallucinations as a core benefit. Grounding is the most direct fix for hallucination about your own data.
- Fresh information. Model weights are frozen at training time. Grounded sources can change daily without retraining.
- Auditability. A claim with a quote can be checked by a person or by code.
- Cost trade-off. Evidence is input tokens. Retrieving five passages is cheaper than sending a whole manual, but may miss the passage that matters. Very long prompts also risk lost-in-the-middle recall.
On BroadNet Inference
BroadNet’s docs do not list an embeddings endpoint, a search tool or a citations feature, so retrieval runs in your code. What the API gives you is room for evidence: a 1,048,576-token context window on DeepSeek V4.1 Flash and GLM 5.3, 262,144 on Qwen3.8 27B, and image input on the models whose docs entry says images or vision. That makes “paste the whole document” a real option; long context vs RAG compares the cost of each, and the invoice guide grounds extraction in a scanned image.