Use retrieved evidence carefully | Introduction to Generative AI
Understand how generative AI works, write source-bounded prompts, validate outputs, evaluate grounded answers and design a responsible career-learning assistant.
Prerequisites: No prior AI experience required. Basic familiarity with reading short code examples is helpful; no paid API account is needed.
Use retrieved evidence carefully
Retrieval-augmented generation combines a search step with a generation step. A system retrieves relevant records from an approved collection, supplies selected passages as context and asks a model to answer using that evidence. Retrieval can improve access to current or private organizational information, but it does not eliminate generation errors. The search system may return irrelevant or outdated material, and the model may still make unsupported claims.
For a small learning example, imagine a course FAQ containing enrollment rules, assessment rules and certificate rules. A question about certificate eligibility should retrieve the certificate rule rather than an unrelated enrollment paragraph. A practical pipeline records document identifiers, preserves access controls during retrieval and asks for an answer with references. Do not place restricted material into a public prompt simply because a search index returned it.
Evaluate retrieval and generation separately. First ask whether the correct source passage was available and retrieved. Then ask whether the generated answer follows that passage. A citation is useful only if it actually supports the statement beside it. When the approved sources do not contain the answer, the system should state that limitation. For high-impact decisions, keep human review and a route to the underlying records.
Worked example
documents = [
{"id": "enrollment", "text": "Candidates may enroll after signing in."},
{"id": "certificate", "text": "Complete every lesson and pass every chapter quiz."}
]
query = "certificate"
# Simple keyword retrieval for a teaching example; not semantic search.
results = [d for d in documents if query in d["id"]]
print([d["id"] for d in results]) # ["certificate"]
Try it yourself
Create three short policy passages and two questions. Record which passage supports each answer. Add a question the passages cannot answer and write an appropriate abstention.
Check your understanding: explain why the example works, state one limitation, and record how you would test it.
1. Understand generative AI
2. Give clear instructions
3. Ground and evaluate answers
4. Build a responsible learning prototype