Skip to content

Challenge 011: Evaluation-Driven RAG

Level: L300 Type: Challenge Time: ~90 min Cost: Free

Scenario

Improve a local RAG configuration only when the golden-set metrics show a measurable gain without citation regressions.

flowchart LR
  B[Baseline] --> S[Specification or golden set]
  S --> I[Implementation]
  I --> V[Local validator]
  V --> R[Review evidence]
  R --> P[PR-ready result]

Task

  1. Complete the starter JSON without changing its public schema.
  2. Run the unit tests and inspect every failed assertion.
  3. Keep the solution local and deterministic.
  4. Run the validator to print the completion code.

Acceptance criteria

  • Scope, non-goals, risks, and validation are explicit.
  • At least four bounded execution steps are present.
  • Tests run without credentials or GitHub Classroom.
  • Evidence is suitable for a pull-request description.

Local validation

cd docs/docs/en/challenges/challenge-011
python -m unittest -v
python validate.py