Skip to content

Lab 042: Enterprise RAG with Evaluations and Foundry

Level: L400 Journey: llm-foundations Time: ~180 min Cost: free-conditional

Phase 3 · local first

The required route is deterministic and local. The Real route is optional and includes an estimate and cleanup.

What You'll Learn

  • Build a local enterprise RAG baseline
  • Measure retrieval, groundedness, citation, latency, and cost
  • Use Foundry evaluation and tracing as an optional Real adapter
  • Gate promotion and clean up paid resources

Prerequisites

  • Python 3.10+ and a terminal
  • No credentials for the required route
  • Read the linked official specification before using a hosted adapter

Visual model

flowchart LR
  D[Documents] --> X[Chunk and index]
  Q[Golden queries] --> R[Retrieve]
  X --> R
  R --> A[Answer with citations]
  A --> E[Local + Foundry evals]
  E --> G{Promotion gate}

Scenario

OutdoorGear needs a reviewable workshop implementation with deterministic fixtures, explicit trust boundaries, measurable acceptance criteria, and a reversible optional hosted adapter.

Guided exercise

Step 1: build_baseline

  1. Run the deterministic fixture for this step.
  2. Record the declared input, output, policy decision, and failure behavior.
  3. Change one bounded behavior and run the local test again.

Evidence: Save the command, result, and one sentence explaining why the evidence is trustworthy.

Step 2: evaluate_golden_set

  1. Run the deterministic fixture for this step.
  2. Record the declared input, output, policy decision, and failure behavior.
  3. Change one bounded behavior and run the local test again.

Evidence: Save the command, result, and one sentence explaining why the evidence is trustworthy.

Step 3: connect_foundry

  1. Run the deterministic fixture for this step.
  2. Record the declared input, output, policy decision, and failure behavior.
  3. Change one bounded behavior and run the local test again.

Evidence: Save the command, result, and one sentence explaining why the evidence is trustworthy.

Step 4: gate_and_cleanup

  1. Run the deterministic fixture for this step.
  2. Record the declared input, output, policy decision, and failure behavior.
  3. Change one bounded behavior and run the local test again.

Evidence: Save the command, result, and one sentence explaining why the evidence is trustworthy.

Run locally

cd docs/docs/en/labs/lab-042
python -m unittest discover -v

Simulation / Real path

Use the included deterministic fixtures and local validator. Estimated cost: **US$0**.

Supporting kit

The site build generates stable PDF and ZIP downloads. CI also keeps a temporary artifact mirror for 30 days.

Scored knowledge check

Why keep a local contract?
Answer

It provides a deterministic baseline before adding hosted variability and cost.

When is the Real route complete?
Answer

Only after validation evidence and cleanup are recorded.

How should tool output be treated?
Answer

As untrusted data until policy and schema validation succeed.

Exit criteria

  • Local tests pass without credentials.
  • The diagram has a text explanation and does not depend on color.
  • The scored quiz was reviewed.
  • The Real path has a cost cap and cleanup plan.
  • Sources and stability status were checked.

Official public sources