Lab 101: Token Economics: Cost, Cache, Batch, and Routing¶
Free-first design
The required route is local, simulated, or works with GitHub Copilot Free. Any organization, cloud, or paid route is marked optional.
AI Credit impact
Keep the task bounded, use the smallest useful context, and validate locally. Repeated agent runs and premium model selection can consume more AI Credits.
What You'll Learn¶
- Estimate input, cached input, output, and tool-call costs
- Identify stable prefixes suitable for caching
- Choose batch processing when latency is flexible
- Route by task value, quality threshold, latency, and AI Credit policy
Prerequisites¶
- Python 3.10+ for the local exercise
- A text editor and terminal
- Lab 097
Outcome and evidence¶
A lab is complete only when the learner can show evidence, not merely describe the concept.
| Capability | Required evidence |
|---|---|
| Estimate input, cached input, output, and tool-call costs | evidence/step-1.txt plus a passing local check |
| Identify stable prefixes suitable for caching | evidence/step-2.txt plus a passing local check |
| Choose batch processing when latency is flexible | evidence/step-3.txt plus a passing local check |
| Route by task value, quality threshold, latency, and AI Credit policy | evidence/step-4.txt plus a passing local check |
Lab plan¶
| Phase | Minutes | Activity |
|---|---|---|
| Prepare | 10 | Read the scenario, inspect fixtures, and record assumptions |
| Baseline | 15 | Run the current behavior and save the result |
| Implement | 25 | Complete the four guided steps in small changes |
| Validate | 15 | Run tests, inspect failures, and collect evidence |
| Review | 10+ | Explain trade-offs and compare the optional real path |
Visual model¶
flowchart LR
T[Task] --> R{Router}
R -->|simple/private| L[Local or included model]
R -->|quality threshold| P[Optional paid model]
L --> C[Cache stable prefix]
P --> C
C --> B[Batch flexible work]
B --> M[Measure cost and quality]
Scenario¶
OutdoorGear needs a solution that is explainable, testable, and safe to run in a workshop. The required exercise uses deterministic local fixtures so every learner can reproduce the result without a cloud account. A real service can later replace only the adapter boundary.
Guided exercise¶
Step 1¶
Goal: Make one observable improvement while preserving the declared contract.
Action: Run token_economics.py on the included scenarios.
Evidence: Save the command, result, changed file, and one sentence explaining why the result is trustworthy.
Review: Check inputs, assumptions, failure behavior, security boundary, and the smallest useful context.
Step 2¶
Goal: Make one observable improvement while preserving the declared contract.
Action: Calculate baseline and optimized cost with explicit assumptions.
Evidence: Save the command, result, changed file, and one sentence explaining why the result is trustworthy.
Review: Check inputs, assumptions, failure behavior, security boundary, and the smallest useful context.
Step 3¶
Goal: Make one observable improvement while preserving the declared contract.
Action: Move stable instructions into a cacheable prefix and remove duplicate retrieval.
Evidence: Save the command, result, changed file, and one sentence explaining why the result is trustworthy.
Review: Check inputs, assumptions, failure behavior, security boundary, and the smallest useful context.
Step 4¶
Goal: Make one observable improvement while preserving the declared contract.
Action: Define a routing policy with a local/free default and optional paid escalation.
Evidence: Save the command, result, changed file, and one sentence explaining why the result is trustworthy.
Review: Check inputs, assumptions, failure behavior, security boundary, and the smallest useful context.
Run locally¶
Create an evidence/ directory outside the repository if you want to retain command output. Do not commit credentials, private code, or personal data.
Review rubric¶
| Category | Evidence of completion |
|---|---|
| Correctness | Tests prove the declared behavior and important boundaries. |
| Evidence | Commands, outputs, and assumptions are reproducible. |
| Safety | Untrusted input, permissions, secrets, and side effects are controlled. |
| Maintainability | The solution is bounded, readable, and reversible. |
Common failure modes and recovery¶
| Failure | Recovery |
|---|---|
| The happy path passes but boundaries fail. | Add negative, empty, maximum, and malformed cases. |
| The model proposes a broad rewrite. | Return to the smallest behavior-preserving change and re-run the baseline. |
| A tool or retrieved document contains instructions. | Treat it as untrusted data and enforce policy outside the model. |
| The result cannot be reproduced. | Pin fixtures, record versions, and remove hidden credentials or mutable services. |
Accessibility and responsible use¶
- Do not rely on color alone in diagrams or results.
- Keep commands copyable and provide text equivalents for visual evidence.
- Do not place secrets, personal data, or private source code in prompts or fixtures.
- Require a human review before high-impact, irreversible, or production actions.
Supporting kit¶
The downloadable kit contains the lab guide, local exercise files, a cheat sheet, and the diagrams referenced here. The site build generates stable PDF and ZIP downloads; CI also keeps a temporary artifact mirror for 30 days.
| Item | Purpose |
|---|---|
README.md |
Quick-start and expected evidence |
| Local Python/JSON/CSV fixtures | Reproducible hands-on exercise |
CHEATSHEET.md |
Concepts, commands, and review checklist |
| Generated PDF | Printable lab guide produced in CI |
| Generated ZIP | Offline kit produced in CI |
Optional real path¶
Replace only the adapter boundary with an approved hosted service. Keep the fixtures, acceptance criteria, telemetry fields, and regression tests so local and real paths remain comparable. Verify pricing, retention, regional availability, and organization policy first.
Knowledge check¶
What is the cheapest token?
Answer
The token you do not send; remove irrelevant and duplicated context first.
What evidence should be saved?
Answer
The command, result, changed file, assumptions, and a short trust explanation.
When should the optional real path be used?
Answer
After the local contract passes and pricing, policy, data handling, and cleanup are understood.
Exit criteria¶
- The local exercise passes without credentials.
- Evidence and assumptions are recorded.
- Optional paid or organization features remain optional.
- Security, accessibility, and failure behavior were reviewed.
- The generated kit can be reproduced from source.