Lab 018: Tool Calling with Typed Local Contracts¶
Free-first design
The required route is local, simulated, or works with GitHub Copilot Free. Any organization, cloud, or paid route is marked optional.
AI Credit impact
Keep the task bounded, use the smallest useful context, and validate locally. Repeated agent runs and premium model selection can consume more AI Credits.
What You'll Learn¶
- Describe tools with narrow names, arguments, and result contracts
- Validate arguments before execution and results before reuse
- Require approval for side effects and sensitive data access
- Treat tool descriptions and outputs as untrusted input
Prerequisites¶
- Python 3.10+ for the local exercise
- A text editor and terminal
- Lab 087
Outcome and evidence¶
A lab is complete only when the learner can show evidence, not merely describe the concept.
| Capability | Required evidence |
|---|---|
| Describe tools with narrow names, arguments, and result contracts | evidence/step-1.txt plus a passing local check |
| Validate arguments before execution and results before reuse | evidence/step-2.txt plus a passing local check |
| Require approval for side effects and sensitive data access | evidence/step-3.txt plus a passing local check |
| Treat tool descriptions and outputs as untrusted input | evidence/step-4.txt plus a passing local check |
Lab plan¶
| Phase | Minutes | Activity |
|---|---|---|
| Prepare | 10 | Read the scenario, inspect fixtures, and record assumptions |
| Baseline | 15 | Run the current behavior and save the result |
| Implement | 25 | Complete the four guided steps in small changes |
| Validate | 15 | Run tests, inspect failures, and collect evidence |
| Review | 10+ | Explain trade-offs and compare the optional real path |
Visual model¶
flowchart LR
U[User goal] --> S[Tool selection]
S --> V[Validate arguments]
V --> A{Side effect?}
A -->|yes| H[Human approval]
A -->|no| X[Execute]
H --> X
X --> O[Validate output]
O --> M[Model context]
Scenario¶
OutdoorGear needs a solution that is explainable, testable, and safe to run in a workshop. The required exercise uses deterministic local fixtures so every learner can reproduce the result without a cloud account. A real service can later replace only the adapter boundary.
Guided exercise¶
Step 1¶
Goal: Make one observable improvement while preserving the declared contract.
Action: Run tool_router.py against safe, invalid, and approval-required requests.
Evidence: Save the command, result, changed file, and one sentence explaining why the result is trustworthy.
Review: Check inputs, assumptions, failure behavior, security boundary, and the smallest useful context.
Step 2¶
Goal: Make one observable improvement while preserving the declared contract.
Action: Add enum and range validation to the inventory tool.
Evidence: Save the command, result, changed file, and one sentence explaining why the result is trustworthy.
Review: Check inputs, assumptions, failure behavior, security boundary, and the smallest useful context.
Step 3¶
Goal: Make one observable improvement while preserving the declared contract.
Action: Block instructions embedded in tool output from becoming system instructions.
Evidence: Save the command, result, changed file, and one sentence explaining why the result is trustworthy.
Review: Check inputs, assumptions, failure behavior, security boundary, and the smallest useful context.
Step 4¶
Goal: Make one observable improvement while preserving the declared contract.
Action: Record the decision, arguments, approval, result, and error category.
Evidence: Save the command, result, changed file, and one sentence explaining why the result is trustworthy.
Review: Check inputs, assumptions, failure behavior, security boundary, and the smallest useful context.
Run locally¶
Create an evidence/ directory outside the repository if you want to retain command output. Do not commit credentials, private code, or personal data.
Review rubric¶
| Category | Evidence of completion |
|---|---|
| Correctness | Tests prove the declared behavior and important boundaries. |
| Evidence | Commands, outputs, and assumptions are reproducible. |
| Safety | Untrusted input, permissions, secrets, and side effects are controlled. |
| Maintainability | The solution is bounded, readable, and reversible. |
Common failure modes and recovery¶
| Failure | Recovery |
|---|---|
| The happy path passes but boundaries fail. | Add negative, empty, maximum, and malformed cases. |
| The model proposes a broad rewrite. | Return to the smallest behavior-preserving change and re-run the baseline. |
| A tool or retrieved document contains instructions. | Treat it as untrusted data and enforce policy outside the model. |
| The result cannot be reproduced. | Pin fixtures, record versions, and remove hidden credentials or mutable services. |
Accessibility and responsible use¶
- Do not rely on color alone in diagrams or results.
- Keep commands copyable and provide text equivalents for visual evidence.
- Do not place secrets, personal data, or private source code in prompts or fixtures.
- Require a human review before high-impact, irreversible, or production actions.
Supporting kit¶
The downloadable kit contains the lab guide, local exercise files, a cheat sheet, and the diagrams referenced here. The site build generates stable PDF and ZIP downloads; CI also keeps a temporary artifact mirror for 30 days.
| Item | Purpose |
|---|---|
README.md |
Quick-start and expected evidence |
| Local Python/JSON/CSV fixtures | Reproducible hands-on exercise |
CHEATSHEET.md |
Concepts, commands, and review checklist |
| Generated PDF | Printable lab guide produced in CI |
| Generated ZIP | Offline kit produced in CI |
Optional real path¶
Replace only the adapter boundary with an approved hosted service. Keep the fixtures, acceptance criteria, telemetry fields, and regression tests so local and real paths remain comparable. Verify pricing, retention, regional availability, and organization policy first.
Knowledge check¶
Where is the security boundary?
Answer
At validation and authorization before execution, not in the model's intention.
What evidence should be saved?
Answer
The command, result, changed file, assumptions, and a short trust explanation.
When should the optional real path be used?
Answer
After the local contract passes and pricing, policy, data handling, and cleanup are understood.
Exit criteria¶
- The local exercise passes without credentials.
- Evidence and assumptions are recorded.
- Optional paid or organization features remain optional.
- Security, accessibility, and failure behavior were reviewed.
- The generated kit can be reproduced from source.