Lab

Mon, 01 Jan 0001 00:00:00 +0000

Time: ~25 min | Day 2 PM (shared with M06)

Learning Objectives

Generate and evaluate AI-produced test suites. Learn when to trust AI-generated tests and when human judgment is essential.

Concept Review: AI-Assisted Testing

What AI Excels At in Testing

Task	AI Capability	Human Role
Unit test generation	Generates happy path + common edge cases from function signatures	Add domain-specific edge cases; verify assertions
Edge case discovery	Suggests scenarios humans often miss (null, boundary, type errors)	Filter out unrealistic scenarios
Test data generation	Creates large, varied datasets	Ensure data is realistic for your domain
Mutation testing	Introduces bugs → checks if tests catch them	Interpret results; strengthen weak tests

AI Testing Limitations

Doesn’t know your business logic: AI can test that a function returns something but not that it returns the right thing for your domain.
Can generate buggy tests: AI tests can contain their own bugs — always review assertions.
Misses implicit requirements: “This should never take more than 100ms” — AI won’t know unless you tell it.

The AI Testing Workflow

1. Provide function + context → AI generates baseline tests
2. Run tests → identify failures
3. Ask AI: "What edge cases am I missing?"
4. Add your domain-specific cases → run again
5. Review all assertions for correctness
6. Commit — human-reviewed, AI-assisted

Exercise: Generate & Validate Tests

Step 1 — Generate Baseline Tests (10 min)

Use your code from M03 or M05, or the sample code from M02 (workshop/m02/sample-code.py).

M07: Testing, QA & Validation with AI on Generative AI Boot Camp for Developers