Module 11 · Neural and language models

Module 11: Fine-tuning language models responsibly

Read the overview and learning outcomes here, then follow the learning route. Return to this page’s run instructions for the complete study.

Perform real parameter-efficient gradient updates on annotated messages, preserve the base model, and audit held-out loss and replay.

This module works independently: no earlier notebook state, Canvas assignment, paid AI account, or GPU is required. Allow approximately 2–3 hours plus first installation/downloads; the default CPU training study can take 15–30 minutes depending on hardware. Small CPU runs demonstrate mechanisms, not production readiness.

Learning outcomes

• M11.1: Distinguish prompting, inference, supervised fine-tuning, and LoRA.
• M11.2: Prepare completion labels with prompt-token loss masking.
• M11.3: Update a small adapter on training-only real annotations.
• M11.4: Compare predeclared base/adapted held-out completion losses without promising improvement.
• M11.5: Save/reload adapter arrays with the exact model revision and disclose limits.

Learning route

Follow foundation → concepts and small cases → guided construction → independent practice. Use solutions after your attempt, then connect your work to the complete reference below.

Entry support and topics

Use the foundation bridge if these concepts are new. Basic Python/functions/files and the train/validation/test distinction are assumed with support from Module 2 and Module 5.

LoRA adds a low-rank update to a frozen linear projection. The guided rank-2 construction exposes the operation; the complete study uses rank 8 in query and value projections across all layers, while base weights remain frozen. Masking prompt labels with -100 prevents the objective from training on every instruction token.

Real data and its limits

UCI SMS Spam Collection; real human-labeled messages. Public messages may contain personal content; processing stays local and reports use hashes/contact masking. See the source register.

Sources, licenses, hashes, and model provenance distinguish external records from original teaching code. No generated dataset or invented labels are used in this module. Public files still require responsible handling. Source versions are fixed after verified acquisition; changed cached/upstream bytes cause an error. The first data/model download requires internet; retain the cache for offline replay.

The complete workflow

Train rank-8 PEFT LoRA on every q_proj and v_proj, using 256 real training messages, batch size 8 and a predeclared 200-step maximum. Prompt and padding labels are masked with -100. Every 25 steps, measure completion loss on 40 disjoint validation messages; retain the lowest-loss checkpoint, including the zero-step adapter if training never improves it. The selected checkpoint is reloaded before validation generation and the predefined base/adapted comparison on 100 disjoint balanced final messages. Compare JSON validity, label correctness and completion loss separately. Save adapter safetensors, configuration and the exact base revision; verify reloaded logits. No invented training messages or labels are used.

Run and inspect

Download the complete source or use a checkout. Work from its ml_ai_it folder. Use the isolated CPU framework environment. Run framework commands in the Linux CPU reference environment; see the support matrix for platform coverage. Reading the reports needs no framework installation. In commands below, python means your environment executable; use its full path if not activated. If a download link does not open, use the offline access route.

python examples/sms-fine-tuning/scripts/run.py --output runs/module11

Use a fresh output directory for a new run; completed studies are not silently overwritten. Open runs/module11/index.html or read summary.txt/summary.json and the companion CSV tables. Read the verified worked report without installing an execution interface.

Key artifacts:

• training_history.csv
• adapter/adapter_model.safetensors
• adapter/adapter_config.json
• study_config.json
• validation_generation.csv
• generation_comparison.csv
• rate_intervals.csv
• split_manifest.json

The shared implementation is under src/practical_ml_ai; scripts and commented notebook call the same functions. Keep the notebook inside the complete source tree. Named results and captioned/scoped tables support interpretation without a chart or screenshot. For interface choices and test coverage, see the accessibility guide.

Read selectively

• Core concept/reference: LoRA primary paper — low-rank update idea, frozen weights, and evaluation distinctions. Read these named sections, not an entire external course.
• Source/model context: central source register, including the relevant official card and license.
• Optional depth: Use a predeclared validation plan for another rank, step budget, or target projection. More complete supervised training and a credible prospective evaluation require additional real data, compute, and review; they are optional depth.

Practice and continue

Complete original practice, then compare worked feedback. Instructor notes map evidence and explain adaptation.

Next: Module 12.

Build understanding step by step

Use the guided construction and mastery check before the full study. New programmers should start with Python Foundation and readiness; use the glossary for notation.