Module 9 · Neural and language models

Module 9: guided construction and mastery

Use this after the foundation bridge and before attempting independent practice; return to the module run instructions afterward. Read, predict, run, change, and explain. The shared runner remains the complete reference; these small steps expose its ideas.

Worked concept

A token ID is an index, not an amount. In this hand example 0 is padding, 1 and 2 index vectors [1,0] and [0,2]. The masked mean of [1,2,0] is [.5,1], not [1/3,2/3]. In the real model, training learns vectors; the hand table only explains the operation.

Predict and construct

Predict the printed values, shape or rejection before running. Use the documented environment. The notebook contains this same runnable construction cell.

import numpy as np
learning_ids = np.array([1,2,0])
learning_embedding = np.array([[0.,0.],[1.,0.],[0.,2.]])
learning_mask = learning_ids != 0
learning_pooled = learning_embedding[learning_ids][learning_mask].mean(axis=0)
print("vectors",learning_embedding[learning_ids],"pooled",learning_pooled)
assert np.allclose(learning_pooled,[.5,1.])

Complete with less support

Inspect the training-fitted real vocabulary. Demonstrate an unknown token and truncation on a training example, then explain what information is unavailable to the classifier.

Debug a plausible misconception

Why is adapting vocabulary on test text leakage? Write a prediction before opening feedback. Include a counterexample or calculation, not only a definition.

Independent transfer

Specify a text-classification task outside spam, including privacy and a credible lexical baseline.

Minimum mastery evidence

Keep the evidence listed in the study workflow and name the module outcome each item supports. The criteria describe developing, meeting and extending the outcome.