Module 9: neural NLP for real messages

{
  "module": 9,
  "framework": "TensorFlow 2.19.1",
  "dataset": "UCI SMS Spam Collection",
  "epochs_requested": 25,
  "best_validation_epoch": 7,
  "weight_change_norm": 72.81667194171665,
  "threshold": 0.15,
  "baseline_threshold": 0.5,
  "selection": "validation loss checkpoint; validation balanced accuracy threshold; patience 5",
  "sample_rows": 5159,
  "test": {
    "n": 1032,
    "tp": 118,
    "tn": 874,
    "fp": 30,
    "fn": 10,
    "precision": 0.7972972972972973,
    "recall": 0.921875,
    "specificity": 0.9668141592920354,
    "selection_rate": 0.1434108527131783,
    "fpr": 0.033185840707964605,
    "balanced_accuracy": 0.9443445796460177,
    "f1": 0.855072463768116,
    "roc_auc": 0.9858095962389382,
    "pr_auc_average_precision": 0.9465715319501918
  },
  "linear_baseline": {
    "n": 1032,
    "tp": 116,
    "tn": 891,
    "fp": 13,
    "fn": 12,
    "precision": 0.8992248062015504,
    "recall": 0.90625,
    "specificity": 0.9856194690265486,
    "selection_rate": 0.125,
    "fpr": 0.014380530973451327,
    "balanced_accuracy": 0.9459347345132743,
    "f1": 0.9027237354085603,
    "roc_auc": 0.9915911919247786,
    "pr_auc_average_precision": 0.9650879074379307
  },
  "saved_model_replay_verified": true,
  "vocabulary_fitted_on": "training only"
}

Checkpoint selection uses validation loss with patience 5; each method selects its own threshold by validation balanced accuracy. Final labels never select weights or thresholds.

The learning-history table is the numerical alternative to a curve: compare training/validation loss and identify divergence without relying on a plotted line.

A small network may underperform the simpler baseline. These finite historical samples and small epoch budgets do not establish production readiness.

For NLP, text IDs replace raw messages in reports. Public messages can still contain personal content; processing is local.