Module 10: likelihood scoring versus free generation

{
  "module": 10,
  "model": "HuggingFaceTB/SmolLM2-135M-Instruct",
  "revision": "12fd25f77366fa6b3b4b768ec3050bf629380bac",
  "real_examples": 100,
  "sampling": "Seeded label-stratified subset of the deduplicated final split; historical sample, not deployment prevalence",
  "strict_schema_rate": 0.99,
  "exact_label_accuracy_including_invalid_as_wrong": 0.39,
  "label_likelihood_accuracy": 0.85,
  "matched_baseline_accuracy": 0.97,
  "majority_accuracy": 0.88,
  "baseline_threshold": 0.45,
  "likelihood_threshold": 0.85,
  "rate_intervals": [
    {
      "method": "free_generation",
      "metric": "schema_validity",
      "successes": 99,
      "observations": 100,
      "rate": 0.99,
      "ci_low": 0.9455138038212947,
      "ci_high": 0.9982325679358592,
      "confidence": 0.95
    },
    {
      "method": "free_generation",
      "metric": "accuracy",
      "successes": 39,
      "observations": 100,
      "rate": 0.39,
      "ci_low": 0.3001687400639442,
      "ci_high": 0.48796982954159257,
      "confidence": 0.95
    },
    {
      "method": "label_likelihood",
      "metric": "accuracy",
      "successes": 85,
      "observations": 100,
      "rate": 0.85,
      "ci_low": 0.7671644040916764,
      "ci_high": 0.9069401471634336,
      "confidence": 0.95
    },
    {
      "method": "lexical",
      "metric": "accuracy",
      "successes": 97,
      "observations": 100,
      "rate": 0.97,
      "ci_low": 0.9154806357094724,
      "ci_high": 0.9897454759759611,
      "confidence": 0.95
    },
    {
      "method": "training_majority",
      "metric": "accuracy",
      "successes": 88,
      "observations": 100,
      "rate": 0.88,
      "ci_low": 0.8018790057328857,
      "ci_high": 0.9300059356298052,
      "confidence": 0.95
    }
  ],
  "mean_generation_seconds": 0.8919392535599957,
  "mean_scoring_seconds": 1.3857195544900025,
  "comparison_contract": "Same IDs and masked first 240 characters; four real training-only demonstrations for both LLM methods; baseline fits training and selects threshold on validation.",
  "decoding": "greedy free generation vs exhaustive conditional JSON-candidate likelihood",
  "contact_masking": "Regex masking reduces contact display; not a privacy guarantee"
}

Candidate scoring always returns a legal label by construction. It does not demonstrate that the model learned JSON generation or that its label is correct.

Invalid free-generated JSON is counted as wrong; no repair, retries or test-selected prompts.

Wilson 95% intervals describe binomial sampling uncertainty, not dataset shift, sampling bias or paired method differences.

Demonstrations are real training records; reports use hashes rather than raw SMS. Latencies exclude model loading and training.