Module 4 · Foundations

Module 4: Exploration and visual communication

Read the overview and learning outcomes here, then follow the learning route. Return to this page’s run instructions for the complete study.

Explore a service-workload table, choose useful charts, and communicate what the data support. You will run a repeatable analysis that produces figures, their actual data tables, and written descriptions. This is descriptive exploration, with no model training. No paid AI account, GPU, earlier notebook state, or Canvas assignment is required. Allow 90–120 minutes plus setup and practice.

Learning outcomes

• M04.1: Justify a preprocessing choice for an exploratory question and explain what information it may remove.
• M04.2: Create labeled distribution, comparison, and relationship/time plots with appropriate scales and units.
• M04.3: Explain a pattern or anomaly with numerical evidence while distinguishing association from causation.
• M04.4: Write a stakeholder summary with chart descriptions, underlying values, and limitations.

You need basic functions, file paths, and dataframe selection/grouping. Use the foundation bridge, Module 2, or Module 3 as needed. You may enter here directly with the complete source archive and setup below.

Learning route

Follow foundation → concepts and small cases → guided construction → independent practice. Use solutions after your attempt, then connect your work to the complete reference below.

A question before a chart

A fictional operations lead asks: “How do request totals differ between two services, and which daily observations deserve investigation?” The unit of observation is one service on one UTC calendar date, not an individual request. The fixture covers January 1–12, 2027; these are invented dates and values, not measured future operations.

A 95th percentile is a value at or below which roughly 95% of the underlying observations fall, depending on the estimation method. Here it is supplied as a fictional daily aggregate. The mean or median of these daily service p95 values is not an overall request-level p95; obtaining that would require the relevant request-level distribution or an appropriate mergeable representation.

Choose preprocessing deliberately

The raw sample contains 25 rows. Normalize the one inconsistent service label, retain the first exact duplicate, and obtain 24 distinct observations. Stop if different measurements share the same date/service key. Keep raw files unchanged.

One observation has missing latency. Retain its 130 requests for workload totals; omit it only from latency-specific views. Dropping the whole row would answer a different question with a smaller workload total. Do not replace unavailable latency with zero.

The 160 ms value is deliberately constructed as a spike, not a known recording error. A 1.5-IQR rule flags it for investigation. IQR is the upper quartile minus the lower quartile; the example uses pandas' default linear-interpolation quantiles. The fences are 10 and 58 ms. Keep the flagged observation and describe it. A statistical flag is not proof that a value is wrong.

Missing date/service rows are not inferred to mean zero traffic. Totals describe only supplied observations. The daily table reports service coverage; the time plot breaks its line across entirely absent dates. In this fixture, both services are present on all 12 dates.

Setup and run

Obtain a checkout or extract the complete source archive. Open a terminal in ml_ai_it. Follow the environment instructions if you do not yet have .venv. Install the additional plotting libraries into that same environment. If a download link does not open, use the offline access route.

Windows PowerShell:

.\.venv\Scripts\python.exe -m pip install -e ".[visualization]" -c environments/constraints-tested.txt
.\.venv\Scripts\python.exe examples/service-workload/scripts/explore.py --input data/samples/service-workload/daily.csv --output runs/module04

macOS or Linux:

.venv/bin/python -m pip install -e ".[visualization]" -c environments/constraints-tested.txt
.venv/bin/python examples/service-workload/scripts/explore.py --input data/samples/service-workload/daily.csv --output runs/module04

The visualization extra adds Matplotlib and Seaborn; NumPy/pandas remain the shared data dependencies. Internet access is normally needed for installation. No dataset download occurs at runtime. Open runs/module04/index.html in a browser, or read summary.txt, chart_descriptions.md, and the paired CSV tables using your preferred editor. Rerunning replaces these generated outputs; use another directory to retain an experiment.

You can also read the generated worked example without installing Python. It contains five figures, short alternatives, detailed descriptions, and labeled HTML tables. This is a teaching demonstration, not a restricted assessment answer.

Match the chart to the question

The four analytical figures share the same cleaned observations, but use the rows relevant to their question. The fifth is an axis-design demonstration. A bar's length encodes magnitude, so cropping its baseline distorts that comparison. Not every line or scatter plot must start at zero; choose and explain a scale appropriate to its question.

Inspect named results

The histogram bins are [0,40): 16 observations, [40,80): 6, [80,120): 0, [120,160): 0, and [160,200]: 1. The scatter contains 23 observations, some at identical coordinates. Its table preserves every pair even when marks overlap.

A defensible summary says that the fictional January 7 monitor observation combines 400 requests and 160 ms daily p95. It does not say traffic caused latency or that these 12 constructed days predict production reliability. Before investigating a real anomaly, ask about instrumentation, coverage, changes, and operational context.

Shared code and notebook

The script imports src/practical_ml_ai/exploration.py. read_workload documents validation/normalization, summarize_workload produces numerical evidence and plot tables, and render_charts creates labeled figures. run_exploration writes the report and returns the summary. The guided notebook calls these same functions; keep it within the complete source folder and follow notebook setup.

Saved outputs include normalized observations, summary.json, named text values, per-chart CSVs/PNGs, descriptions, and an HTML report. HTML tables have captions and column headers. Charts use labels and, where grouping matters, marker shapes as well as colors. These provisions need manual task-based testing; interface test coverage and effective routes.

Read selectively

• Core concepts: Matplotlib quick start: a Figure and Axes, plotting, labels, and legends.
• Core/reference: pandas descriptive statistics: count, mean, and percentiles; check which values are missing.
• Implementation reference: Seaborn statistical relationships: scatter plots and using style alongside hue. Our example reads local synthetic data, rather than the tutorial's internet-loaded dataset.
• Accessibility reference: WAI complex images: short alternatives plus detailed descriptions and data.

Read only the named sections. Current documentation may describe newer versions than the tested environment; the repository records its exercised versions.

Complete practice and the operations memo, then consult worked feedback. Module 5 introduces regression and held-out evaluation. When exploration informs model choices, use training/development data and preserve final test data for its planned evaluation; no predictive evaluation is performed here.

Instructor adaptation

See instructor guidance for evidence selection, accessible routes, and local adoption.

Build understanding step by step

Use the guided construction and mastery check before the full study. New programmers should start with Python Foundation and readiness; use the glossary for notation.