Guided analysis practice

Work through the course-kit datasets like decision artifacts

Each lab turns a small dataset into a structured memo: validity first, method second, decision last.

Browse labs →View courses

Analysis labs

Five guided dataset exercises

Use the notebook when you want executable analysis, the exercise when you want prompts, and the answer key when you want calibration.

Experimental Design

Onboarding Experiment Readout

Should the new onboarding checklist ship, iterate, or hold?

  1. Check assignment, missingness, and guardrail readiness before lift.
  2. Estimate activation, support tickets, and paid conversion by arm.
  3. Inspect acquisition-channel caveats without overclaiming segments.
  4. Write a ship, iterate, hold, or rerun decision memo.
Open labNotebookDatasetExerciseAnswer keyCourse
Causal Inference

Feature Adoption and Retention

Is collaboration adoption plausibly causing retention, or just marking stronger teams?

  1. Compute the naive retention association.
  2. Check imbalance in pre-treatment team and activity variables.
  3. Name confounders, mediators, and overlap concerns.
  4. Recommend adjustment, sensitivity analysis, or a randomized prompt design.
Open labNotebookDatasetExerciseAnswer keyCourse
Reinforcement Learning

Logged Support Triage Policy

Can a candidate support-routing policy be evaluated safely from logs?

  1. Summarize the behavior policy and observed action mix.
  2. Identify weak state-action support and low behavior probabilities.
  3. Inspect reward, escalation, CSAT, and fallback guardrails.
  4. Write a staged rollout or no-go policy-readiness note.
Open labNotebookDatasetExerciseAnswer keyCourse
Bayesian Statistics

Channel Conversion Model Criticism

Should rollout decisions pool, split, or partially pool channel-level evidence?

  1. Compute conversion rates and treatment lift by channel.
  2. Compare noisy small-channel estimates with the pooled pattern.
  3. Name what posterior predictive checks would matter for rollout.
  4. Write a partial-pooling model criticism note.
Open labNotebookDatasetExerciseAnswer keyCourse
A/B Testing at Scale

Experiment Registry Audit

Which experiments can proceed, which need review, and which should be blocked?

  1. Summarize risk tiers and experiment statuses.
  2. Find blocked, review-needed, and high-risk experiments.
  3. Check guardrails, randomization unit, and archive readiness.
  4. Write a proportional review-board recommendation.
Open labNotebookDatasetExerciseAnswer keyCourse
Reinforce OS

Bandit Experiment Readout

Should the winning arm be deployed, and was the bandit allocation strategy justified?

  1. Check assignment balance and verify idempotency in the trial log.
  2. Inspect allocation drift — how quickly did Thompson sampling shift toward the better arm?
  3. Read prob_better_than_control and lift_hdi_low for the primary metric and any guardrails.
  4. Write a deploy, extend, or redesign decision memo with stopping-rule rationale.
Open labNotebookDatasetExerciseAnswer keyCourse