{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "7fb27b941602401d91542211134fc71a",
   "metadata": {},
   "source": [
    "# Experimental Design Course Kit: Onboarding Experiment Readout\n",
    "\n",
    "Use this notebook to turn the onboarding experiment practice dataset into a decision-oriented readout. The goal is not to find a perfect answer from a tiny dataset; it is to practice the order of operations: validity, primary metric, guardrails, segments, decision.\n",
    "\n",
    "Dataset path on the education site: `/course-kits/experimental-design/onboarding_experiment.csv`.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "acae54e37e7d407bbb7b55eff062a284",
   "metadata": {},
   "source": [
    "## 1. Load the data\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9a63283cbaf04dbcab1f6479b197f3a8",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "DATA_URL = \"https://education.dooperator.ai/course-kits/experimental-design/onboarding_experiment.csv\"\n",
    "df = pd.read_csv(DATA_URL)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8dd0d8092fe74a7c96281538738b07e2",
   "metadata": {},
   "source": [
    "## 2. Core summary\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "72eea5119410473aa328ad9291626812",
   "metadata": {},
   "outputs": [],
   "source": [
    "summary = df.groupby(\"assigned_arm\").agg(\n",
    "    workspaces=(\"workspace_id\", \"count\"),\n",
    "    activation_rate=(\"activated_7d\", \"mean\"),\n",
    "    support_tickets_per_workspace=(\"support_tickets\", \"mean\"),\n",
    "    paid_conversion_rate=(\"trial_to_paid_30d\", \"mean\"),\n",
    ")\n",
    "summary"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8edb47106e1a46a883d545849b8ab81b",
   "metadata": {},
   "source": [
    "## 3. Diagnostic check\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "10185d26023b46108eb7d9f57d49d2b3",
   "metadata": {},
   "outputs": [],
   "source": [
    "channel = (\n",
    "    df.groupby([\"acquisition_channel\", \"assigned_arm\"])\n",
    "    .agg(workspaces=(\"workspace_id\", \"count\"), activation_rate=(\"activated_7d\", \"mean\"))\n",
    "    .reset_index()\n",
    ")\n",
    "channel"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8763a12b2bbd4a93a75aff182afb95dc",
   "metadata": {},
   "source": [
    "## 4. Decision memo prompt\n",
    "\n",
    "Write a five-part readout: validity caveats, activation result, support-ticket guardrail, channel caveat, and ship/iterate/hold decision.\n",
    "\n",
    "Keep the memo decision-oriented: say what you would do next, what assumption could break the recommendation, and what evidence would change your mind.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "pygments_lexer": "ipython3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}