# Worked Example: Channel Conversion Model

## Decision

Decide whether to roll out a treatment that changes onboarding copy across acquisition channels.

## Generative Story

Each channel has its own baseline conversion rate. The treatment may improve conversion overall, but the effect can vary by channel. Low-volume channels should borrow information from high-volume channels rather than producing extreme estimates.

## Candidate Models

Complete pooling:

```text
conversion ~ treatment
```

Use as a stable baseline, but it may hide channel differences.

No pooling:

```text
conversion ~ channel + treatment:channel
```

Use to inspect heterogeneity, but expect noisy estimates in small channels.

Partial pooling:

```text
channel baseline effects share a group distribution
channel treatment effects share a group distribution
```

Use as the primary model because it balances stability and heterogeneity.

## Priors

Baseline conversion:

```text
alpha_channel ~ Normal(logit(0.22), 1.0)
```

Average treatment effect:

```text
mu_tau ~ Normal(0, 0.35)
```

Channel variation:

```text
sigma_tau ~ Exponential(2)
```

Prior predictive checks:

- baseline rates should usually fall between 5% and 60%
- treatment lifts above 20 percentage points should be possible but rare
- channels should differ, but not imply impossible rates

## Computation

Before interpreting results:

- R-hat should be near 1.00
- effective sample size should be adequate
- there should be no divergent transitions
- trace plots should mix well
- posterior draws should not pile up against prior boundaries

If diagnostics fail, rescale, reparameterize, or strengthen priors before reporting.

## Posterior Predictive Checks

Compare replicated data to observed data for:

- overall conversion rate
- conversion by channel
- treatment-control difference by channel
- low-volume channel extremes
- number of unusually high or low conversion cells

If replicated data misses channel-level variability, revise the hierarchy or likelihood.

## Decision Summary

A strong report might say:

> The treatment is probably beneficial overall, but uncertainty remains in low-volume channels. The partial-pooling model estimates a positive average lift with the strongest evidence in organic and paid search. Because partner and enterprise channels are noisy and strategically important, ship to low-risk channels first and keep a holdback for high-value segments.

## Strongest Critique

The model assumes channel exchangeability. If enterprise traffic is generated by a fundamentally different sales process, partial pooling may borrow too much from self-serve channels. Enterprise should either have a separate model or a much more conservative rollout rule.