Reward Experiment

Let’s say you designed an experiment to measure the impact financial rewards have on users’ response rates.

The result shows that the treatment group with $\$10$ rewards has a 30% response rate, while the control group without rewards has a 50% response rate.

Can you explain what happened and how you could improve this experimental design?