fortune-uwha/udacity-course-survey

Designing an A/B test

📘 Online Udacity A/B test results: Design an A/B Test

📖 Introduction(Task description:Free Trial Screener)

• Goal to maximize course completion by user

• Two options: 'start free trial', 'access course materials'

Description of Experimented Change

• If user clicked free trial, they are given a prompt to indicate how much of their time can be devoted

• If time >= 5 hours / week, proceed to checkout

• If time < 5 hours / week, message popup indicating courses generally require higher time commitments

At this point, the student would have the option to continue enrolling in the free trial, or access the course materials for free instead. The screenshot below shows what the experiment looks like.

Udacity’s hope is that…:

1. By setting clearer expectations upfront, there is a reduction of frustrated users leaving the free trial

2. No significant reduction in user numbers continuing to the free trial and eventual completion.

3. Overall improvement in user experience