MaxDiff Feature Survey

Methodology

We had a list of 17 features that we wanted to get in front of Kit users to understand which feature they value the most and which feature they value the least.
We created a MaxDiff survey that tested 17 sets of 3 features for creators to rank to understand what’s important to them relative to other features.
We sent this survey out in-app to all active Kit users. Over 5 days, around 24,000 users saw the survey and 650 responded.

Rank ↑	Feature	Share of preference (%)	Most Valuable	Least Valuable	P-Value
1	Personalized sends	15.10%	790 (85.1%)	138 (14.9%)	< 0.001
2	Behavioral segmentation	11.10%	546 (70%)	234 (30%)	< 0.001
3	Custom subscriber scoring	9.40%	488 (63.3%)	283 (36.7%)	< 0.001
4	Subscriber journeys	8.50%	479 (60.1%)	318 (39.9%)	< 0.001
5	Quarterly business audits	7.60%	489 (56.4%)	378 (43.6%)	< 0.001
6	Breakdowns	6.30%	384 (51.5%)	362 (48.5%)	0.841
7	Custom dashboards	6.30%	411 (51.4%)	388 (48.6%)	0.832
8	Attribution insights	6.30%	387 (51.4%)	366 (48.6%)	0.888
9	A/B test email content	5.70%	395 (49.4%)	405 (50.6%)	1.447
10	Cohort analysis	5.40%	346 (48.2%)	372 (51.8%)	0.664
11	Dedicated account manager	5.20%	426 (47.2%)	476 (52.8%)	0.192
12	Period-over-period comparisons	4.70%	347 (45.5%)	415 (54.5%)	0.028
13	Enrich subscribers	2.70%	324 (37.8%)	533 (62.2%)	< 0.001
14	Baseline benchmarks	2.70%	310 (37.6%)	514 (62.4%)	< 0.001
15	Subscriber quality predictor	1.90%	287 (34.8%)	537 (65.2%)	< 0.001
16	Facebook custom audiences	1.30%	306 (32.6%)	634 (67.4%)	< 0.001
17	A/B test subject lines	0.00%	221 (27.5%)	583 (72.5%)	< 0.001

Personalized sends stands out by far for all plans and sizes—creators really want to be able to send emails that are personalized to the subscriber who is receiving it.
Got validation around the reporting features we’ve been thinking about, specifically behavioral segmentation, custom subscriber scoring, subscriber journeys.
Facebook custom audience is the least popular feature which might already point to a problem of our Creator Pro Plan.

A/B test up to 5 subject lines, subscriber predictor, and subscriber enrichment were surprisingly low.
- Could this be because the majority of creators who responded did not have a subscriber size where these features would be helpful?
Dedicated account managers did not carry the same value to creators as we think.
- Could this be because we don’t market the work that dedicated account managers do?
Quarterly business audits are more popular than dedicated account managers.
- Could this be a sign that they just want reassurance that what they are doing is correct?
- Could this be influenced by knowing that dedicated account managers typically come with a hefty price tag?