User Signup Segmentation

We’ve recently re-designed the signup flow for Butter to personalize the user experience and help us monitor how different groups use the product, so I’ve been thinking a lot about user activation in saas products lately.

In the process, I realized that the signup moment offers a rather low-effort chance to (1) ask questions to personalize the user experience and (2) ask questions to segment users for cohort analysis.

The latter can be very powerful when used to break down retention and activation reports or to prioritize and evaluate feedback, but my sense is that way too few startups make use of this simple method.

So, it felt like a good time to write down some observations and share my learnings about how signup segmentation can help personalize the user experience and monitor product/market fit.

Before we get to the nitty-gritty though, let’s take a walk down analogy road.

Suit or Hawaiian shirt?

In my younger years, I worked in a men’s clothing store.

Sales training 101 was to assess the person coming into the store by asking questions, and a typical exchange would go along the lines of:

Hello sir, what are you looking for today?

Well, I’m going to a birthday party, and I need a nice suit.

Got it. Are you looking for something only for finer occasions, or do you want something more casual?

Not really sure, casual, I guess.

Alright, and how much you’d like to spend?

Hmm, in the range of $400-$500.

Okay, let’s take your measurements and I’ll make some suggestions.

At this point, I have gathered enough data points to tailor his experience.

I know what he’s not looking for, so I can leave out my suggestion about the Hawaiian shirt we have on sale. I also know what he is looking for, and I can help him get his job done as fast and effectively as possible.

Had a woman come in looking for a present for her husband, I would have given her a completely different experience. I might even have suggested the Hawaiian shirt on sale!

Unfortunately, many product teams treat new users walking into their stores very differently and often leave them glancing around to figure things out for themselves.

For a company aiming to leverage their product as a growth driver — or in jargon, be product-led — not personalizing the user experience is simply a huge missed opportunity for activating users and driving growth.

But that’s just one side of the coin. The benefit of software and returning users is that you can continue to learn from them as they return to your store repeatedly and continuously improve whatever you are offering them.

Alright, so how should you welcome new users?

Well, it depends, but I think it starts with assessing who’s signing up by asking them questions, so you know whether they’re looking for a suit or a Hawaiian shirt.

3 strategies

There are a few ways of attacking this depending on the company type and stage, but from what I’ve seen, saas companies tend to use one or more of these three strategies:

1. Sales-driven: questions that characterize the value of an account
   We’ve all seen the Title, Company Name, and Company Size questions. This is a typical enterprise saas move to make things easy for sales and customer success. We see this with Salesforce and Marketo.
2. Activation-driven: questions to personalize or tailor the user experience
   We see it with questions like, what tools are you looking to integrate? What is your team currently working on? How do you plan to use x product? Notion, Airtable, and Coda use this information nicely by serving templates and guides based on your answers.
3. PMF-driven: questions to segment users into cohorts and monitor product/market fit
   Much fewer startups seem to ask for information that will help them monitor product/market fit across user groups. Superhuman and Mighty, for example, have a fascinating approach to this with 15-20 question pre-access surveys.

If you’re an early-stage saas startup, you likely have an emergent product strategy and mostly care about understanding which group of users indicate product/market fit, so you can learn from them and steer the ship towards it.

To get there, we need to know things like; who our users are, why they signed up, how they planned to use our product, how they actually use our product, what value it creates, who it creates value for, if it’s enough for them to stay, and if it’s enough for them to pay.

You can be very sophisticated about building behavior-driven cohorts from users’ actions to answer some of these questions, and I think ultimately, this is the best way to get data you can trust.

However, segmenting users on signup by asking questions that are use-case related and persona building to build question-driven cohorts, albeit not bulletproof, will get us 90% of the way with minimal effort.

At Butter, we do a mix of both. For our signup segmentation, we follow strategies 2 and 3 to both personalize the user experience and segment users into question-driven cohorts.

Let’s look at how we’ve gone about it and our data infrastructure that makes it all work.

Signing up to Butter

On signup, we simply ask users how they’re planning to use Butter and present them with 8 pre-defined options.

One for each of the use cases that we’re building for and one for each use case that we know is attracted to Butter but not our core target audience.

We also ask for their role and whether they’re planning to use it with colleagues or clients.

This data then gets added to the user object in the database, and we use Segment to distribute it to Intercom, Mixpanel, and Customer.io as a user profile property.

The distribution to all our tools is the important bit here. That’s how we use the answers from the signup questions to personalize the experience for each group of users; teachers, independent facilitators, agencies, etc.

How do we do that?

Remember the guy looking for a suit and the woman for a present? The same thing applies here.

• When we chat with a user on Intercom and see in the right-side context menu that they’re a consultant at an agency, then we understand what they’re trying to accomplish much faster.
• When we send out onboarding emails via customer.io, we can personalize the content for each of the cohorts to make it more useful. Large advisories, for example, are interested in very different features than an independent facilitator.
• When we analyze feedback and feature requests, we evaluate whether the user belongs to the core workshop group and, along with ten other metrics, how we should prioritize it.
• When we run surveys, we analyze the PMF scores, the areas of improvement, the benefits they’re getting, etc., by these signup answers.

I believe the personalization value alone should make it obvious that onboarding segmentation could have a real impact on the user experience, but from the angle of using this information to identify product/market fit, the most important destination for us is Mixpanel; our product analytics tool.

There, we can analyze user behavior and slice all the reports to answer things like:

• Acquisition: out of all signups, who are we attracting to Butter?
• Activation: which of these cohorts end up running real workshops, and which don’t?
• Frequency: What’s the average frequency of use for each of the cohorts?
• Retention: how well do we retain professional workshop facilitators compared with teachers?
• Feature adoption: is this feature being used by the intended cohort? Which features are most popular for each cohort?

For the Mixpanel nerds out there, we store these answers as user properties. This way, we can use it in the Breakdowns feature in all our reports.

For example, here’s how we use it in retention reports:

Amplitude allows for the exact same thing if that’s how you roll.

The bottom line is that this User Use Case property permeates every single piece of insight and feedback we look at, giving us a constant pulse on which group of users is showing the strongest indications of product/market fit.

Cool Thomas, I see the value, but personally, I hate filling out these qualification questions and I’m not convinced I want to make my users do that.

Well, I agree. Most of the time, I hate it too. And I’m definitely no expert, but I do believe the solution lies in designing a pleasant experience like much else, so I will give you my take on what I’ve found to be important, as well as a few examples I like.

Balancing insights and personalization

Whenever we are asked to do something, we ask ourselves: what’s in it for me?

The same is definitely true when you ask people questions on signup, and the last thing we want is people choosing random options to get it over with quickly. That will screw up your data, so you might as well leave it be if you don’t have the resources to design a delightful experience in the first place.

If you do, I think the key is to make users actually want to answer these questions, and the only way to do that is to show them that there’s something in it for them!

If I had asked someone in the clothing store how much they earn for a living, they would probably have been slightly pissed off.

So, what I’ve found to be important, is striking a balance between questions that will give you insight and questions that will actually improve the experience for the user.

Ideally, they go hand in hand, but we’re all too familiar with the question: what’s the size of your company?

Which seemingly serves no other purpose than making it easy for some customer success or salesperson to characterize the value of you as an account.

Baremetrics is a good example of one which clearly has its eyes on the lead-qualifying ball.

Funding stage… Really? Let me put in Series G.

On the other hand, Jira is a good example where the sole purpose feels like it’s to help me get set up.

To no surprise, Stripe does this nicely as well by making it optional and explaining exactly what they’ve got their eyes on.

If you’re intrigued by this and looking for some design inspiration, I will encourage you to try and sign up for autocode.com. I’m not sure whether their signup segmentation is effective, but it’s definitely delightful!

So yeah, that’s my Note for signup segmentation: ask good questions, make it delightful, and strike the right balance, so you don’t suggest a Hawaiian shirt to someone looking for a suit.